Developer

The server-side data import framework API has two distinct uses.

Extend data import framework

The data import framework Models are defined by plugins around a key object, such as a user or person. One plugin will declare the model exists, and then other plugins extend the model to describe the data they manage.

To extend the capabilities of the data import framework or perform transformations during the import process, plugins can provide custom logic and value filters for use in control files.

Readers can be implemented to import new file formats.

Use framework for importing data

Other plugins can use the data import framework for importing data, using the same server-side API as the batch import, User sync and REST API. This enables custom data import processes to be implemented with very small amounts of code.

Additional concepts

As well as the core data import framework concepts, the API uses these terms:

Reader

Input files are converted into Records by a Reader. This is implemented as a function which calls an iterator with every row from an input file.

Target

When a record is processed, each of the Destinations creates Targets. These are the actual objects where the data is stored, which might be JavaScript objects, store objects, and so on. The Model knows how to load, create and save data to the various databases.

Transformation

Transformations control ingesting data from a single record, which is transformed via Destinations into Targets. These Targets may then be committed to save them into the databases.