Data import concepts
Haplo applications are highly configurable, so the schema of each application will be unique. The framework handles this by providing generic ways of describing and mapping data, along with a user interface to describe the data model in your application.
Key concepts are described below.
The Model describes the schema and data structures within the Haplo application.
Application developers configure their plugins to expose relevant parts of the object store schema and plugin specific data structures.
Each model has a name, for example
haplo:person is the standard model describing a person and their associated data. The model is built up by the installed plugins, each providing a description of their model and the mechanism for importing the data.
To explore a Model, start by choosing Data import from the main menu.
The Model contains one or more Destinations which describe the data model. Destinations are roughly equivalent to tables in a relational database.
For example, the user sync plugin provides Destinations for the user account, their ‘profile’ (a store object which describes the person), and other plugins extend this Model with other Destinations, for example, the user’s project.
Destinations can be used to create new objects and other data in your application, or update existing data.
A Destination contains one or more Names, which describe the values that can be stored in that Destination. Names are roughly equivalent to the columns in a relational database, although they tend to be multivalue.
The Control is a JSON file which describes a data import. This includes:
- the name of the Model that the import is using,
- the format of the input files,
- data transformations and mappings,
- Instructions describing how to use the data from the input files.
The Control file contains Instructions on how to interpret the data, with limited support for conditionals and looping.
These Instructions describe how to take the data from the input files, transform them, and apply them to Names within the Destinations.
A Batch is is made up of a collection of input files and a Control file which are imported into an application.
Batches may be a one-off import, a regular import to synchronise data in a user sync, or represent a single transaction when a REST API is called.
The input files in a Batch contain Records, which are distinct collections of data representing one ‘thing’. This can range in complexity from a single row from a database table, to a complex JSON structure describing a person and their project.