1 | # LoopBack DataSource and Connector Guide
|
2 |
|
3 | ## Overview
|
4 | LoopBack is centered around models, which represent data and behaviors. The
|
5 | concept of `DataSource` is introduced to encapsulate business logic to exchange
|
6 | data between models and various data sources. Data sources are typically
|
7 | databases that provide create, retrieve, update, and delete (CRUD) functions.
|
8 | LoopBack also generalize other backend services, such as REST APIs, SOAP Web
|
9 | Services, and Storage Services, as data sources.
|
10 |
|
11 | Data sources are backed by connectors which implement the data exchange logic
|
12 | using database drivers or other client APIs. In general, connectors are not used
|
13 | directly by application code. The `DataSource` class provides APIs to configure
|
14 | the underlying connector and exposes functions via `DataSource` or model classes.
|
15 |
|
16 | ![model-datasource-connector](datasource-connector.png "LoopBack Model, DataSource, and Connector")
|
17 |
|
18 | The diagram above illustrates the relationship between LoopBack `Model`,
|
19 | `DataSource`, and `Connector`.
|
20 |
|
21 | 1. Define the Model using [LoopBack Definition Language (LDL)](definition-language.md).
|
22 | Now we have a model definition in plain JSON or JavaScript object.
|
23 |
|
24 | 2. Create an instance of ModelBuilder or DataSource. Please note that
|
25 | DataSource extends from ModelBuilder. ModelBuilder is responsible for compiling
|
26 | model definitions to JavaScript constructors representing model classes.
|
27 | DataSource inherits that function from ModelBuilder. In addition, DataSource
|
28 | adds behaviors to model classes by mixing in methods from the DataAccessObject
|
29 | into the model class.
|
30 |
|
31 | 3. Use ModelBuilder or DataSource to build a JavaScript constructor (i.e,
|
32 | the model class) from the model definition. Model classes built from ModelBuilder
|
33 | can be later attached to a DataSource to receive the mixin of data access
|
34 | functions.
|
35 |
|
36 | 4. As part of step 2, DataSource initializes the underlying Connector with
|
37 | a settings object which provides configurations to the connector instance.
|
38 | Connector collaborates with DataSource to define the functions as
|
39 | DataAccessObject to be mixed into the model class. The DataAccessObject
|
40 | consists of a list of static and prototype methods. It can be CRUD operations
|
41 | or other specific functions depending on the connector's capabilities.
|
42 |
|
43 | ## LoopBack DataSource
|
44 |
|
45 | DataSource is the unified interface for LoopBack applications to integrate with
|
46 | backend systems. It's a factory for data access logic around model classes. With
|
47 | the ability to plug in various connectors, DataSource provides the necessary
|
48 | abstraction to interact with databases or services to decouple the business
|
49 | logic from plumbing technologies.
|
50 |
|
51 | ### Creating a DataSource
|
52 |
|
53 | The `DataSource` constructor is available from `loopback-datasource-juggler` module:
|
54 |
|
55 | var DataSource = require('loopback-datasource-juggler').DataSource;
|
56 |
|
57 | `DataSource` constructor accepts two arguments:
|
58 | - connector: The name or instance of the connector module
|
59 | - settings: An object of properties to configure the connector
|
60 |
|
61 |
|
62 | ```
|
63 | var dataSource = new DataSource({
|
64 | connector: require('loopback-connector-mongodb'),
|
65 | host: 'localhost',
|
66 | port: 27017,
|
67 | database: 'mydb'
|
68 | });
|
69 | ```
|
70 |
|
71 | #### connector
|
72 |
|
73 | The `connector` argument passed the DataSource constructor can be one of the following:
|
74 |
|
75 | * The connector module from `require(connectorName)`
|
76 | * The full name of the connector module, such as 'loopback-connector-oracle'
|
77 | * The short name of the connector module, such as 'oracle', which will be converted
|
78 | to 'loopback-connector-<shortName>'
|
79 | * A local module under ./connectors/<connectorName> folder
|
80 |
|
81 | ```
|
82 | var ds1 = new DataSource('memory');
|
83 | var ds2 = new DataSource('loopback-connector-mongodb');
|
84 | var ds3 = new DataSource(require('loopback-connector-oracle'));
|
85 | ```
|
86 |
|
87 | **Note**: LoopBack provides a built-in connector named as `memory` to use in-memory
|
88 | store for CRUD operations.
|
89 |
|
90 | #### settings
|
91 |
|
92 | The `settings` argument configures the connector. Settings object format and defaults
|
93 | depends on specific connector, but common fields are:
|
94 |
|
95 | * `host`: Database host
|
96 | * `port`: Database port
|
97 | * `username`: Username to connect to database
|
98 | * `password`: Password to connect to database
|
99 | * `database`: Database name
|
100 | * `debug`: Turn on verbose mode to debug db queries and lifecycle
|
101 |
|
102 | For connector-specific settings refer to connector's readme file.
|
103 |
|
104 | ## Creating a Model
|
105 |
|
106 | `DataSource` extends from `ModelBuilder`, which is a factory for plain model
|
107 | classes that only have properties. `DataSource` connected with specific databases
|
108 | or other backend systems using `Connector`.
|
109 |
|
110 | var DataSource = require('loopback-datasource-juggler').DataSource;
|
111 | var ds = new DataSource('memory');
|
112 |
|
113 | var User = ds.define('User', {
|
114 | name: String,
|
115 | bio: String,
|
116 | approved: Boolean,
|
117 | joinedAt: Date,
|
118 | age: Number
|
119 | });
|
120 |
|
121 | All model classes within single data source shares same connector type and one
|
122 | database connection or connection pool. But it's possible to use more than one
|
123 | data source to connect with different databases.
|
124 |
|
125 | Alternatively, a plain model constructor created from `ModelBuilder` can be
|
126 | attached a `DataSource`.
|
127 |
|
128 | var ModelBuilder = require('loopback-datasource-juggler').ModelBuilder;
|
129 | var builder = new ModelBuilder();
|
130 |
|
131 | var User = builder.define('User', {
|
132 | name: String,
|
133 | bio: String,
|
134 | approved: Boolean,
|
135 | joinedAt: Date,
|
136 | age: Number
|
137 | });
|
138 |
|
139 | var DataSource = require('loopback-datasource-juggler').DataSource;
|
140 | var ds = new DataSource('memory');
|
141 |
|
142 | User.attachTo(ds); // The CRUD methods will be mixed into the User constructor
|
143 |
|
144 |
|
145 | ## More DataSource Features
|
146 |
|
147 | In addition to data access functions mixed into the model class, `DataSource`
|
148 | also provides APIs to interact with the underlying backend system.
|
149 |
|
150 | ### Discovering model definitions from the data source
|
151 |
|
152 | Some connectors provide discovery capability so that we can use DataSource to
|
153 | discover model definitions from existing database schema.
|
154 |
|
155 | The following APIs allow UI or code to discover database schema definitions that
|
156 | can be used to build LoopBack models.
|
157 |
|
158 | // List database tables and/or views
|
159 | ds.discoverModelDefinitions({views: true, limit: 20}, cb);
|
160 |
|
161 | // List database columns for a given table/view
|
162 | ds.discoverModelProperties('PRODUCT', cb);
|
163 | ds.discoverModelProperties('INVENTORY_VIEW', {owner: 'STRONGLOOP'}, cb);
|
164 |
|
165 | // List primary keys for a given table
|
166 | ds.discoverPrimaryKeys('INVENTORY', cb);
|
167 |
|
168 | // List foreign keys for a given table
|
169 | ds.discoverForeignKeys('INVENTORY', cb);
|
170 |
|
171 | // List foreign keys that reference the primary key of the given table
|
172 | ds.discoverExportedForeignKeys('PRODUCT', cb);
|
173 |
|
174 | // Create a model definition by discovering the given table
|
175 | ds.discoverSchema(table, {owner: 'STRONGLOOP'}, cb);
|
176 |
|
177 | You can also discover and build model classes in one shot:
|
178 |
|
179 | // Start with INVENTORY table and follow the primary/foreign relationships to discover associated tables
|
180 | ds.discoverAndBuildModels('INVENTORY', {visited: {}, relations: true}, function (err, models) {
|
181 |
|
182 | // Now we have an object of models keyed by the model name
|
183 | // Find the 1st record for Inventory
|
184 | models.Inventory.findOne({}, function (err, inv) {
|
185 | if(err) {
|
186 | console.error(err);
|
187 | return;
|
188 | }
|
189 | console.log("\nInventory: ", inv);
|
190 |
|
191 | // Follow the product relation to get information about the product
|
192 | inv.product(function (err, prod) {
|
193 | console.log("\nProduct: ", prod);
|
194 | console.log("\n ------------- ");
|
195 | });
|
196 | });
|
197 | });
|
198 |
|
199 | In addition to the asynchronous APIs, `DataSource` also provides the synchronous
|
200 | ones. Please refer to the DataSource API references.
|
201 |
|
202 | ### Synchronizing model definitions against the data source
|
203 |
|
204 | DataSource instance have two methods for updating db structure: `automigrate` and
|
205 | `autoupdate` for relational databases.
|
206 |
|
207 | The `automigrate` method drop table (if exists) and create it again, `autoupdate`
|
208 | method generates ALTER TABLE query. Both method accepts an optional array of
|
209 | model names and a callback function to be called when migration/update done. If
|
210 | the `models` argument is not present, all models are checked.
|
211 |
|
212 | In the following example, we create first version of the CustomerTest model, use
|
213 | `automigrate` to create the database table, redefine the model with second
|
214 | version, and use `autoupdate` to alter the database table.
|
215 |
|
216 | // Create the 1st version of 'CustomerTest'
|
217 | ds.createModel(schema_v1.name, schema_v1.properties, schema_v1.options);
|
218 |
|
219 | // Create DB table for the model
|
220 | ds.automigrate(schema_v1.name, function () {
|
221 |
|
222 | // Discover the model properties from DB table
|
223 | ds.discoverModelProperties('CUSTOMER_TEST', function (err, props) {
|
224 | console.log(props);
|
225 |
|
226 | // Redefine the 2nd version of 'CustomerTest'
|
227 | ds.createModel(schema_v2.name, schema_v2.properties, schema_v2.options);
|
228 |
|
229 | // Alter DB table
|
230 | ds.autoupdate(schema_v2.name, function (err, result) {
|
231 | ds.discoverModelProperties('CUSTOMER_TEST', function (err, props) {
|
232 | console.log(props);
|
233 | });
|
234 | });
|
235 | });
|
236 | });
|
237 |
|
238 |
|
239 | To check if any db changes required use `isActual` method. It accepts
|
240 | and a `callback` argument, which receive boolean value depending on db state:
|
241 |
|
242 | - false if db structure outdated
|
243 | - true when dataSource and db is in sync
|
244 |
|
245 |
|
246 | dataSource.isActual(models, function(err, actual) {
|
247 | if (!actual) {
|
248 | dataSource.autoupdate(models, function(err, result) {
|
249 | ...
|
250 | });
|
251 | }
|
252 | });
|
253 |
|
254 | ## LoopBack Connector
|
255 |
|
256 | Connectors implement the logic to integrate with specific backend systems, such
|
257 | as databases or REST services.
|
258 |
|
259 | ### LoopBack Connector Modules
|
260 |
|
261 | | Type | Package Name |
|
262 | | --------- |:--------------------------------------------------------------------------------------:|
|
263 | | Memory | [Built-in](https://github.com/strongloop/loopback-datasource-juggler) |
|
264 | | MongoDB | [loopback-connector-mongodb](https://github.com/strongloop/loopback-connector-mongodb) |
|
265 | | Oracle | [loopback-connector-oracle](https://github.com/strongloop/loopback-connector-oracle) |
|
266 | | REST | [loopback-connector-rest](https://github.com/strongloop/loopback-connector-rest) |
|
267 | | MySQL | [loopback-connector-mysql](https://github.com/strongloop/loopback-connector-mysql) |
|
268 |
|
269 |
|
270 | ### Interaction between DataSource and Connector
|
271 |
|
272 | #### Initializing connector
|
273 |
|
274 | The connector module can export an `initialize` function to be called by the
|
275 | owning DataSource instance.
|
276 |
|
277 | exports.initialize = function (dataSource, postInit) {
|
278 |
|
279 | var settings = dataSource.settings || {}; // The settings is passed in from the dataSource
|
280 |
|
281 | var connector = new MyConnector(settings); // Construct the connector instance
|
282 | dataSource.connector = connector; // Attach connector to dataSource
|
283 | connector.dataSource = dataSource; // Hold a reference to dataSource
|
284 | ...
|
285 | };
|
286 |
|
287 | The DataSource calls the `initialize` method with itself and an optional `postInit`
|
288 | callback function. The connector receives the settings from the `dataSource`
|
289 | argument and use it to configure connections to backend systems.
|
290 |
|
291 | Please note connector and dataSource set up a reference to each other.
|
292 |
|
293 | Upon initialization, the connector might connect to database automatically.
|
294 | Once connection established dataSource object emit 'connected' event, and set
|
295 | `connected` flag to true, but it is not necessary to wait for 'connected' event
|
296 | because all queries cached and executed when dataSource emit 'connected' event.
|
297 |
|
298 | To disconnect from database server call `dataSource.disconnect` method. This
|
299 | call is forwarded to the connector if the connector have ability to connect/disconnect.
|
300 |
|
301 | #### Accessing data/services
|
302 |
|
303 | The connector instance can have an optional property named as DataAccessObject
|
304 | that provides static and prototype methods to be mixed into the model constructor.
|
305 | DataSource has a built-in DataAccessObject to support CRUD operations. The
|
306 | connector can choose to use the CRUD DataAccessObject or define its own.
|
307 |
|
308 | When a method is invoked from the model class or instance, it's delegated to the
|
309 | DataAccessObject which is backed by the connector.
|
310 |
|
311 | For example,
|
312 |
|
313 | User.create() --> dataSource.connector.create() --> Oracle.prototype.create()
|
314 |
|
315 | ## Building your own connectors
|
316 |
|
317 | LoopBack connectors provide access to backend systems including databases, REST
|
318 | APIs and other services. Connectors are not used directly by application code.
|
319 | We create a DataSource to interact with the connector.
|
320 |
|
321 | For example,
|
322 |
|
323 | var DataSource = require('loopback-datasource-juggler').DataSource;
|
324 | var oracleConnector = require('loopback-connector-oracle');
|
325 |
|
326 | var ds = new DataSource(oracleConnector, {
|
327 | host : 'localhost',
|
328 | database : 'XE',
|
329 | username : 'username',
|
330 | password : 'password',
|
331 | debug : true
|
332 | });
|
333 |
|
334 |
|
335 | ### Implementing a generic connector
|
336 |
|
337 | A connector module can implement the following methods to interact with the data
|
338 | source.
|
339 |
|
340 | exports.initialize = function (dataSource, postInit) {
|
341 |
|
342 | var settings = dataSource.settings || {}; // The settings is passed in from the dataSource
|
343 |
|
344 | var connector = new MyConnector(settings); // Construct the connector instance
|
345 | dataSource.connector = connector; // Attach connector to dataSource
|
346 | connector.dataSource = dataSource; // Hold a reference to dataSource
|
347 |
|
348 | /**
|
349 | * Connector instance can have an optional property named as DataAccessObject that provides
|
350 | * static and prototype methods to be mixed into the model constructor. The property can be defined
|
351 | * on the prototype.
|
352 | */
|
353 | connector.DataAccessObject = function() {};
|
354 |
|
355 | /**
|
356 | * Connector instance can have an optional function to be called to handle data model definitions.
|
357 | * The function can be defined on the prototype too.
|
358 | * @param model The name of the model
|
359 | * @param properties An object for property definitions keyed by propery names
|
360 | * @param settings An object for the model settings
|
361 | */
|
362 | connector.define = function(model, properties, settings) {
|
363 | ...
|
364 | };
|
365 |
|
366 | connector.connect(..., postInit); // Run some async code for initialization
|
367 | // process.nextTick(postInit);
|
368 | }
|
369 |
|
370 | Another way is to directly export the connection function which takes a settings object.
|
371 |
|
372 | module.exports = function(settings) {
|
373 | ...
|
374 | }
|
375 |
|
376 | ### Implementing a CRUD connector
|
377 |
|
378 | To support CRUD operations for a model class that is attached to the
|
379 | dataSource/connector, the connector needs to provide the following functions:
|
380 |
|
381 | /**
|
382 | * Create a new model instance
|
383 | */
|
384 | CRUDConnector.prototype.create = function (model, data, callback) {
|
385 | };
|
386 |
|
387 | /**
|
388 | * Save a model instance
|
389 | */
|
390 | CRUDConnector.prototype.save = function (model, data, callback) {
|
391 | };
|
392 |
|
393 | /**
|
394 | * Check if a model instance exists by id
|
395 | */
|
396 | CRUDConnector.prototype.exists = function (model, id, callback) {
|
397 | };
|
398 |
|
399 | /**
|
400 | * Find a model instance by id
|
401 | */
|
402 | CRUDConnector.prototype.find = function find(model, id, callback) {
|
403 | };
|
404 |
|
405 | /**
|
406 | * Update a model instance or create a new model instance if it doesn't exist
|
407 | */
|
408 | CRUDConnector.prototype.updateOrCreate = function updateOrCreate(model, data, callback) {
|
409 | };
|
410 |
|
411 | /**
|
412 | * Delete a model instance by id
|
413 | */
|
414 | CRUDConnector.prototype.destroy = function destroy(model, id, callback) {
|
415 | };
|
416 |
|
417 | /**
|
418 | * Query model instances by the filter
|
419 | */
|
420 | CRUDConnector.prototype.all = function all(model, filter, callback) {
|
421 | };
|
422 |
|
423 | /**
|
424 | * Delete all model instances
|
425 | */
|
426 | CRUDConnector.prototype.destroyAll = function destroyAll(model, callback) {
|
427 | };
|
428 |
|
429 | /**
|
430 | * Count the model instances by the where criteria
|
431 | */
|
432 | CRUDConnector.prototype.count = function count(model, callback, where) {
|
433 | };
|
434 |
|
435 | /**
|
436 | * Update the attributes for a model instance by id
|
437 | */
|
438 | CRUDConnector.prototype.updateAttributes = function updateAttrs(model, id, data, callback) {
|
439 | };
|
440 |
|
441 |
|
442 |
|
443 |
|
444 |
|