Monday, April 4, 2016

Node JS Rest API with Unit of Work and Generic Repository patterns

Introduction 


Why Node JS


Around 80% of application processing time involves with IO operations and not for computation in many of the applications. It indicates that CPU is idle 80% of the time. Therefore it is imperative that we should look into to reduce this bottleneck to improve application performance and take maximum out of CPU processing power. At the other hand, at high levels of concurrency (thousands of connections) your server needs to go for asynchronous non-blocking IO model. We cannot go ahead with creating multiple threads for each connection.  So this is where Node JS is excel with. Node JS single threaded asynchronous  event looping model handles thousands of connections concurrently and is good candidate for IO operation heavy applications.  

At the same time it is backed by 70k nodejs pre-built modules manged by NPM and it cuts down development time enormously.

Why Unit of Work

Unit of work pattern create singleton class to maintain single database context when communicating with a repository.It is really required to handle transaction management. When dealing with multiple data sources, developer can use this class to maintain single connection for each data sources.

Why Generic Repository

As you know in general, repository pattern allows us to:


  1. Making code more readable.
  2. Layering code.
  3. Improve unit testing
  4. Improve code re-usability. 
  5. Segregate responsibility to each repository class.


Further, creating repository class per each entity generates more redundant code. Generic Repository pattern eliminates that problem and provides single class for all the entities.

Workspace



Dependencies

package.json holds package information and dependencies of the rest API. Running "npm install" command at your workspace, which has package.json file inside, will install all the dependencies you required for running this rest API

package.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
{
  "name": "api",
  "version": "1.0.0",
  "description": "Node js rest API",
  "main": "index.js",
  "scripts": {
    "test": "node index.js"
  },
  "repository": {
    "type": "git",
    "url": "git+https://bitbucket.org/username/repositoryname.git"
  },
  "keywords": [
    "NodeJS",
    "Rest",
    "API"
  ],
  "author": "Author Name)",
  "license": "ISC",
  "homepage": "",
  "dependencies": {
    "bcrypt": "^0.8.5",
    "body-parser": "^1.14.2",
    "config": "^1.19.0",
    "cors": "^2.7.1",
    "crypto": "0.0.3",
    "express": "^4.13.4",
    "express-validator": "^2.19.1",
    "express-winston": "^1.2.0",
    "mongorepository": "~1.1.1",
    "mysql": "^2.10.2",
    "q": "^1.4.1",
    "repository": "^0.2.0",
    "underscore": "^1.8.3",
    "winston": "^2.2.0"
  }
}


Challenges

When it comes to implementing generic repository class, there are some challenges depending on data store that you selected. Because there are no out of the box modules available for creating repository for some of the data store. But if you take MongoDB, you can easily create Generic Repository class, as mongorepository module is available.

Mongodb Generic Repository class 

libs/mongoGenericRepository.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
var Repository = require('mongorepository').Repository;
 
 
function MongoGenericRepository(connection, entityName) {
    if (connection === undefined || connection === null) {
        throw new Error('Argument Expected `connection`, got nothing');
    }
 
    this.connection = connection;
    this.entityName = entityName;
    //create an instance
    this.repository = new Repository(this.connection, this.entityName);
}
  
var mongoGenericRepository = {
    getAll: function () {
        //todo: Implement get all
    },
 
    getById: function (id) {
        //todo: Implement get by id
    },
     
    getByField: function (field , fieldValue, columns) {
        //todo: Implement get by field
    },
 
    delete: function (id) {
        //todo: Implement delete
    },
      
    deleteByField: function (field , fieldValue) {
        //todo: Implement delete by field
    },
 
    updateById: function (data, id) {
       //todo: Implement update by id
    },
     
    updateByField: function (field , fieldValue, data) {
        //todo: update by field
    },
 
    save: function (data) {
        //todo: Implement save
    }
};
 
module.exports = mongoGenericRepository;


MySQL Generic Repository class

But to work with MySQL, there was no such module. But MySQL repository module is available with some limitation. So we can customize that module to come up with mySqlGenericRepository class.
Create the folder call node_modules_custom and move the repository folder at node_modules folder. Then edit repository class located at repository/lib/repository.js  as below to pass the entity name as a parameter to the constructor. And also edit repository/lib/util.js as below to return updated record instead of id. Then we need to move mysql module located at node_modules to node_modules_custom as custom repository module is looking for mysql module within node_modules_custom.




After all, we can define mySqlGenericRepository.js class at libs folder as below.

libs/mySqlGenericRepository.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
var Repository = require("../node_modules_custom/repository");
 
 
var mySQLGenericRepository = Repository.extend({
    getAll: function (limit) {
        var query;
        if (limit) {
            query = 'select * from ' + this.entity + ' limit ?';
        }
        else {
            var query = 'select * from ' + this.entity;
        }
         
        return this.query(query, [limit]);
    },
 
    getById: function (id) {
        var query;
        query = 'select * from '+ this.entity +' where id = ?';
        return this.query(query, [id]);
    },
     
    getByField: function (field , fieldValue, columns) {
        var query;
        if (columns) {
            query = 'select ?? from ?? where ?? = ?';
            return this.query(query, [columns , this.entity , field , fieldValue]);
        }
        else {
            query = 'select * from ?? where ?? = ?';
            return this.query(query, [this.entity, field , fieldValue]);
        }
    },
 
    delete: function (id) {
        var query;
        query = 'delete ' + this.entity + ' where id = ?';
        return this.update(query, [id]);
    },
     
    deleteByField: function (field , fieldValue) {
        var query;
        query = 'delete ' + this.entity + ' where ' + field +' = ?';
        return this.update(query, [fieldValue]);
    },
 
    updateById: function (data, id) {
        var query;
        query = 'UPDATE ' + this.entity + ' set ? WHERE id = ? ';
        return this.update(query, [data , id]);
    },
     
    updateByField: function (field , fieldValue, data) {
        var query;
        query = 'UPDATE ' + this.entity + ' set ? WHERE ' + field  +' = ? ';
        return this.update(query, [data , fieldValue]);
    },
 
    save: function (data) {
        var query;
        query = 'insert into ' + this.entity + ' set ?';
        return this.update(query , [data]);
    }
 
 
});
 
module.exports = mySQLGenericRepository;


Unit of Work class 

As indicated in the introduction, this class holds connection object and references to each repository classes. If you need to change data source from MySQL to MongoDB, you just need to create instances of repositories using mongoGenericRepository class by passing mongoConnection as a parameter. Then there won't be any business logic changes. So it act as Abstract Factory pattern. Config module has been used to retrieve database connection information.

libs/unitOfWork.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
var config = require('config');
var mysql = require('../node_modules_custom/mysql')
var MySqlGenericRepository = require('../libs/mySqlGenericRepository.js');
var MongoGenericRepository = require('../libs/mongoGenericRepository.js');
 
 
var mySqlDbConfig = config.get('COMMON.mySqlDbConfig');
var mySqlConnection = mysql.createConnection(mySqlDbConfig);
var mongoConnection = config.get('COMMON.mongoDbConfig');
 
var studentRepository = new MySqlGenericRepository(mySqlConnection, 'student');
 
module.exports =  { studentRepository : studentRepository} ;


default.json file should be there as below for holding config values.

config/default.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
{
  "COMMON": {
      "mySqlDbConfig": {
        "host": "localhost",
        "port": 3306,
        "database": "common",
        "user" : "username",
        "password" : "password"
   
      },
      "mongoDbConfig": "username:password@localhost/dbname",
      "log" : {
          "fileName" : "api-error-logs.log",
          "logsDirectory" : "logs"
        }     
    }
}


End Points

Now we have middleware implemented with Unit of Work pattern and Generic repository pattern. Lets we look at how we can implement end points using node express module.

Handlers

Handlers are the classes where you can implement your business logic on the entity data returned from the data store or the entity data that you are going save. Sample student handler class is as follows.

handlers/studentHandler.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
var unitOfWork = require('../libs/unitOfWork.js');
 
var studentHandler = function() {
    this.createStudent = handleCreateStudentRequest;
    this.getStudent = handleGetStudentRequest;
    this.getStudents = handleGetStudentsRequest;
    this.updateStudent = handleUpdateStudentRequest;
    this.deleteStudent = handleDeleteStudentRequest;
    this.getStudentsByAge = handleGetStudentsByAgeRequest;
};
 
function handleCreateStudentRequest(req, res, next) {
 
    var student = req.body;  // get Student object as json object (comes from the request)
    // save the Student and check for errors
    unitOfWork.studentRepository.save(student).then(function (result) {
        res.json({ message: 'Student created!', id: result.insertId });
    }, function (err) {
        res.status(500);
        res.send(err);
        return next(new Error(err));
    });
 
}
 
function handleGetStudentsRequest(req, res, next) {
    unitOfWork.studentRepository.getAll().then(function (result) {
        res.json(result);
    }, function (err) {
        res.status(500);
        res.send(err);
        return next(new Error(err));
    });
}
 
function handleGetStudentRequest(req, res, next) {
    unitOfWork.studentRepository.getById(req.params.id).then(function (result) {
        res.json(result);
    }, function (err) {
        res.status(500);
        res.send(err);
        return next(new Error(err));
    });
}
 
function handleGetStudentsByAgeRequest(req, res, next) {
    unitOfWork.studentRepository.getByField("age" , req.params.statusId, ["id" , "name"] ).then(function (result) {
        res.json(result);
    }, function (err) {
        res.status(500);
        res.send(err);
        return next(new Error(err));
    });
}
 
 
function handleUpdateStudentRequest(req, res, next) {
}
 
function handleDeleteStudentRequest(req, res, next) {
}
module.exports = studentHandler;

Routes

 route.js file contains all the routes defined and route handlers mapped for the each route.

routes.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
function setup(router, handlers) {
 
    // ROUTES FOR API
    // =============================================================================
    router.post('/students', handlers.student.createStudent);
    router.get('/students', handlers.student.getStudents);
    router.get('/students/:id', handlers.student.getStudent);
    router.get('/students/age/:ageInt', handlers.student.getStudentsByAge);
    router.put('/students/:id', handlers.student.updateStudent);
    router.delete('/students/:id', handlers.student.deleteStudent);
    
}
 
exports.setup = setup;


Server

Server.js has been used to define all the Node JS http server specific information for running the node express server.

server.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
// server.js
 
// BASE SETUP
// =============================================================================
 
// call the packages we need
var express    = require('express');        // call express
var app        = express();                 // define our app using express
var studentHandler = require('./handlers/studentHandler');
var routes = require('./routes');
var bodyParser = require('body-parser');
var cors = require('cors');
var expressValidator = require('express-validator');
var expressWinston = require('express-winston');
var winston = require('winston');
var cfg = require("config");
 
 
//var port = process.env.PORT || 8090;        // set our port
var port = 8081;        // set our port
var router = express.Router();              // get an instance of the express Router
 
// middleware to use for all requests
router.use(function (req, res, next) {
    // do logging
    console.log('Something is happening.');
    next(); // make sure we go to the next routes and don't stop here
});
 
// test route to make sure everything is working (accessed at GET http://localhost:8080/api)
router.get('/', function(req, res) {
    res.json({ message: 'Welcome to node js rest service!' });  
});
 
 
// configure app to use bodyParser()
// this will let us get the data from a POST
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
app.use(expressValidator());
app.use(express.Router());
app.use(cors());
 
 
// REGISTER BASE ROUTES -------------------------------
// all of our routes will be prefixed with /common
app.use('/restapi', router);
 
//Define route handlers
var handlers = {
  student: new studentHandler()//,
  //user: new userHandler()
};
 
function start() {
  routes.setup(router, handlers);
   
  // express-winston errorLogger makes sense AFTER the router.
  app.use(expressWinston.errorLogger({
    transports: [
      new winston.transports.File({
          level: 'error',
          filename: __dirname + '/' + cfg.get("COMMON.log.logsDirectory") + '/' + cfg.get("COMMON.log.fileName"),
          handleExceptions: true,
          json: true,
          maxsize: 5242880, //5MB
          maxFiles: 5,
          colorize: false,
          timestamp:true
      }),
      new winston.transports.Console({
        json: true,
        colorize: true
      })
    ]
  }));
 
  app.listen(port);
  console.log("Express server listening on port %d in %s mode", port, app.settings.env);
}
 
 
exports.start = start;


Index

Finally index.js has been used to run the Node JS http server.
command: node index.js

index.js
var server = require('./server'); server.start(); console.log("Successfully started web server. Waiting for incoming connections...");

Testing

http://localhost:8081/restapi can be used to make sure that the rest api up and running or not.
http://localhost:8081/restapi/students can be used to get all the students, if you have properly setup the student entity in the database.

Git hub


Conclusion 


1. With this model implemented in your API, you will be able to get lot of advantages.
           It cuts down development time enormously by eliminating redundant code. how?
         
           If you wanted introduce new rest route for existing repository, you just need to:
           I)    Add handler method to related handler class and apply business logic on it.
           II)   Add rest route at route.js and map handler method.

           If you want to introduce new rest route with new repository, you just need to:
           I)   Add handler class and methods as you can see in studentHandler.js above
           II)  Import it at server.js and create new instance of it
                ex: var userHandler = require('./handlers/userHandler');      
                      var handlers = {
                      student: new studentHandler(),
                      user: new userHandler() };

           III)  Add new reset routes at route.js and map relevant handler methods for each route.
2. You can mock repository and implement unit testing on handler class without having any trouble.
3. Abstract Factory pattern in built
4. New team member, who is new to Node JS, can easily adopt to model as it's high level of abstraction.
Posted by at

2 comments:

  1. AnonymousSeptember 8, 2016 at 6:34 PM

    This comment has been removed by the author.

    ReplyDelete
  2. seravina danniellaOctober 31, 2017 at 11:15 PM

    Your blog has given me that thing which I never expect to get from all over the websites. Its very easy to understand and very helpful. Nice post guys!


    Melbourne App Developer

    ReplyDelete

Subscribe to: Post Comments (Atom)