Create Node.js and csvtojson Lambda Layer for AWS Lambda Custom Runtime

This post will cover how to use Node.js and csvtojson in AWS Lambda Custom Runtime which can be useful for use cases wherein CSV to JSON conversion is required as a part of a Lambda function in a serverless architecture. 

Have a quick look at Using JQ in AWS Lambda Custom Runtime via AWS Lambda Layer for a quick reference to understanding how AWS Lambda Custom Runtime is bootstrapped and how AWS Layers works.

Creating Node.js Lambda Layer

Setup for Node.js for AWS Lambda Custom Runtime is actually quite easy, all that is needed is aws-lambda-custom-node-runtime (v1.0.2). Install NPM if not already done, and run the following command:

aws-lambda-custom-node-runtime 11.3.0

The above command will generate a directory named node-v11.3.0-linux-x64 (at the path where the command is run) with all the necessary files required to run Node.js.

Now zip the entire node-v11.3.0-linux-x64 directory using the following command

zip -r node-v11.3.0-linux-x64.zip node-v11.3.0-linux-x64

The zip archive, i.e node-v11.3.0-linux-x64.zip, is our Node.js Lambda layer.

Creating CSVTOJSON Lambda Layer

To create a CSVTOJSON bundle, run the following command

npm install csvtojson --save

This will create a node_modules directory at the path at which the command is executed. Now, similar to what was done for building the Node.js layer, run the following command to build the CSVTOJSON Lambda Layer

zip -r node_dependencies.zip node_modules

The node_dependencies.zip is now our CSVTOJSON Lambda layer. 

Separating Lambda layers as Node.js layer and node dependencies layer helps because doing so allows the layers to be reused across multiple Lambda functions. Doing this, the Node.js Lambda layer can be reused across multiple Lambda functions and each Lambda function can have its own set of node dependencies. But it all boils down to how AWS Lambdas are used in your serverless architecture and there might be differences in best practices. 

Using Node.js and CSVTOJSON Lambda Layers

Now, the built Lambda Layers can be uploaded to the AWS Lambda Layers section, and a test-run in the following way can be done to verify whether they are working fine:

(Visit Using JQ in AWS Lambda Custom Runtime via AWS Lambda Layer for more details on what the function handler)

function handler () {
    EVENT_DATA=$1
    
    cd /opt
    ./node-v11.3.0-linux-x64/bin/node --version
    ./node-v11.3.0-linux-x64/bin/node node_modules/csvtojson/bin/csvtojson version
}

When the above handler is triggered, the Node and CSVTOJSON versions can be expected in the success output.