In AWS Lambdas, if you require a file like ../../../../lib
you're out of luck. Serverless will not resolve that file correctly when creating the zip. This happens when the serverless.yml
files are buried deep in your project directories. Node modules dependencies are handled correctly. (thats good).
To add lib
, models
, etc. correctly, I'm trying an approach using environment variables.
Here's our API directory structure. All of our shared files, like lib
and models
are at the root level. Many lambdas may need access to the same mode or library.
We have hundreds of lambdas and have run into issues when having so many in one serverless.yml
file. This approach allows to have one project, The API, with many serverless.yml
dividing it all up into manageable services that can share code.
projectRoot
|__ lib // shared helpers
|__ lib1.js
|__ lib2.js
|__ ...
|__ models // shared models
|__ service1Model.js
|__ service2Model.js
|__ ...
|__ node_modules
|__ test # shared test helpers
|__ ...
|__ package.json
|__ services // lambdas that act like controllers
|__service1
|__ serverless.yml
|__ tests
|__ create_test.js
|__ rpc1_test.js
|__ ...
|__ create.js
|__ rpc1.js
|__ ...
|__service2
|__ serverless.yml
|__ tests
|__ read_test.js
|__ rpc_test.js
|__ ...
|__read.js
|__rpc.js
|__ ...
|__ ...
Keep lambda dependencies small. Only include files that are needed for the function. Explicitly require only the models needed. In our case, all the libs are used. NOTE how we are setting environment variables.
# serverless.yml
service: service1
provider: aws
functions:
create:
handler: create.handler
package:
exclude:
include:
- create.js
- ../../models/serviceModel1.js
- ../../lib/**
environment:
LIB_PATH: lib
MODELS_PATH: models
An example of what our lambda might look like. The important section is how we include the dependencies and how the path name is resolved using the environment variable found in process.env
.
const lib = require(require('path').resolve(process.env.LIB_PATH, 'lib1'));
const model = require(require('path').resolve(process.env.MODELS_PATH, 'service1Model'));
module.exports.handler(event, context, callback) => {
...
Setting the environment variables for the local working project directory:
#!/bin/bash
export LIB_PATH="$(pwd)/lib";
export MODELS_PATH="$(pwd)/models";
Of course, these environment variables will need to be updated when changing projects.
If you're using something like ava
for TDD, you can export the env vars when you execute the test:
source scripts/set_env_vars.sh && ava -svw services/service1/test/foo_unit.js"
A second option, using the environment variable approach, is to have the environment variable represent the structure at a higher level.
For example, the base of the project would contain a src
directory that then contains the lib
and models
directories. This would only require defining one environment variable to represent the code path.
projectRoot
|__ src
|__ lib // shared helpers
|__ lib1.js
|__ lib2.js
|__ ...
|__ models // shared models
|__ service1Model.js
|__ service2Model.js
|__ ...
|__ node_modules
|__ test # shared test helpers
|__ ...
|__ package.json
|__ services // lambdas that act like controllers
|__service1
|__ serverless.yml
|__ tests
|__ create_test.js
|__ rpc1_test.js
|__ ...
|__ create.js
|__ rpc1.js
|__ ...
|__service2
|__ serverless.yml
|__ tests
|__ read_test.js
|__ rpc_test.js
|__ ...
|__read.js
|__rpc.js
|__ ...
|__ ...
After 24 hours of thinking of this, simply use the environment variable to represent the project base.
export BASE_PATH=/path/to/project
. Leave models
and lib
at the root. In serverless.yml
, set the Lambda environment variable to ./
.
Then the require becomes:
const lib = require(require('path').resolve(process.env.BASE_PATH, 'lib/lib1'));
const model = require(require('path').resolve(process.env.BASE_PATH, 'model/service1Model'));
I love talking myself through these problems. ❤️
For my future self, we ended up relying heavily on NPM, node package manager. For directories that encapsulated functionality, we used a index.js and package.json files. Yarn is used to manage dependencies, using cached modules unless changes are made.