Serverless Webpack in Lambda Simplified 101

AWS Lambda is a Serverless, Event-Driven Compute Service offered by Amazon as part of Amazon Web Services. It is a computing service that runs code in response to events and maintains the computing resources required by that code automatically.

Lambda functions can be built using a Serverless Webpack plugin. If you want to use the most recent Javascript version with Babel, use custom resource loaders, optimize your packaged functions individually, and more, this plugin is for you.

This blog explains the different aspects of Serverless Webpack in Lambda. In addition to that, it describes AWS Lambda as well.

Table Of Contents

• What is AWS Lambda?
  • Key Features of AWS Lambda
• Understanding Serverless Webpack
  • Serverless Webpack Aspects: Install
  • Serverless Webpack Aspects: Configure
  • Serverless Webpack Aspects: Node Modules
  • Serverless Webpack Aspects: Support for Docker Images as Custom Runtimes
  • Serverless Webpack Aspects: Usage
  • Serverless Webpack Aspects: Provider Support
  • Serverless Webpack Aspects: Plugin Support
• Conclusion

What is AWS Lambda?

AWS Lambda is a serverless compute service that is triggered by events that allow you to run code for virtually any application or backend service without having to provision or manage servers. You can call Lambda from more than 200 AWS services and SaaS apps, and you only pay for what you use.

The term “serverless” computing refers to the fact that you don’t need to run these functions on your servers. AWS Lambda is a fully managed service that handles all of your infrastructure requirements.

AWS Lambda users write functions, which are self-contained applications written in one of the supported languages and runtimes, and upload them to AWS Lambda, which then executes them in a fast and flexible manner. 

Lambda functions can be used to do anything from serving web pages to processing data streams to calling APIs and integrating with other AWS services.

AWS manages AWS Lambda’s entire infrastructure layer. Customers don’t have much control over how the system works, but they also don’t have to worry about updating the underlying machines, avoiding network contention, and so on—AWS handles all of that.

Each Lambda function has its container in which it runs. Lambda packages a function into a new container and runs it on an AWS multi-tenant cluster of machines. Each function’s container is assigned its required RAM and CPU capacity before the functions begin to run. When the functions are completed, the RAM allocated at the start is multiplied by the length of time the function was active. Customers are then charged based on the amount of memory allocated and the length of time it took for the function to run.

It also integrates with a variety of other AWS services, including API Gateway, DynamoDB, and RDS, and forms the foundation for AWS Serverless solutions. Lambda is compatible with a wide range of popular languages and runtimes, making it a good fit for Serverless developers.

Key Features of AWS Lambda

• Concurrency and Scaling Controls: Concurrency and scaling controls, such as concurrency limits and provisioned concurrency, allow you to fine-tune the scalability and reactivity of your production applications.
• Functions Defined as Container Images: To develop, test, and deploy your Lambda functions, it lets you use your favorite container image tooling, processes, and dependencies.
• Code Signing: Code signing for Lambda offers trust and integrity controls, allowing you to ensure that only unmodified code published by authorized developers is deployed in your Lambda services.
• Lambda Extensions: Lambda extensions may be used to enhance your Lambda functions. You can use extensions, for example, to make it easier to combine Lambda with your chosen tools for monitoring, observability, security, and governance.
• Blueprints of Function: A function blueprint contains sample code that demonstrates how to utilize Lambda in conjunction with other AWS services or third-party applications. Sample code and function setup settings for the Node.js and Python run-times are included in the blueprints.
• Database Accessibility: A database proxy controls a pool of database connections and transmits function queries. This allows a function to attain large levels of concurrency without draining database connections.
• Access to File Systems: A function can be configured to mount an Amazon Elastic File System (Amazon EFS) file system to a local directory. With Amazon EFS, your function code may securely and concurrently access and alter shared resources

Understanding Serverless Webpack

A serverless plugin uses webpack to bundle your functions individually.

This plugin has the following features/advantages when compared to other webpack/optimization plugins:

• There is no configuration.
• The sls package, sls deploy, and sls deploy function are all supported.
• Individual functions are packaged into Lambda deployment packages (zip) that only contain the code needed to run the function (no bloat)
• Because dependencies are isolated, an array of webpack configurations is used instead of a single webpack configuration with multiple entry points, resulting in better tree-shaking.

This plugin is based on Serverless Webpack partially.

Here are some critical aspects of Serverless Webpack:

• Serverless Webpack Aspects: Install
• Serverless Webpack Aspects: Configure
• Serverless Webpack Aspects: Node Modules
• Serverless Webpack Aspects: Support for Docker Images as Custom Runtimes
• Serverless Webpack Aspects: Usage
• Serverless Webpack Aspects: Provider Support
• Serverless Webpack Aspects: Plugin Support

Serverless Webpack Aspects: Install

The code below is used to install serverless Webpack: 

$ npm install serverless-webpack --save-dev

Include the following code in your serverless.yml file:

plugins:
  - serverless-webpack

Serverless Webpack Aspects: Configure

The Serverless webpack plugin is configured by adding a custom: webpack object to your serverless.yml file with your specific configuration. If any settings are missing, they will be set to reasonable defaults.

The settings are described in detail in the sections below. The default settings are:

custom:
  webpack:
    webpackConfig: 'webpack.config.js' # Name of webpack configuration file
    includeModules: false # Node modules configuration for packaging
    packager: 'npm' # Packager that will be used to package your external modules
    excludeFiles: src/**/*.test.js # Provide a glob for files to ignore

Webpack Configuration File

The plugin will look in the service directory by default for a webpack.config.js file. You can also use serverless.yml to specify another file or configuration.

custom:
  webpack:
    webpackConfig: ./folder/my-webpack.config.js

This is an example of a basic Webpack configuration:

// webpack.config.js

module.exports = {
  entry: './handler.js',
  target: 'node',
  module: {
    loaders: [ ... ]
  }
};

Alternatively, the Webpack configuration can export an asynchronous object (such as a promise or an async function) that the plugin will await and resolve to the final configuration object. This is useful if the configuration relies on asynchronous functions, such as defining the current AWS user’s AccountId inside AWS lambda@edge, which does not allow for the definition of normal process environment variables.

This is how a basic Webpack promise configuration might look:

// Version if the local Node.js version supports async/await
// webpack.config.js

const webpack = require('webpack')
const slsw = require('serverless-webpack');

module.exports = (async () => {
  const accountId = await slsw.lib.serverless.providers.aws.getAccountId();
  return {
    entry: './handler.js',
    target: 'node',
    plugins: [
      new webpack.DefinePlugin({
        AWS_ACCOUNT_ID: `${accountId}`,
      }),
    ],
    module: {
      loaders: [ ... ]
    }
  };
})();

// Version with promises
// webpack.config.js

const webpack = require('webpack')
const slsw = require('serverless-webpack');
const BbPromise = require('bluebird');

module.exports = BbPromise.try(() => {
  return slsw.lib.serverless.providers.aws.getAccountId()
  .then(accountId => ({
    entry: './handler.js',
    target: 'node',
    plugins: [
      new webpack.DefinePlugin({
        AWS_ACCOUNT_ID: `${accountId}`,
      }),
    ],
    module: {
      loaders: [ ... ]
    }
  }));
});

serverless-webpack Lib Export Helper

serverless-webpack exposes a lib object that can be used in your webpack.config.js to simplify configuration and create fully dynamic configurations. This is the recommended method of configuring Webpack because the plugin will handle as much of the configuration (and subsequent changes to your services) as possible.

Automatic Entry Resolution

You can let the plugin choose the appropriate handler entry points during the build process. Then you won’t have to worry about adding or removing features from your service:

// webpack.config.js
const slsw = require('serverless-webpack');

module.exports = {
  ...
  entry: slsw.lib.entries,
  ...
};

It’s also possible to add custom entries that aren’t part of the SLS Build Process:

// webpack.config.js
const _ = require('lodash');
const slsw = require('serverless-webpack');

module.exports = {
  ...
  entry: _.assign({
    myCustomEntry1: './custom/path/something.js'
  }, slsw.lib.entries),
  ...
};

Full Customization

The serverless and options properties in the lib export allow you to access the Serverless instance and command-line options.

The current stage e.g is accessible through slsw.lib.options.stage.

This enables you to build a fully customized dynamic configuration that can test anything in the Serverless framework with no limitations.

You have access to everything a Serverless plugin would have, including the current stage and the complete service definition.

Both properties should be treated with caution and never written to, as doing so will change the running framework and cause unpredictable behavior.

If you have use cases that require complete customization, you can showcase your solution in the plugin examples.

Invocation State

State variables in lib.webpack can be used to dynamically configure the build based on the state of a plugin.

isLocal: If any known mechanism is used in the current Serverless invocation that runs code locally, the boolean property “lib.webpack.isLocal” is set to true.

mode: slsw.lib.webpack.isLocal ? "development" : "production"

Output

If no output configuration is specified, bundles will be written to the.webpack directory automatically. If you’re creating your own output configuration, make sure to include a libraryTarget to ensure compatibility with external dependencies:

// webpack.config.js
const path = require('path');

module.exports = {
  // ...
  output: {
    libraryTarget: 'commonjs',
    path: path.resolve(__dirname, '.webpack'),
    filename: '[name].js'
  }
  // ...
};

Stats

The plugin will print a lot of bundle information to your console by default. If you’re not happy with the current output information, you can change it in your webpack.config.js file.

// webpack.config.js

module.exports = {
  // ...
  stats: 'minimal'
  // ...
};

Serverless Webpack Aspects: Node Modules

The plugin tries to bundle all dependencies by default. In some cases, such as selectively importing, excluding built-in packages (such as aws-sdk), and handling webpack-incompatible modules, you may not want to include all modules.

You could use Webpack’s externals configuration to add external modules in this case. With the custom: webpack: includeModules option in serverless.yml: those modules can be included in the Serverless bundle.

// webpack.config.js
var nodeExternals = require('webpack-node-externals');

module.exports = {
  // we use webpack-node-externals to excludes all node deps.
  // You can manually set the externals too.
  externals: [nodeExternals()]
};

# serverless.yml
custom:
  webpack:
    includeModules: true # enable auto-packing of external modules

All external modules will be excluded from the bundled files. If a webpack chunk uses an excluded module, it will be packed into the Serverless artifact in the node modules directory.

If you want to use a different package file, set packagePath to your custom “package.json”: 

# serverless.yml
custom:
  webpack:
    includeModules:
      packagePath: '../package.json' # relative path to custom package.json file.

All of the above external dependencies’ peerDependencies will also be packed into the Serverless artifact. Node modules in the same directory as package.json (current working directory or packagePath) are used by default.

However, in some configurations (such as monorepo), node modules are located in the parent directory, which is not the same as package.json. Set nodeModulesRelativeDir to the directory where node modules are located.

# serverless.yml
custom:
  webpack:
    includeModules:
      nodeModulesRelativeDir: '../../' # relative path to current working directory.

PeerDependencies are installed by default when using NPM 8. When possible, you’ll use package-lock.json to avoid adding all transitive dependencies to your package.json. If your project is part of a monorepo, you can specify the package-lock.json file’s location:

# serverless.yml
custom:
  webpack:
    includeModules:
      nodeModulesRelativeDir: '../../' # relative path to the current working directory.
    packagerOptions:
      lockFile: '../../package-lock.json' # relative path to package-lock.json

Runtime Dependencies

The plugin will cause an error if a runtime dependency is found in the devDependencies section and thus would not be packaged unless you explicitly exclude it or move it to the dependencies section.

AWS-SDK

The AWS-SDK is an exception to the runtime dependency error. Because AWS provides it already in their Lambda environment, all projects using the AWS-SDK usually have it listed in devDependencies. The aws-sdk is automatically excluded in this case, and only an informative message is printed (in —verbose mode).

The warning is issued because silently ignoring anything goes against the declarative nature of Serverless’ service definition. As a result, the correct way to define the handling for the aws-sdk is to do so in the same way that you would for all other excluded modules.

# serverless.yml
custom:
  webpack:
    includeModules:
      forceExclude:
        - aws-sdk

Packagers

You can choose which packager your external modules will be packaged with. The packager configuration can be used to configure the packager. It can currently be either ‘npm’ or ‘yarn,’ with npm being the default.

# serverless.yml
custom:
  webpack:
    packager: 'yarn' # Defaults to npm
    packagerOptions: {} # Optional, depending on the selected packager

Only then will locked versions be handled correctly, as the plugin uses the generated (and usually committed) package-lock file that your favorite packager generates.

Specific options may be supported by each packager and can be set in the packagerOptions configuration setting.

NPM

The plugin packages external modules using NPM by default. However, if you’re using npm, you should use any version 5.5 >=5.7.1 because the versions in between have some nasty bugs.

The following packagerOptions are supported by the NPM packager:

Option	Type	Default	Description
noInstall	bool	false	Do not run npm install (assume install completed)

lockFile string ./package-lock.json Use relative path to lock file

The package-lock.json file will be used instead of modules installed in node modules when using NPM version >= 7.0.0. This improves NPM >= 8.0.0’s ability to automatically install peer dependencies. The correct version will be detected by the plugin.

Yarn

Using yarn will change the entire packaging pipeline, so use a yarn.lock file instead.

The following packagerOptions are supported by the yarn packager:

Option	Type	Default	Description
ignoreScripts	bool	false	Do not execute package.json hook scripts on install
noInstall	bool	false	Do not run yarn install (assume install completed)
noFrozenLockfile	bool	false	Do not require an up-to-date yarn.lock
networkConcurrency	int		Specify the number of concurrent network requests

Common Packager Options

Some settings are shared by all packagers and have an impact on the packaging like custom scripts can be chosen specifically to run by the packager.

Custom Scripts

After the function/service packages have been installed, you can specify custom scripts to run. These scripts are generic and can be used in any package. json.

Warning: They have very few specific use cases, so you should first see if your use case can be covered by webpack plugins. They should never access files that are not in their current working directory, which is the compiled function folder if one exists. A valid use case would be to start anything available as binary from node_modules.

custom:
  webpack:
    packagerOptions:
      scripts:
        - npm rebuild grpc --target=6.1.0 --target_arch=x64 --target_platform=linux --target_libc=glibc

Forced Inclusion

It’s possible to have dynamic requirements in your code, which means that you’ll need modules that are only known at runtime. Because Webpack is unable to detect such externals, the compiled package will be missing critical dependencies. In such cases, you can use the forceInclude array property to force the plugin to include specific modules. However, the module must be included in the package’s production dependencies for your service:

# serverless.yml
custom:
  webpack:
    includeModules:
      forceInclude:
        - module1
        - module2

Forced Exclusion

If a module in your dependencies is already installed at your provider’s environment, you can forcefully exclude it.

They will not be packaged if they are added to the forceExclude array property.

# serverless.yml
custom:
  webpack:
    includeModules:
      forceExclude:
        - module1
        - module2

When you specify a module in both the forceInclude and forceExclude arrays, the exclude array wins, and the module is not packaged.

Local Modules

In your package, you can use file: version references. To use a node module from a local folder, use json (for example, “mymodule”: “file:../../myOtherProject/mymodule”). You can use this to test deployments on a local machine using different module versions or modules before they are officially released.

Exclude Files with Similar Names

If you have a project structure that uses something like index.js and a co-located index.test.js then you have likely seen an error like WARNING: More than one matching handlers found for index. Using index.js.

This config option allows you to exclude files from function resolution that match a glob( A glob is a filepath matching string made up of literal and/or wildcard characters). Simply add the following: excludeFiles: **/*.test.js (with whatever glob you want to exclude).

# serverless.yml
custom:
  webpack:
    excludeFiles: **/*.test.js

This is also useful for TypeScript-based projects.

Exclude Files with Regular Expression

Before adding files to the zip file, you can use this config option to filter for files that match a regex pattern. Simply type excludeRegex:.ts|test|.map into the command line (with whatever regex you want to exclude).

# serverless.yml
custom:
  webpack:
    excludeRegex: .ts|test|.map

Keep Output Directory After Packaging

After the build, the output directory (defaults to.webpack) can be kept and KeepOutputDirectory: true is all that is required.

# serverless.yml
custom:
  webpack:
    keepOutputDirectory: true

This is useful if you want to upload the source maps to your error reporting system or simply have them on hand for post-processing.

Nodejs Custom Runtime

AllowCustomRuntime: true can be used if you’re using a nodejs custom runtime.

exampleFunction:
  handler: path/to/your/handler.default
  runtime: provided
  allowCustomRuntime: true

Service Level Packaging

If individual packaging is not enabled in your service (serverless.yml), the plugin creates a single ZIP file (the service package) that contains all node modules used in the service. This is the quickest packaging method, but it is not the best, because node modules that are not required by some functions are always packaged.

Optimization / Individual Packaging per Function

Individual packaging is a better way to deal with packaging in your service:

# serverless.yml
---
package:
  individually: true

This will use Webpack’s multi-compiler feature to switch the plugin to per-function packaging. Webpack does this by compiling and optimizing each function separately, removing unnecessary imports, and significantly reducing code sizes. With this strategy, tree-shaking(Tree shaking is a term used in JavaScript to describe the process of removing dead code) makes sense.

Webpack now detects the required external node modules per function, and the plugin only bundles the ones that are required into the function artifacts. As a result, depending on the functions, the deployed artifacts are smaller, and cold-start times (the time it takes to install the functions in the cloud at runtime) are also shorter.

You will not be able to configure the entry config in webpack because the individual packaging will apply the automatic entry resolution (see above). If you try to override the entry in webpack.config.js with unsupported values, an error will be thrown.

Individual packaging requires more time during the packaging phase, but you’ll be repaid twice during runtime.

Individual Packaging Concurrency

# serverless.yml
custom:
  webpack:
    concurrency: 5 # desired concurrency, defaults to the number of available cores
    serializedCompile: true # backward compatible, this translates to concurrency: 1

You can run each webpack build individually, which helps save memory and improves overall build performance in some cases.

Serverless Webpack Aspects: Support for Docker Images as Custom Runtimes

In the year 2021, AWS Lambda and serverless webpack began supporting Docker images as custom runtimes. Details on how to use these features with a serverless.yml can be found in the serverless documentation.

Entrypoint is inherited from the shared Docker image in the following example, while the command is provided as a function override:

# serverless.yml
functions:
  myFunction1:
    image:
      name: public.ecr.aws/lambda/nodejs:12
      command:
        - app.handler1
  myFunction2:
    image:
      name: public.ecr.aws/lambda/nodejs:12
      command:
        - app.handler2

If you want to use a remote docker image but first need to run the webpack process, you can do so as follows:

# serverless.yml
functions:
  myFunction1:
    image: public.ecr.aws/lambda/nodejs:latest

Serverless Webpack Aspects: Usage

Automatic Bundling

Webpack will be automatically bundled with the standard Serverless deploy procedure:

• Create the Serverless project with serverless create -t aws-nodejs
• Install Serverless Webpack as above.
• Deploy with serverless deploy.

Run a Function Locally

The plugin is completely compatible with serverless invoke local. You can use the following methods to run your bundled functions locally:

$ serverless invoke local --function <function-name>

Invoke local supports the following options:

• The name of the function to run is —function or -f (required).
• The function input event is triggered by —path or -p (optional).
• Inline JSON data is used as the function input event with —data or -d (optional).

Run a Function with an Existing Compiled Output

On CI systems, you’ll most likely run multiple integration tests by invoking local in sequential order. To improve this, you can compile once and then run multiple invokes on the compiled output; you don’t have to compile before each invoke.

custom:
  webpack:
    noBuild: true

Run a Function Locally on Source Changes

Alternatively, use the —watch option with serverless invoke local: to run a function every time the source files change.

$ serverless invoke local --function <function-name> --path event.json --watch

When the sources are changed, the function will be run again with the new sources. The command will keep an eye on the process until it is finished.

If your sources are on a file system that doesn’t support events, you can use the —webpack-use-polling=time in ms> option to enable polling. If you don’t specify a value, it defaults to 3000 milliseconds.

All of the options that invoke local supports can be used as usual:

• The name of the function to run is specified by —function or -f (required).
• path or -p (optional) is the path to a JSON file that will be used as the function input event 
• data or -d (optional) is inline JSON data that will be used as the function input event

Usage with Serverless Run 

With the plugin, you can use the serverless run command. The serverless run command, as documented by Serverless, can be used to test a local service in the Serverless Emulator. Before uploading it to the event gateway, the command will compile the code.

Serverless Run with Webpack Watch Mode

With serverless run —watch, you can enable source watch mode. The plugin will then monitor for changes in the source code, recompile, and deploy the code to the event gateway. As a result, you can simply keep the event gateway running while testing new code.

Usage with Serverless-Offline

The plugin works well with serverless-offline to locally simulate AWS Lambda and AWS API Gateway.

Make sure that serverless-webpack comes before serverless-offline in your serverless.yml file, as the order matters:

plugins: ...
  - serverless-webpack
  ...
  - serverless-offline
  ...

To begin the Lambda/API simulation, run serverless offline or serverless offline start.

In contrast to serverless offline, the start command will fire an init and an end lifecycle hook required by serverless-offline and, for example, serverless-dynamodb-local to turn off resources (see below).

When triggered through serverless offline, the plugin defaults to watch mode, which automatically recompiles your code if it detects a change in the used sources. It may take a few seconds for the emulated endpoints to update after a change.

If your sources are on a file system that does not support events, such as a mounted volume in a Docker container, you can use the —webpack-use-polling=time in ms> option to enable polling. If you leave the value blank, it defaults to 3000 milliseconds.

Select the —no-build option if you don’t want the plugin to be built when using serverless-offline.

Custom Paths

Use the —location option if you want to override the default path in your Webpack configuration.

Serverless-dynamodb-local

Configure your service as before but add the serverless-dynamodb-local plugin:

plugins:

•   – serverless-webpack
•  – serverless-dynamodb-local
• – serverless-offline

Run serverless offline start.

Other Useful Options

With —dontPrintOutput and —noTimeout, you can reduce the amount of clutter created by serverless-offline.

You might want to disable the automatic watch mode with the —webpack-no-watch switch if you’re using serverless offline to run your integration tests.

Bundle with Webpack

To simply bundle and view the output, use:

$ serverless webpack --out dist

Options include

• -o or —out (optional) The output folder. The default value is.webpack.

Simulate API Gateway Locally

The serve command is no longer available. See the serverless-offline plugin for an example of how to achieve the same functionality.

vscode Debugging

To debug your serverless functions, you can use local or serverless-offline.

Serverless Webpack Aspects: Provider Support

The providers listed below support plugin commands. ?? means that the command hasn’t been tried with that provider.

AWS Lambda	Apache OpenWhisk	Azure Functions	Google Cloud Functions
webpack	✔︎	✔︎	⁇	⁇
invoke local	✔︎	✔︎	⁇	⁇
invoke local –watch	✔︎	✔︎	⁇	⁇

Serverless Webpack Aspects: Plugin Support

Serverless-webpack explicitly supports the following serverless plugins:

Plugin	NPM
serverless-offline	npm v8.7.0
serverless-step-functions-offline	npm.v2.1.4

Conclusion

This article talks about Serverless Webpack in Lambda extensively. It also gives a brief introduction to AWS Lambda and its key features.

visit our website to explore hevo

Hevo Data, a No-code Data Pipeline provides you with a consistent and reliable solution to manage data transfer between a variety of sources and a wide variety of Desired Destinations, with a few clicks. Hevo Data with its strong integration with 100+ sources (including 40+ free sources) allows you to not only export data from your desired data sources & load it to the destination of your choice, but also transform & enrich your data to make it analysis-ready so that you can focus on your key business needs and perform insightful analysis using BI tools.

Want to take Hevo for a spin? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also have a look at the unbeatable pricing that will help you choose the right plan for your business needs.