Serverless Webpack in Lambda Simplified 101

In today’s fast-paced development world, serverless architectures are gaining significant popularity due to their scalability, cost-effectiveness, and simplicity. One key tool that facilitates the development and deployment of serverless applications is Serverless Webpack. It enables developers to bundle their code and dependencies efficiently, optimizing serverless functions like AWS Lambda.

This blog will take you through the essential aspects of working with Serverless Webpack, including installation, configuration, packaging, and advanced usage features. Whether you’re looking to enhance your serverless development workflow or better manage AWS Lambda functions, this guide provides all the information you need.

Understanding Serverless Webpack

A serverless plugin uses webpack to bundle your functions individually.

• There is no configuration.
• The sls package, sls deploy, and sls deploy function are all supported.
• Individual functions are packaged into Lambda deployment packages (zip) that only contain the code needed to run the function (no bloat)
• Because dependencies are isolated, an array of webpack configurations is used instead of a single webpack configuration with multiple entry points, resulting in better tree-shaking.

Here are some critical aspects of Serverless Webpack:

• Install
• Configure
• Node Modules
• Support for Docker Images as Custom Runtimes
• Aspects: Usage
• Provider Support
• SPlugin Support

Install

The code below is used to install serverless Webpack: 

$ npm install serverless-webpack --save-dev

Include the following code in your serverless.yml file:

plugins:
  - serverless-webpack

Configure

The Serverless webpack plugin is configured by adding a custom: webpack object to your serverless.yml file with your specific configuration. If any settings are missing, they will be set to reasonable defaults.

The settings are described in detail in the sections below. The default settings are:

custom:
  webpack:
    webpackConfig: 'webpack.config.js' # Name of webpack configuration file
    includeModules: false # Node modules configuration for packaging
    packager: 'npm' # Packager that will be used to package your external modules
    excludeFiles: src/**/*.test.js # Provide a glob for files to ignore

Webpack Configuration File

The plugin will look in the service directory by default for a webpack.config.js file. You can also use serverless.yml to specify another file or configuration.

custom:
  webpack:
    webpackConfig: ./folder/my-webpack.config.js

This is an example of a basic Webpack configuration:

// webpack.config.js

module.exports = {
  entry: './handler.js',
  target: 'node',
  module: {
    loaders: [ ... ]
  }
};

Alternatively, the Webpack configuration can export an asynchronous object (such as a promise or an async function) that the plugin will await and resolve to the final configuration object. This is useful if the configuration relies on asynchronous functions, such as defining the current AWS user’s AccountId inside AWS lambda@edge, which does not allow for the definition of normal process environment variables.

This is how a basic Webpack promise configuration might look:

// Version if the local Node.js version supports async/await
// webpack.config.js

const webpack = require('webpack')
const slsw = require('serverless-webpack');

module.exports = (async () => {
  const accountId = await slsw.lib.serverless.providers.aws.getAccountId();
  return {
    entry: './handler.js',
    target: 'node',
    plugins: [
      new webpack.DefinePlugin({
        AWS_ACCOUNT_ID: `${accountId}`,
      }),
    ],
    module: {
      loaders: [ ... ]
    }
  };
})();

// Version with promises
// webpack.config.js

const webpack = require('webpack')
const slsw = require('serverless-webpack');
const BbPromise = require('bluebird');

module.exports = BbPromise.try(() => {
  return slsw.lib.serverless.providers.aws.getAccountId()
  .then(accountId => ({
    entry: './handler.js',
    target: 'node',
    plugins: [
      new webpack.DefinePlugin({
        AWS_ACCOUNT_ID: `${accountId}`,
      }),
    ],
    module: {
      loaders: [ ... ]
    }
  }));
});

serverless-webpack Lib Export Helper

serverless-webpack exposes a lib object that can be used in your webpack.config.js to simplify configuration and create fully dynamic configurations. This is the recommended method of configuring Webpack because the plugin will handle as much of the configuration (and subsequent changes to your services) as possible.

Automatic Entry Resolution

You can let the plugin choose the appropriate handler entry points during the build process. Then you won’t have to worry about adding or removing features from your service:

// webpack.config.js
const slsw = require('serverless-webpack');

module.exports = {
  ...
  entry: slsw.lib.entries,
  ...
};

It’s also possible to add custom entries that aren’t part of the SLS Build Process:

// webpack.config.js
const _ = require('lodash');
const slsw = require('serverless-webpack');

module.exports = {
  ...
  entry: _.assign({
    myCustomEntry1: './custom/path/something.js'
  }, slsw.lib.entries),
  ...
};

Full Customization

The serverless and options properties in the lib export allow you to access the Serverless instance and command-line options.

The current stage e.g is accessible through slsw.lib.options.stage.

This enables you to build a fully customized dynamic configuration that can test anything in the Serverless framework with no limitations.

You have access to everything a Serverless plugin would have, including the current stage and the complete service definition.

Both properties should be treated with caution and never written to, as doing so will change the running framework and cause unpredictable behavior.

If you have use cases that require complete customization, you can showcase your solution in the plugin examples.

Invocation State

State variables in lib.webpack can be used to dynamically configure the build based on the state of a plugin.

isLocal: If any known mechanism is used in the current Serverless invocation that runs code locally, the boolean property “lib.webpack.isLocal” is set to true.

mode: slsw.lib.webpack.isLocal ? "development" : "production"

Output

If no output configuration is specified, bundles will be written to the.webpack directory automatically. If you’re creating your own output configuration, make sure to include a libraryTarget to ensure compatibility with external dependencies:

// webpack.config.js
const path = require('path');

module.exports = {
  // ...
  output: {
    libraryTarget: 'commonjs',
    path: path.resolve(__dirname, '.webpack'),
    filename: '[name].js'
  }
  // ...
};

Stats

The plugin will print a lot of bundle information to your console by default. If you’re not happy with the current output information, you can change it in your webpack.config.js file.

// webpack.config.js

module.exports = {
  // ...
  stats: 'minimal'
  // ...
};

Node Modules

The plugin tries to bundle all dependencies by default. In some cases, such as selectively importing, excluding built-in packages (such as aws-sdk), and handling webpack-incompatible modules, you may not want to include all modules.

You could use Webpack’s externals configuration to add external modules in this case. With the custom: webpack: includeModules option in serverless.yml: those modules can be included in the Serverless bundle.

// webpack.config.js
var nodeExternals = require('webpack-node-externals');

module.exports = {
  // we use webpack-node-externals to excludes all node deps.
  // You can manually set the externals too.
  externals: [nodeExternals()]
};

# serverless.yml
custom:
  webpack:
    includeModules: true # enable auto-packing of external modules

All external modules will be excluded from the bundled files. If a webpack chunk uses an excluded module, it will be packed into the Serverless artifact in the node modules directory.

If you want to use a different package file, set packagePath to your custom “package.json”: 

# serverless.yml
custom:
  webpack:
    includeModules:
      packagePath: '../package.json' # relative path to custom package.json file.

All of the above external dependencies’ peerDependencies will also be packed into the Serverless artifact. Node modules in the same directory as package.json (current working directory or packagePath) are used by default.

However, in some configurations (such as monorepo), node modules are located in the parent directory, which is not the same as package.json. Set nodeModulesRelativeDir to the directory where node modules are located.

# serverless.yml
custom:
  webpack:
    includeModules:
      nodeModulesRelativeDir: '../../' # relative path to current working directory.

PeerDependencies are installed by default when using NPM 8. When possible, you’ll use package-lock.json to avoid adding all transitive dependencies to your package.json. If your project is part of a monorepo, you can specify the package-lock.json file’s location:

# serverless.yml
custom:
  webpack:
    includeModules:
      nodeModulesRelativeDir: '../../' # relative path to the current working directory.
    packagerOptions:
      lockFile: '../../package-lock.json' # relative path to package-lock.json

Runtime Dependencies

The plugin will cause an error if a runtime dependency is found in the devDependencies section and thus would not be packaged unless you explicitly exclude it or move it to the dependencies section.

AWS-SDK

The AWS-SDK is an exception to the runtime dependency error. Because AWS provides it already in their Lambda environment, all projects using the AWS-SDK usually have it listed in devDependencies. The aws-sdk is automatically excluded in this case, and only an informative message is printed (in —verbose mode).

The warning is issued because silently ignoring anything goes against the declarative nature of Serverless’ service definition. As a result, the correct way to define the handling for the aws-sdk is to do so in the same way that you would for all other excluded modules.

# serverless.yml
custom:
  webpack:
    includeModules:
      forceExclude:
        - aws-sdk

Packagers

You can choose which packager your external modules will be packaged with. The packager configuration can be used to configure the packager. It can currently be either ‘npm’ or ‘yarn,’ with npm being the default.

# serverless.yml
custom:
  webpack:
    packager: 'yarn' # Defaults to npm
    packagerOptions: {} # Optional, depending on the selected packager

Only then will locked versions be handled correctly, as the plugin uses the generated (and usually committed) package-lock file that your favorite packager generates.

Specific options may be supported by each packager and can be set in the packagerOptions configuration setting.

NPM

The plugin packages external modules using NPM by default. However, if you’re using npm, you should use any version 5.5 >=5.7.1 because the versions in between have some nasty bugs.

The following packagerOptions are supported by the NPM packager:

Option	Type	Default	Description
noInstall	bool	false	Do not run npm install (assume install completed)

lockFile string ./package-lock.json Use relative path to lock file

The package-lock.json file will be used instead of modules installed in node modules when using NPM version >= 7.0.0. This improves NPM >= 8.0.0’s ability to automatically install peer dependencies. The correct version will be detected by the plugin.

Yarn

Using yarn will change the entire packaging pipeline, so use a yarn.lock file instead.

The following packagerOptions are supported by the yarn packager:

Option	Type	Default	Description
ignoreScripts	bool	false	Do not execute package.json hook scripts on install
noInstall	bool	false	Do not run yarn install (assume install completed)
noFrozenLockfile	bool	false	Do not require an up-to-date yarn.lock
networkConcurrency	int		Specify the number of concurrent network requests

Common Packager Options

Some settings are shared by all packagers and have an impact on the packaging like custom scripts can be chosen specifically to run by the packager.

Custom Scripts

After the function/service packages have been installed, you can specify custom scripts to run. These scripts are generic and can be used in any package. json.

Warning: They have very few specific use cases, so you should first see if your use case can be covered by webpack plugins. They should never access files that are not in their current working directory, which is the compiled function folder if one exists. A valid use case would be to start anything available as binary from node_modules.

custom:
  webpack:
    packagerOptions:
      scripts:
        - npm rebuild grpc --target=6.1.0 --target_arch=x64 --target_platform=linux --target_libc=glibc

Forced Inclusion

It’s possible to have dynamic requirements in your code, which means that you’ll need modules that are only known at runtime. Because Webpack is unable to detect such externals, the compiled package will be missing critical dependencies. In such cases, you can use the forceInclude array property to force the plugin to include specific modules. However, the module must be included in the package’s production dependencies for your service:

# serverless.yml
custom:
  webpack:
    includeModules:
      forceInclude:
        - module1
        - module2

Forced Exclusion

If a module in your dependencies is already installed at your provider’s environment, you can forcefully exclude it.

They will not be packaged if they are added to the forceExclude array property.

# serverless.yml
custom:
  webpack:
    includeModules:
      forceExclude:
        - module1
        - module2

When you specify a module in both the forceInclude and forceExclude arrays, the exclude array wins, and the module is not packaged.

Local Modules

In your package, you can use file: version references. To use a node module from a local folder, use json (for example, “mymodule”: “file:../../myOtherProject/mymodule”). You can use this to test deployments on a local machine using different module versions or modules before they are officially released.

Exclude Files with Similar Names

If you have a project structure that uses something like index.js and a co-located index.test.js then you have likely seen an error like WARNING: More than one matching handlers found for index. Using index.js.

This config option allows you to exclude files from function resolution that match a glob( A glob is a filepath matching string made up of literal and/or wildcard characters). Simply add the following: excludeFiles: **/*.test.js (with whatever glob you want to exclude).

# serverless.yml
custom:
  webpack:
    excludeFiles: **/*.test.js

This is also useful for TypeScript-based projects.

Exclude Files with Regular Expression

Before adding files to the zip file, you can use this config option to filter for files that match a regex pattern. Simply type excludeRegex:.ts|test|.map into the command line (with whatever regex you want to exclude).

# serverless.yml
custom:
  webpack:
    excludeRegex: .ts|test|.map

Keep Output Directory After Packaging

After the build, the output directory (defaults to.webpack) can be kept and KeepOutputDirectory: true is all that is required.

# serverless.yml
custom:
  webpack:
    keepOutputDirectory: true

This is useful if you want to upload the source maps to your error reporting system or simply have them on hand for post-processing.

Nodejs Custom Runtime

AllowCustomRuntime: true can be used if you’re using a nodejs custom runtime.

exampleFunction:
  handler: path/to/your/handler.default
  runtime: provided
  allowCustomRuntime: true

Service Level Packaging

If individual packaging is not enabled in your service (serverless.yml), the plugin creates a single ZIP file (the service package) that contains all node modules used in the service. This is the quickest packaging method, but it is not the best, because node modules that are not required by some functions are always packaged.

Optimization / Individual Packaging per Function

Individual packaging is a better way to deal with packaging in your service:

# serverless.yml
---
package:
  individually: true

This will use Webpack’s multi-compiler feature to switch the plugin to per-function packaging. Webpack does this by compiling and optimizing each function separately, removing unnecessary imports, and significantly reducing code sizes. With this strategy, tree-shaking(Tree shaking is a term used in JavaScript to describe the process of removing dead code) makes sense.

Webpack now detects the required external node modules per function, and the plugin only bundles the ones that are required into the function artifacts. As a result, depending on the functions, the deployed artifacts are smaller, and cold-start times (the time it takes to install the functions in the cloud at runtime) are also shorter.

You will not be able to configure the entry config in webpack because the individual packaging will apply the automatic entry resolution (see above). If you try to override the entry in webpack.config.js with unsupported values, an error will be thrown.

Individual packaging requires more time during the packaging phase, but you’ll be repaid twice during runtime.

Individual Packaging Concurrency

# serverless.yml
custom:
  webpack:
    concurrency: 5 # desired concurrency, defaults to the number of available cores
    serializedCompile: true # backward compatible, this translates to concurrency: 1

You can run each webpack build individually, which helps save memory and improves overall build performance in some cases.

Serverless Webpack Aspects: Support for Docker Images as Custom Runtimes

In the year 2021, AWS Lambda and serverless webpack began supporting Docker images as custom runtimes. Details on how to use these features with a serverless.yml can be found in the serverless documentation.

Entrypoint is inherited from the shared Docker image in the following example, while the command is provided as a function override:

# serverless.yml
functions:
  myFunction1:
    image:
      name: public.ecr.aws/lambda/nodejs:12
      command:
        - app.handler1
  myFunction2:
    image:
      name: public.ecr.aws/lambda/nodejs:12
      command:
        - app.handler2

If you want to use a remote docker image but first need to run the webpack process, you can do so as follows:

# serverless.yml
functions:
  myFunction1:
    image: public.ecr.aws/lambda/nodejs:latest

Usage

Automatic Bundling

Webpack will be automatically bundled with the standard Serverless deploy procedure:

• Create the Serverless project with serverless create -t aws-nodejs
• Install Serverless Webpack as above.
• Deploy with serverless deploy.

Run a Function Locally

The plugin is completely compatible with serverless invoke local. You can use the following methods to run your bundled functions locally:

$ serverless invoke local --function <function-name>

Invoke local supports the following options:

• The name of the function to run is —function or -f (required).
• The function input event is triggered by —path or -p (optional).
• Inline JSON data is used as the function input event with —data or -d (optional).

Run a Function with an Existing Compiled Output

On CI systems, you’ll most likely run multiple integration tests by invoking local in sequential order. To improve this, you can compile once and then run multiple invokes on the compiled output; you don’t have to compile before each invoke.

custom:
  webpack:
    noBuild: true

Run a Function Locally on Source Changes

Alternatively, use the —watch option with serverless invoke local: to run a function every time the source files change.

$ serverless invoke local --function <function-name> --path event.json --watch

When the sources are changed, the function will be run again with the new sources. The command will keep an eye on the process until it is finished.

If your sources are on a file system that doesn’t support events, you can use the —webpack-use-polling=time in ms> option to enable polling. If you don’t specify a value, it defaults to 3000 milliseconds.

All of the options that invoke local supports can be used as usual:

• The name of the function to run is specified by —function or -f (required).
• path or -p (optional) is the path to a JSON file that will be used as the function input event 
• data or -d (optional) is inline JSON data that will be used as the function input event

Usage with Serverless Run 

With the plugin, you can use the serverless run command. The serverless run command, as documented by Serverless, can be used to test a local service in the Serverless Emulator. Before uploading it to the event gateway, the command will compile the code.

Serverless Run with Webpack Watch Mode

With serverless run —watch, you can enable source watch mode. The plugin will then monitor for changes in the source code, recompile, and deploy the code to the event gateway. As a result, you can simply keep the event gateway running while testing new code.

Usage with Serverless-Offline

The plugin works well with serverless-offline to locally simulate AWS Lambda and AWS API Gateway.

Make sure that serverless-webpack comes before serverless-offline in your serverless.yml file, as the order matters:

plugins: ...
  - serverless-webpack
  ...
  - serverless-offline
  ...

To begin the Lambda/API simulation, run serverless offline or serverless offline start.

In contrast to serverless offline, the start command will fire an init and an end lifecycle hook required by serverless-offline and, for example, serverless-dynamodb-local to turn off resources (see below).

When triggered through serverless offline, the plugin defaults to watch mode, which automatically recompiles your code if it detects a change in the used sources. It may take a few seconds for the emulated endpoints to update after a change.

If your sources are on a file system that does not support events, such as a mounted volume in a Docker container, you can use the —webpack-use-polling=time in ms> option to enable polling. If you leave the value blank, it defaults to 3000 milliseconds.

Select the —no-build option if you don’t want the plugin to be built when using serverless-offline.

Custom Paths

Use the —location option if you want to override the default path in your Webpack configuration.

Serverless-dynamodb-local

Configure your service as before but add the serverless-dynamodb-local plugin:

plugins:

•   – serverless-webpack
•  – serverless-dynamodb-local
• – serverless-offline

Run serverless offline start.

Other Useful Options

With —dontPrintOutput and —noTimeout, you can reduce the amount of clutter created by serverless-offline.

You might want to disable the automatic watch mode with the —webpack-no-watch switch if you’re using serverless offline to run your integration tests.

Bundle with Webpack

To simply bundle and view the output, use:

$ serverless webpack --out dist

Options include

• -o or —out (optional) The output folder. The default value is.webpack.

Simulate API Gateway Locally

The serve command is no longer available. See the serverless-offline plugin for an example of how to achieve the same functionality.

vscode Debugging

To debug your serverless functions, you can use local or serverless-offline.

Serverless Webpack Aspects: Provider Support

The providers listed below support plugin commands. ?? means that the command hasn’t been tried with that provider.

AWS Lambda	Apache OpenWhisk	Azure Functions	Google Cloud Functions
webpack	✔︎	✔︎	⁇	⁇
invoke local	✔︎	✔︎	⁇	⁇
invoke local –watch	✔︎	✔︎	⁇	⁇

Serverless Webpack Aspects: Plugin Support

Serverless-webpack explicitly supports the following serverless plugins:

Plugin	NPM
serverless-offline	npm v8.7.0
serverless-step-functions-offline	npm.v2.1.4

What is AWS Lambda?

• AWS Lambda is a serverless compute service that is triggered by events that allow you to run code for virtually any application or backend service without having to provision or manage servers. You can call Lambda from more than 200 AWS services and SaaS apps, and you only pay for what you use.

Key Features of AWS Lambda

• Concurrency and Scaling Controls
• Functions Defined as Container Images
• Code Signing
• Lambda Extensions
• Blueprints of Function
• Database Accessibility
• Access to File Systems

Conclusion

In conclusion, integrating Serverless Webpack with AWS Lambda significantly improves the efficiency and performance of serverless applications. By leveraging Webpack’s bundling capabilities, optimizing packaging, and utilizing advanced features like local function testing and Docker support, developers can streamline the deployment process. With a solid understanding of configuration, node modules, and best practices, you can build scalable, cost-effective serverless solutions that are easier to manage and maintain. Hevo can further enhance this process by providing seamless data integration and real-time analytics for your serverless applications.

Want to give Hevo a try? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite firsthand. You may also have a look at the amazing price, which will assist you in selecting the best plan for your requirements.

FAQs