Running Rust Lambda: The Ultimate Guide 2022

• April 12th, 2022

Rust Lambda FI

AWS Lambda is a serverless computing platform that allows you to run code without the need for server provisioning or management. In reaction to events like HTTP requests, the platform calls your code. On Lambda, there is currently no official support for the Rust programming language.

To run Rust code on Lambda, you’ll need to create a custom runtime. This post will show you how to run Rust Lambda. It’s aimed at people with a basic familiarity with Rust and systems programming who want to create serverless functions on AWS. You’ll create a basic Rust app and deploy it to AWS Lambda.

Table of Contents

What is Rust?

Rust is a well-established technology with a wide range of applications. It helps you to maintain control over low-level details as a systems programming language. You have the option of storing data on the stack (which is used for static memory allocation) or the heap (which is used for dynamic memory allocation). RAII (Resource Acquisition Is Initialization) is a programming idiom that is most commonly associated with C++ but is also present in Rust i.e. when an object is no longer in scope, its destructor is called and its owned resources are freed. You won’t have to perform anything by hand, and you’ll be safe against resource leakage problems.

Rust was designed to provide high performance similar to that of C and C++, but with a focus on code safety, which is these two languages’ Achilles heel. Rust, on the other hand, offers more than just memory safety. Other reasons why well-known software heavyweights now employ this programming language include high performance when processing massive volumes of data, concurrent programming capability, and an effective compiler. Rust is used in production by Firefox, Dropbox, Cloudflare, and many other firms ranging from startups to major multinationals.

Key Features of Rust

Listed below are a few key features of Rust:

  • Rust is a statically typed programming language that emphasizes performance and safety, particularly in the areas of concurrency and memory management. It has a syntax that is comparable to C++. It’s an open-source project that started as a research project at Mozilla. The Rust Foundation took over the torch in 2021 and is now steering the language’s evolution.
  • There are two ways to write code in Rust: Safe Rust and Unsafe Rust. Safe Rust places additional constraints on the programmer (for example, object ownership management), ensuring that the code runs correctly. Unsafe Rust allows the programmer more freedom (it may work with raw C-like pointers, for example), but the code may break.
  • One of Rust’s main advantages is its dual-mode concept. In C++, on the other hand, you don’t realize you’ve created dangerous code until your software crashes or a security breach occurs.
  • High performance with memory protection.
  • Concurrent programming is supported.
  • The crates.io repository has an increasing number of Rust packages.
  • The language is being developed by a thriving community.
  • Stability and backward compatibility are guaranteed (Rust has been designed for the next 40 years).

What is AWS Lambda?

AWS Lambda is Amazon’s event-driven, serverless computing technology, which is part of Amazon Web Services. As a result, you won’t have to worry about deciding which AWS resources to launch or how to maintain them. Instead, you must upload the code to Lambda, where it will execute.

The code in AWS Lambda is performed in reaction to events in AWS services such as adding/deleting files in an S3 bucket, making an HTTP call to the Amazon API gateway, and so on. Amazon Lambda, on the other hand, can only be used to perform background operations.

Instead of managing operating system (OS) access control, OS patching, right-sizing, provisioning, scaling, and other tasks, the AWS Lambda function allows you to focus on your core product and business logic.

Advantages and Disadvantages of AWS Lambda

Here are a few advantages of using AWS Lambda:

  • AWS Lambda is a very versatile tool to employ.
  • It enables you to grant access to resources, such as VPCs, using the console’s WYSIWYG (What You See Is What You Get) editor.
  • It’s available as an Eclipse and Visual Studio plugin.
  • You don’t have to bother about managing or supplying servers because it’s a serverless design.
  • There is no need to create a Virtual Machine.

Here are a few disadvantages of using AWS Lambda:

  • Because AWS Lambda relies exclusively on AWS for infrastructure, you won’t be able to install any other software until your code requires it.
  • Its memory capacity ranges from 128 to 1536 MB.
  • The maximum size of an event request is 128 KB.
  • The maximum size of an event request is 128 KB.
  • Lambda functions allow you to write logs to CloudWatch alone. This is the only way to keep track of or debug your functions.
  • It has a 5-minute code execution timeout.

Set up Seamless Data Ingestion Pipelines from AWS Sources using Hevo

Hevo Data, a Fully-managed Automated Data Pipeline solution, can help you automate, simplify & enrich your data flow from various AWS sources such as AWS S3 and AWS Elasticsearch in a matter of minutes. Hevo’s end-to-end Data Management offers streamlined preparation of Data Pipelines for your AWS account. Additionally, Hevo completely automates the process of not only extracting data from AWS S3 and AWS Elasticsearch but also enriching the data and transforming it into an analysis-ready form without having to write a single line of code.

Get Started with Hevo for Free

With Hevo’s out-of-the-box connectors and blazing-fast Data Pipelines, you can extract & aggregate data from 100+ Data Sources (including 40+ Free Sources) including AWS S3 and AWS Elasticsearch straight into your Data Warehouse, Database, or any destination. To further streamline and prepare your data for analysis, you can process and enrich Raw Granular Data using Hevo’s robust & built-in Transformation Layer without writing a single line of code!

Experience an entirely automated hassle-free Data Pipeline from AWS Services using Hevo. Try our 14-day full access free trial today!

How to Run Rust Lambda?

To start deploying Rust Lambda, follow the given steps:

Rust Lambda Process: Prerequisites

Rust Lambda Process: Creating an SDK Project

The first step in the Rust Lambda process is, that you must create the project. Even though this post is about our Rust Lambda, CDK will be at the heart of your project, and the Lambda code will be contained within it.

  • Step 1: Create a directory for the new project on the command line.
mkdir tutorial-rust-lambda
cd tutorial-rust-lambda
  • Step 2: To start a new project, use the CDK CLI. Typescript will be used.
CDK init app --language typescript

Setting an output directory for built scripts is something that you could do before starting a Typescript project. Add to the compilerOptions object in the tsconfig.json file. This will assist you in keeping the clutter out of our source files. Also, make sure that the exclude field is set to disregard our output path.

20c20,21
<     "typeRoots": ["./node_modules/@types"]
--------
>     "typeRoots": ["./node_modules/@types"],
>     "outDir": "dist"
22c23
<   "exclude": ["cdk.out"]
--------
>   "exclude": ["cdk.out", "dist"]
  • Step 3: After you’ve built the project, the next step in Rust Lambda Process is that you may start the Typescript compiler watch agent to get ready for development. An NPM run-script will be included in the project’s package.json.
npm run watch
  • Step 4: For the last step in Rust Lambda Process you must execute what is known as bootstrapping to work with CDK in the way you want. This is because the Lambda will be packaged as an asset. Locate your AWS account ID and select an AWS region, then run the following commands, substituting those values:
CDK bootstrap AWS://{AWS_ACCOUNT_ID}/{AWS_DEFAULT_REGION}

Rust Lambda Process: Write a Lambda Function in Rust

You’ll construct a very simple Rust program that will accept an event with the necessary format (a JSON string with a “name” field) and answer with “Hello name”. 

  • Step 1: Start the Rust Lambda process by configuring your Rust project within your CDK project. Your Lambda code does not need to be in your CDK project again. If your Lambda code grows into a large project, you can package it and publish it to AWS S3, then access that S3 object from CDK, but for this sample, you can do it the easy way.
cargo new lambda/hello
  • Step 2: Then, in order to build your Rust Lambda function, you’ll need to add certain dependencies to the project. In your editor, open lambda/hello/Cargo.toml and add the following to [dependencies]:
lambda_runtime = "0.3.0"
log = "0.4.14"
serde = "1.0.126"
simple_logger = "1.11.0"
tokio = "1.6.1"
  • Step 3: In the next step for the Rust Lambda process, you can write your lambda/hello/src/main.rs.
// This example requires the following input to succeed:
// { "name": "some name" }

use lambda_runtime::{handler_fn, Context, Error};
use log::LevelFilter;
use serde::{Deserialize, Serialize};
use simple_logger::SimpleLogger;

/// This is also a made-up example. Requests come into the runtime as unicode
/// strings in json format, which can map to any structure that implements `serde::Deserialize`
/// The runtime pays no attention to the contents of the request payload.
#[derive(Deserialize)]
struct Request {
    name: String,
}

/// This is a made-up example of what a response structure may look like.
/// There is no restriction on what it can be. The runtime requires responses
/// to be serialized into json. The runtime pays no attention
/// to the contents of the response payload.
#[derive(Serialize)]
struct Response {
    req_id: String,
    msg: String,
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    // required to enable CloudWatch error logging by the runtime
    // can be replaced with any other method of initializing `log`
    SimpleLogger::new().with_level(LevelFilter::Info).init().unwrap();

    let func = handler_fn(my_handler);
    lambda_runtime::run(func).await?;
    Ok(())
}

pub(crate) async fn my_handler(event: Request, ctx: Context) -> Result<Response, Error> {
    // extract some useful info from the request
    let name = event.name;

    // prepare the response
    let resp = Response {
        req_id: ctx.request_id,
        msg: format!("Hello {}!", name),
    };

    // return `Response` (it will be serialized to JSON automatically by the runtime)
    Ok(resp)
}
  • Step 4: Now, for the last step in Rust Lambda process, try running cargo build to test if everything is working.
pushd lambda/hello
cargo build
popd

Rust Lambda Process: Developing a Blueprint for Applications Infrastructure

Although the CDK is quite powerful and provides the capabilities to build big applications in AWS using high-level structures, you only need to create a basic Lambda function for your requirements.

  • Step 1: To begin the Rust Lambda process, you’ll need to install the Lambda library dependencies. Keep in mind that you generally don’t need to provide the version as I am doing here, but it may be useful to align all of your CDK library requirement versions and avoid incompatibilities.
npm i @aws-cdk/aws-lambda@1.107.0
  • Step 2: In the next step of the Rust Lambda process, you’ll create a Lambda function in the lib/tutorial-rust-lambda-stack.ts file that represents your CloudFormation stack.
import * as cdk from '@aws-cdk/core';
import * as lambda from '@aws-cdk/aws-lambda';

export class TutorialRustLambdaStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const target = 'x86_64-unknown-linux-musl';
    const hello = new lambda.Function(this, 'HelloHandler', {
      code: lambda.Code.fromAsset('lambda/hello', {
        bundling: {
          command: [
            'bash', '-c',
            `rustup target add ${target} && cargo build --release --target ${target} && cp target/${target}/release/hello /asset-output/bootstrap`
          ],
          image: cdk.DockerImage.fromRegistry('rust:1.52-slim')
        }
      }),
      functionName: 'hello',
      handler: 'main',
      runtime: lambda.Runtime.PROVIDED_AL2
    });
  }
}

You should be able to deploy the Lambda at this point in the Rust Lambda process. Because the docker image must be pulled (for the first time), the crates.io index must be changed, and the Lambda must be built, this command may take some time (only when you make changes). 

  • Step 3: The user will be prompted about IAM policy and statement changes as well. You can accept these, and your Lambda function will be live after the deployment is complete.
cdk deploy
  • Step 4: Using the AWS CLI, you can test the Lambda from the command line.
aws lambda invoke --function-name hello --payload '{"name":"Nick"}' output.json

You should now see a message if you look at the contents of output.json.

cat output.json
# {"req_id":"88226fb0-670c-4f0e-b772-7e677ccec71d","msg":"Hello Nick!"}

Rust Lambda Process: Adapting Lambda Into an API Endpoint

Follow the given instructions to adapt Lambda into an API Endpoint:

  • Step 1: You can begin with the Lambda change. First, under lambda/hello/Cargo.toml, you’ll need to change your dependencies again. You can get rid of the serde dependence and replace it with the following:
aws_lambda_events = "0.4.0"
http = "0.2.4"
  • Step 2: For the second step in the Rust Lambda process, you are going to be using aws-lambda-events, a good third-party tool that offers serializable Rust structs for the various AWS event definitions. Your Lambda code can now be updated. You will need some new use statements at the start of lambda/hello/src/main.rs to bring the API Gateway event definitions into scope, as well as some other things. You can also get rid of the use serde.
use aws_lambda_events::event::apigw::{ApiGatewayProxyRequest, ApiGatewayProxyResponse};
use aws_lambda_events::encodings::Body;
use http::header::HeaderMap;
  • Step 3: Remove the Request and Response struct definitions, and replace any references to them with ApiGatewayProxyRequest and ApiGatewayProxyResponse.
  • Step 4: Then in the Rust Lamda process, instead of returning your old Response, alter the my_handler method to produce an ApiGatewayProxyResponse. You can take the URL path from the request context and return it in the response body along with some content.
pub(crate) async fn my_handler(event: ApiGatewayProxyRequest, _ctx: Context) -> Result<ApiGatewayProxyResponse, Error> {
    // extract some useful info from the request
    let path = event.path.unwrap();

    // prepare the response
    let resp = ApiGatewayProxyResponse {
        status_code: 200,
        headers: HeaderMap::new(),
        multi_value_headers: HeaderMap::new(),
        body: Some(Body::Text(format!("Hello from '{}'", path))),
        is_base64_encoded: Some(false),
    };

    // return `Response` (it will be serialized to JSON automatically by the runtime)
    Ok(resp)
}

What Makes Data Ingestion from AWS Sources using Hevo’s Data Pipeline Best-In-Class

Loading data from AWS Sources such as AWS S3 and AWS Elasticsearch can be a mammoth task if the right set of tools is not leveraged. Hevo’s No-Code Automated Data Pipeline empowers you with a fully-managed solution for all your data collection, processing, and loading needs. Hevo’s native integration with S3 and Elasticsearch empowers you to transform and load data straight to a Data Warehouse such as Redshift, Snowflake, BigQuery & more!

This data loading lets you effortlessly connect to 100+ Sources (including 40+ free sources) and leverage Hevo’s blazing-fast Data Pipelines to help you seamlessly extract, transform, and load data to your desired destination such as a Data Warehouse.

Our platform has the following in store for you:

  • Data Transformations: Best-in-class & Native Support for Complex Data Transformation at fingertips. Code & No-code Fexibilty designed for everyone.
  • Smooth Schema Mapping: Fully-managed Automated Schema Management for incoming data with the desired destination.
  • Quick Setup: Hevo with its automated features, can be set up in minimal time. Moreover, with its simple and interactive UI, it is extremely easy for new customers to work on and perform operations.
  • Transformations: Hevo provides preload transformations to make your incoming data from AWS S3 and AWS Elasticsearch fit for the chosen destination. You can also use drag and drop transformations like Date and Control Functions, JSON, and Event Manipulation to name a few.
  • Live Support: The Hevo team is available round the clock to extend exceptional support to its customers through chat, email, and support calls.

Want to take Hevo for a spin? Sign up here for a 14-day free trial and experience the feature-rich Hevo.

  • Step 5: To create API Gateway resources, add a new dependency.
npm i @aws-cdk/aws-apigateway@1.107.0
  • Step 6: Then, at the start of your lib/tutorial-rust-lambda-stack.ts file, import that library.
import * as apigw from '@aws-cdk/aws-apigateway';
  • Step 7: Finally, when you’ve defined your Lambda function, you’ll create a new API Gateway Lambda Rest API resource. As the handler, you’ll feed it your Lambda function.
const gw = new apigw.LambdaRestApi(this, 'HelloEndpoint', {
  handler: hello
});
  • Step 8: You can now deploy your CDK application once more.
cdk deploy

You need to copy the API Gateway endpoint URL from the result of your CDK deploy this time to test. You should see the second last part labeled Outputs at the very end of the CDK command line output. There should be an output value for your URL in that section. For instance, it should seem as follows:

✅  TutorialRustLambdaStack

Outputs:
TutorialRustLambdaStack.HelloEndpointB03699DE = https://0448ubzy8d.execute-api.ca-central-1.amazonaws.com/prod/

Copy that URL, and you can send a request to it with any route you want, and your Lambda should respond with a response. You have completed the Rust Lambda Process.

curl -sL https://0448ubzy8d.execute-api.ca-central-1.amazonaws.com/prod/the/test/path
# Hello from '/the/test/path'

Conclusion

That’s all there is to it. You created an API Gateway to deliver traffic to your Lambda, which is running your Rust code, with literally one more line of code. You successfully ran Rust Lambda.

However, as a Developer, extracting complex data from a diverse set of data sources like Databases, CRMs, Project management Tools, Streaming Services, and Marketing Platforms to your Database can seem to be quite challenging. If you are from non-technical background or are new in the game of data warehouse and analytics, Hevo Data can help!

Visit our Website to Explore Hevo

Hevo Data will automate your data transfer process, hence allowing you to focus on other aspects of your business like Analytics, Customer Management, etc. This platform allows you to transfer data from 100+ multiple sources to Cloud-based Data Warehouses like Snowflake, Google BigQuery, Amazon Redshift, etc. It will provide you with a hassle-free experience and make your work life much easier.

Want to take Hevo for a spin? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite first hand.

You can also have a look at our unbeatable pricing that will help you choose the right plan for your business needs!

No-Code Data Pipeline for Your Data Warehouse