SNS Lambda Trigger Functions & Events: Simplified 101

AWS Lambda and Amazon SNS work seamlessly together, allowing you to trigger Lambda functions using Amazon SNS notifications. When a message is published to an SNS topic with a Lambda function subscribed to it, the function is invoked with the message payload. The Lambda function can process the information, forward it to other services, or publish the message to additional SNS topics.

Beyond message delivery status attributes, Amazon SNS now also supports Lambda message notifications. This integration provides delivery status support for Lambda destinations. This article explores AWS SNS and Lambda integration, detailing how to use triggers and events, along with a brief introduction to both services.

What is AWS Lambda?

AWS Lambda is a serverless compute service that lets you run code in response to events without the need to provision or manage servers. It integrates with over 200 AWS services and SaaS apps, and you only pay for the computing power you use.

The “serverless” model means you don’t need to worry about infrastructure. AWS Lambda takes care of everything, so you can focus on writing functions. These functions are self-contained applications, written in one of the supported languages, that you upload to Lambda for execution.

Lambda works seamlessly with other AWS services like API Gateway, DynamoDB, and RDS, and is a key component of AWS Serverless solutions. Its compatibility with a variety of languages and runtimes makes it an ideal choice for serverless developers.

Key Features of AWS Lambda

• Concurrency and Scaling Controls: Concurrency and scaling controls, such as concurrency limits and provisioned concurrency, allow you to fine-tune the scalability and reactivity of your production applications.
• Functions Defined as Container Images: To develop, test, and deploy your Lambda functions, it lets you use your favorite container image tooling, processes, and dependencies.
• Code Signing: Code signing for Lambda offers trust and integrity controls, allowing you to ensure that only unmodified code published by authorized developers is deployed in your Lambda services.
• Lambda Extensions: Lambda extensions may be used to enhance your Lambda functions. You can use extensions, for example, to make it easier to combine Lambda with your chosen tools for monitoring, observability, security, and governance.
• Blueprints of Function: A function blueprint contains sample code that demonstrates how to utilize Lambda in conjunction with other AWS services or third-party applications. Sample code and function setup settings for the Node.js and Python run-times are included in the blueprints.
• Database Accessibility: A database proxy controls a pool of database connections and transmits function queries. This allows a function to attain large levels of concurrency without draining database connections.
• Access to File Systems: A function can be configured to mount an Amazon Elastic File System (Amazon EFS) file system to a local directory. With Amazon EFS, your function code may securely and concurrently access and alter shared resources

What is Amazon SNS?

Amazon Simple Notification Service (SNS) is a cloud service that manages the delivery of push notifications from applications to subscribed endpoints and clients. To ensure message reliability, SNS stores all messages across multiple availability zones to prevent loss.

SNS allows you to send messages to various devices, including Windows, Google, and Apple, using an API or the AWS Management Console. Once a message is published, it can be sent to multiple recipients at once. To get started, developers create a “Topic” as an entry point for subscribers, who then receive notifications when a message is published to that topic.

SNS provides a flexible, fully managed messaging platform, enabling seamless message distribution to all subscribers in a publisher/subscriber setup.

Key Features of Amazon SNS

Some of the key features of Amazon SNS are as follows:

• Message Encryption: It offers Encrypted Topics to keep your messages safe from unauthorized and unknown access. By using Encryption Keys provided by AWS KMS, Server-Side Encryption protects the message contents stored in Amazon SNS Topics.
• Message Filtering: Each subscriber receives every message published to the Topic by default. A subscriber can assign a Filter Policy to the Topic subscription in order to receive a subset of the messages. The message is delivered to the subscribed endpoint when the incoming message attributes match the Filter Policy attributes. Otherwise, the message is discarded. 
• Message Archiving & Analytics: Amazon SNS allows the user to push notifications to additional Archiving and Analytics endpoints such as Amazon Simple Storage Service (Amazon S3) buckets, Amazon Redshift tables, and more

Understanding AWS SNS Lambda Integration

To understand AWS SNS Lambda Integration you need to thoroughly look at the topics below:

• Setting Up a DynamoDB Table
• Setting Up a Lambda Function
• Invoking the Lambda Function
• Subscribing to SNS Topics with Lambda Functions
• Publishing Messages Invoking the Lambda Function

1. Setting Up a Dynamodb Table

As a first step, create a table in DynamoDB named snslambda with the primary hash key SnsTopicArn and the primary range key SnsPublishTime. Then, with the addition of a global secondary index on SnsMessageId, the table can be queried with a SNS MessageId. Please refer to Amazon DynamoDB documentation for instructions on creating DynamoDB tables.

2. Setting Up a Lambda Function

Create the following Lambda function in the Lambda console and name it messageStore:

console.log('Loading event');
var aws = require('aws-sdk');
var ddb = new aws.DynamoDB({params: {TableName: 'snslambda'}});

exports.handler = function(event, context) {
  var SnsMessageId = event.Records[0].Sns.MessageId;
  var SnsPublishTime = event.Records[0].Sns.Timestamp;
  var SnsTopicArn = event.Records[0].Sns.TopicArn;
  var LambdaReceiveTime = new Date().toString();
  var itemParams = {Item: {SnsTopicArn: {S: SnsTopicArn},
  SnsPublishTime: {S: SnsPublishTime}, SnsMessageId: {S: SnsMessageId},
  LambdaReceiveTime: {S: LambdaReceiveTime} }};
  ddb.putItem(itemParams, function() {
    context.done(null,'');
  });
};

This Lambda function extracts the MessageId, Timestamp, and TopicArn from the SNS message it receives. (See the following for the exact SNS message structure). This data is then stored in the DynamoDB table you created earlier. Moreover, it also stores the time from the point of view of the Lambda function.

DynamoDB can also be accessed from Lambda functions by adding a policy to the lambda_exec_role in IAM. A sample policy looks like this:

{
  "Version": "2012-10-17",
  "Statement":[
    {
      "Sid":"Stmt1428510662000",
	 "Effect":"Allow",
      "Action":[
        "dynamodb:GetItem",
        "dynamodb:DeleteItem",
        "dynamodb:PutItem",
        "dynamodb:Scan",
        "dynamodb:Query",
        "dynamodb:UpdateItem",
        "dynamodb:BatchWriteItem",
        "dynamodb:BatchGetItem",
        "dynamodb:DescribeTable"
      ],
      "Resource":["arn:aws:dynamodb:us-east-1:123456789012:table/snslambda"]
    }
  ]
}

3. Invoking the Lambda Function

To enable SNS to invoke your Lambda function, you must assign a policy document to it once your Lambda function has been created. You can do this by calling the AddPermissions API. A policy document is passed as an argument to the AddPermissions API. The following is an example policy document for your Lambda function, messageStore, and lambda_topic:

{
  "Statement":[
    {"Condition":
      {"ArnLike":{"AWS:SourceArn":"arn:aws:sns:us-east-1:123456789012:lambda_topic"}},
      "Resource":"arn:aws:lambda:us-east-1:123456789023:function:messageStore",
      "Action":"lambda:invokeFunction",
      "Principal":{"Service":"sns.amazonaws.com"},
      "Sid":"sns invoke","Effect":"Allow"
    }],
  "Id":"default",
  "Version":"2012-10-17"
}

When using the Lambda console, the policy is automatically configured, making the Lambda function ready to subscribe to an SNS topic. However, if you’re sending messages from an opt-in region to a default-enabled region, the Lambda function’s policy must be updated to replace the principal sns.amazonaws.com with sns.opt-in-region.amazonaws.com.

For example, if you want to subscribe a Lambda function in US East (North Virginia) to an SNS topic in Asia Pacific (Hong Kong), change the principal to sns.ap-east-1.amazonaws.com. This change is required for regions launched after March 20, 2019, like Hong Kong, Bahrain, Milano, and Cape Town. Note that AWS Lambda doesn’t support message delivery between default-enabled regions or between opt-in regions.

4. Subscribing to SNS Topics with Lambda Functions

• Step 1: Navigate to the Topics page using the SNS console.
  • First, determine an SNS topic to which the Lambda function should be subscribed.
  • Next, create a new SNS topic if you don’t already have one by clicking the Create new topic button, filling in details such as the Topic name and Display name, and clicking Make a topic.
  • As an example, you will use lambda_topic, an SNS topic. After identifying or creating the SNS topic, click the ARN to open the Topic Details page.
  • A SNS topic can also be created and its ARN retrieved using the SNS APIs. An example is shown below:

String topicArn = 
  snsClient.createTopic(new CreateTopicRequest("lambda_topic" /*topic Name*/).getTopicArn();

• Step 2:  Click the Create subscription button on the Topic Details page.
  • Next, choose AWS Lambda from the Protocol drop-down list in the dialog box that appears. Click Create subscription after selecting the ARN of the Lambda function in the Endpoint field. 
  • A new subscription will be listed on your Topic Details page in the table of subscriptions associated with your SNS topic. ARN should be the Lambda function endpoint and Lambda as the protocol for this subscription.
  • Additionally, Lambda functions can subscribe to an SNS topic using SNS APIs. For instance:

String subscriptionArn = 
  snsClient.subscribe(new SubscribeRequest(topicArn,"lambda", 
    "arn:aws:lambda:us-east-1:123456789012:function:messageStore"));

5. Publishing Messages Invoking the Lambda Function

As soon as the Lambda function subscribes to an SNS topic, the next step is to publish a message to the topic and observe its execution via the Lambda console.

• First, you should go to the Topic Details page for the SNS topic whose Lambda function is subscribed to using the SNS console. 
• After clicking Publish to a topic, you will be directed to the Publish a message page. On this page, you can enter a subject, choose a message format, and enter the message payload for the Lambda function.
• Messages can be entered directly in the Message text area if Raw (same message body for all protocols) is selected.
• If you choose JSON, you need to encode the message according to the JSON schema that SNS expects. For assistance in converting your SNS message to the expected format, you can use the JSON message generator.
• You can now publish your message payload after you have finished creating it. By publishing to the SNS topic, the message will attempt to be delivered to all subscribers for that topic through SNS.
• This will invoke the Lambda function and pass a JSON formatted SNSEvent parameter as input as it attempts to dispatch a message to the subscribed Lambda function. An example SNSEvent is as follows:

{
  "Records":[
    {
      "EventSource":"aws:sns",
      "EventVersion": "1.0",
      "EventSubscriptionArn": "arn:aws:sns:us-east-1:123456789012:lambda_topic:0b6941c3-f04d-4d3e-a66d-b1df00e1e381",
      "Sns":{
        "Type": "Notification",
        "MessageId":"95df01b4-ee98-5cb9-9903-4c221d41eb5e",
	"TopicArn":"arn:aws:sns:us-east-1:123456789012:lambda_topic",
        "Subject":"TestInvoke",
	"Message":"<message payload>",
        "Timestamp":"2015-04-02T07:36:57.451Z",
	"SignatureVersion":"1",
	"Signature":"r0Dc5YVHuAglGcmZ9Q7SpFb2PuRDFmJNprJlAEEk8CzSq9Btu8U7dxOu++uU",
        "SigningCertUrl":"http://sns.us-east-1.amazonaws.com/SimpleNotificationService-d6d679a1d18e95c2f9ffcf11f4f9e198.pem",
	"UnsubscribeUrl":"http://cloudcast.amazon.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:123456789012:example_topic:0b6941c3-f04d-4d3e-a66d-b1df00e1e381",
	"MessageAttributes":{"key":{"Type":"String","Value":"value"}}
      }
    }
  ]
}

• When a Lambda function is invoked and begins to execute, you can view the results in the Lambda console.
• Furthermore, you should also see new records being added to your DynamoDB table for every message published to the SNS topic.
• Messages can also be published to an SNS topic to invoke a Lambda function subscribed to the topic. See the code given below:

snsClient.publish(topicArn, "example_message");

Conclusion 

This article discusses AWS SNS Lambda and explains how to use trigger functions and events in detail.

Hevo Data, a No-code Data Pipeline provides you with a consistent and reliable solution to manage data transfer between a variety of sources and a wide variety of Desired Destinations, with a few clicks. Hevo Data with its strong integration with 150+ sources (including 60+ free sources) allows you to not only export data from your desired data sources & load it to the destination of your choice, but also transform & enrich your data to make it analysis-ready so that you can focus on your key business needs and perform insightful analysis.

SIGN UP for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also have a look at the unbeatable pricing that will help you choose the right plan for your business needs.

FAQs