SNS Lambda Trigger Functions & Events: Simplified 101

AWS Lambda and Amazon SNS integrate, allowing you to call Lambda functions via Amazon SNS notifications. Whenever a message is published to an SNS topic with a Lambda function subscribed to it, the Lambda function is invoked with the message payload. As an input parameter, the Lambda function receives the message payload and can manipulate the information within the message, publish the message to other SNS topics, or forward the message to another AWS service.

In addition to message delivery status attributes, Amazon SNS also supports Lambda message notifications. As a result, delivery status support has also been added for Lambda destinations.

This article talks about AWS SNS Lambda integration and how to use the trigger functions and events. In addition to that, it gives a brief introduction to AWS Lambda and AWS SNS.

What is AWS Lambda?

AWS Lambda is a serverless compute service that is triggered by events that allow you to run code for virtually any application or backend service without having to provision or manage servers. You can call Lambda from more than 200 AWS services and SaaS apps, and you only pay for what you use.

The term “serverless” computing refers to the fact that you don’t need to run these functions on your servers. AWS Lambda is a fully managed service that handles all of your infrastructure requirements.

AWS Lambda users write functions, which are self-contained applications written in one of the supported languages and runtimes, and upload them to AWS Lambda, which then executes them in a fast and flexible manner. 

It also integrates with a variety of other AWS services, including API Gateway, DynamoDB, and RDS, and forms the foundation for AWS Serverless solutions. Lambda is compatible with a wide range of popular languages and runtimes, making it a good fit for Serverless developers.

Key Features of AWS Lambda

• Concurrency and Scaling Controls: Concurrency and scaling controls, such as concurrency limits and provisioned concurrency, allow you to fine-tune the scalability and reactivity of your production applications.
• Functions Defined as Container Images: To develop, test, and deploy your Lambda functions, it lets you use your favorite container image tooling, processes, and dependencies.
• Code Signing: Code signing for Lambda offers trust and integrity controls, allowing you to ensure that only unmodified code published by authorized developers is deployed in your Lambda services.
• Lambda Extensions: Lambda extensions may be used to enhance your Lambda functions. You can use extensions, for example, to make it easier to combine Lambda with your chosen tools for monitoring, observability, security, and governance.
• Blueprints of Function: A function blueprint contains sample code that demonstrates how to utilize Lambda in conjunction with other AWS services or third-party applications. Sample code and function setup settings for the Node.js and Python run-times are included in the blueprints.
• Database Accessibility: A database proxy controls a pool of database connections and transmits function queries. This allows a function to attain large levels of concurrency without draining database connections.
• Access to File Systems: A function can be configured to mount an Amazon Elastic File System (Amazon EFS) file system to a local directory. With Amazon EFS, your function code may securely and concurrently access and alter shared resources

What is Amazon SNS?

Amazon Simple Notification Service (SNS) is a Cloud Service that coordinates the delivery of push notifications from software applications to subscribed Endpoints and Clients. To prevent message loss, all messages published to Amazon SNS are warehoused across multiple availability zones.

Amazon SNS enables users to push messages to Windows, Google, Apple, and certain Internet-connected smart devices by using an Application Programming Interface (API) or the AWS Management Console. Once a message is published to the service, it can be sent to multiple recipients. Service users can also utilize a single publish request to send direct messages to multiple devices or a single subscriber. To get started with Amazon SNS, developers must first create a Topic, which serves as an entry point for subscribers who want to receive notifications about a specific subject. When developers have an update for subscribers, they publish a message to a topic, and Amazon SNS distributes the message to all appropriate subscribers.

SNS is built on a flexible messaging platform that is fully managed and facilitates the publishing and subscribing of messages. It allows you to coordinate the delivery of messages to subscribers of your application.

It’s a way to send notifications from one service to another via a publisher/subscriber arrangement. Data about a topic is published and sent along by one service. The SNS will pass this along to all subscribers. The main focus here is on the topic. 

Key Features of Amazon SNS

Some of the key features of Amazon SNS are as follows:

• Message Encryption: It offers Encrypted Topics to keep your messages safe from unauthorized and unknown access. By using Encryption Keys provided by AWS KMS, Server-Side Encryption protects the message contents stored in Amazon SNS Topics.
• Message Filtering: Each subscriber receives every message published to the Topic by default. A subscriber can assign a Filter Policy to the Topic subscription in order to receive a subset of the messages. The message is delivered to the subscribed endpoint when the incoming message attributes match the Filter Policy attributes. Otherwise, the message is discarded. 
• Message Archiving & Analytics: Amazon SNS allows the user to push notifications to additional Archiving and Analytics endpoints such as Amazon Simple Storage Service (Amazon S3) buckets, Amazon Redshift tables, and more

Understanding AWS SNS Lambda Integration

To understand AWS SNS Lambda Integration you need to thoroughly look at the topics below:

• Setting Up a DynamoDB Table
• Setting Up a Lambda Function
• Invoking the Lambda Function
• Subscribing to SNS Topics with Lambda Functions
• Publishing Messages Invoking the Lambda Function

1. Setting Up a Dynamodb Table

As a first step, create a table in DynamoDB named snslambda with the primary hash key SnsTopicArn and the primary range key SnsPublishTime. Then, with the addition of a global secondary index on SnsMessageId, the table can be queried with a SNS MessageId. Please refer to Amazon DynamoDB documentation for instructions on creating DynamoDB tables.

2. Setting Up a Lambda Function

Create the following Lambda function in the Lambda console and name it messageStore:

console.log('Loading event');
var aws = require('aws-sdk');
var ddb = new aws.DynamoDB({params: {TableName: 'snslambda'}});

exports.handler = function(event, context) {
  var SnsMessageId = event.Records[0].Sns.MessageId;
  var SnsPublishTime = event.Records[0].Sns.Timestamp;
  var SnsTopicArn = event.Records[0].Sns.TopicArn;
  var LambdaReceiveTime = new Date().toString();
  var itemParams = {Item: {SnsTopicArn: {S: SnsTopicArn},
  SnsPublishTime: {S: SnsPublishTime}, SnsMessageId: {S: SnsMessageId},
  LambdaReceiveTime: {S: LambdaReceiveTime} }};
  ddb.putItem(itemParams, function() {
    context.done(null,'');
  });
};

This Lambda function extracts the MessageId, Timestamp, and TopicArn from the SNS message it receives. (See the following for the exact SNS message structure). This data is then stored in the DynamoDB table you created earlier. Moreover, it also stores the time from the point of view of the Lambda function.

DynamoDB can also be accessed from Lambda functions by adding a policy to the lambda_exec_role in IAM. A sample policy looks like this:

{
  "Version": "2012-10-17",
  "Statement":[
    {
      "Sid":"Stmt1428510662000",
	 "Effect":"Allow",
      "Action":[
        "dynamodb:GetItem",
        "dynamodb:DeleteItem",
        "dynamodb:PutItem",
        "dynamodb:Scan",
        "dynamodb:Query",
        "dynamodb:UpdateItem",
        "dynamodb:BatchWriteItem",
        "dynamodb:BatchGetItem",
        "dynamodb:DescribeTable"
      ],
      "Resource":["arn:aws:dynamodb:us-east-1:123456789012:table/snslambda"]
    }
  ]
}

3. Invoking the Lambda Function

To enable SNS to invoke your Lambda function, you must assign a policy document to it once your Lambda function has been created. You can do this by calling the AddPermissions API. A policy document is passed as an argument to the AddPermissions API. The following is an example policy document for your Lambda function, messageStore, and lambda_topic:

{
  "Statement":[
    {"Condition":
      {"ArnLike":{"AWS:SourceArn":"arn:aws:sns:us-east-1:123456789012:lambda_topic"}},
      "Resource":"arn:aws:lambda:us-east-1:123456789023:function:messageStore",
      "Action":"lambda:invokeFunction",
      "Principal":{"Service":"sns.amazonaws.com"},
      "Sid":"sns invoke","Effect":"Allow"
    }],
  "Id":"default",
  "Version":"2012-10-17"
}

The policy is automatically configured for you if you are using the Lambda console. The Lambda function is ready to subscribe to an SNS topic as soon as the policy is assigned.

It is essential to note that when using Amazon SNS to send messages from opt-in regions to regions that are enabled by default, the policy created in the AWS Lambda function must be altered to replace the principal sns.amazonaws.com with sns.opt-in-region.amazonaws.com. 

Using an example, if you desire to subscribe an AWS Lambda function in US East (North Virginia) to an SNS topic in the Asia Pacific (Hong Kong), you will need to change its principal to sns.ap-east-1.amazonaws.com in the AWS Lambda function’s policy. These regions are any newly launched regions that have decided to opt-in after March 20, 2019. At the time of this update, they include Asia Pacific (Hong Kong), Middle East (Bahrain), European Union (Milano), and Africa (Cape Town). The regions launched before March 20, 2019, are still enabled by default. 

Please note that delivery within an opt-in region from a region that is enabled by default to a region that is enabled by default is not supported by AWS Lambda. Moreover, SNS messages from one opt-in region to another opt-in region will not be forwarded to another opt-in region.

4. Subscribing to SNS Topics with Lambda Functions

• Step 1: Navigate to the Topics page using the SNS console.
  • First, determine an SNS topic to which the Lambda function should be subscribed.
  • Next, create a new SNS topic if you don’t already have one by clicking the Create new topic button, filling in details such as the Topic name and Display name, and clicking Make a topic.
  • As an example, you will use lambda_topic, an SNS topic. After identifying or creating the SNS topic, click the ARN to open the Topic Details page.
  • A SNS topic can also be created and its ARN retrieved using the SNS APIs. An example is shown below:

String topicArn = 
  snsClient.createTopic(new CreateTopicRequest("lambda_topic" /*topic Name*/).getTopicArn();

• Step 2:  Click the Create subscription button on the Topic Details page.
  • Next, choose AWS Lambda from the Protocol drop-down list in the dialog box that appears. Click Create subscription after selecting the ARN of the Lambda function in the Endpoint field. 
  • A new subscription will be listed on your Topic Details page in the table of subscriptions associated with your SNS topic. ARN should be the Lambda function endpoint and Lambda as the protocol for this subscription.
  • Additionally, Lambda functions can subscribe to an SNS topic using SNS APIs. For instance:

String subscriptionArn = 
  snsClient.subscribe(new SubscribeRequest(topicArn,"lambda", 
    "arn:aws:lambda:us-east-1:123456789012:function:messageStore"));

5. Publishing Messages Invoking the Lambda Function

As soon as the Lambda function subscribes to an SNS topic, the next step is to publish a message to the topic and observe its execution via the Lambda console.

• First, you should go to the Topic Details page for the SNS topic whose Lambda function is subscribed to using the SNS console. 
• After clicking Publish to a topic, you will be directed to the Publish a message page. On this page, you can enter a subject, choose a message format, and enter the message payload for the Lambda function.
• Messages can be entered directly in the Message text area if Raw (same message body for all protocols) is selected.
• If you choose JSON, you need to encode the message according to the JSON schema that SNS expects. For assistance in converting your SNS message to the expected format, you can use the JSON message generator.
• You can now publish your message payload after you have finished creating it. By publishing to the SNS topic, the message will attempt to be delivered to all subscribers for that topic through SNS.
• This will invoke the Lambda function and pass a JSON formatted SNSEvent parameter as input as it attempts to dispatch a message to the subscribed Lambda function. An example SNSEvent is as follows:

{
  "Records":[
    {
      "EventSource":"aws:sns",
      "EventVersion": "1.0",
      "EventSubscriptionArn": "arn:aws:sns:us-east-1:123456789012:lambda_topic:0b6941c3-f04d-4d3e-a66d-b1df00e1e381",
      "Sns":{
        "Type": "Notification",
        "MessageId":"95df01b4-ee98-5cb9-9903-4c221d41eb5e",
	"TopicArn":"arn:aws:sns:us-east-1:123456789012:lambda_topic",
        "Subject":"TestInvoke",
	"Message":"<message payload>",
        "Timestamp":"2015-04-02T07:36:57.451Z",
	"SignatureVersion":"1",
	"Signature":"r0Dc5YVHuAglGcmZ9Q7SpFb2PuRDFmJNprJlAEEk8CzSq9Btu8U7dxOu++uU",
        "SigningCertUrl":"http://sns.us-east-1.amazonaws.com/SimpleNotificationService-d6d679a1d18e95c2f9ffcf11f4f9e198.pem",
	"UnsubscribeUrl":"http://cloudcast.amazon.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:123456789012:example_topic:0b6941c3-f04d-4d3e-a66d-b1df00e1e381",
	"MessageAttributes":{"key":{"Type":"String","Value":"value"}}
      }
    }
  ]
}

• When a Lambda function is invoked and begins to execute, you can view the results in the Lambda console.
• Furthermore, you should also see new records being added to your DynamoDB table for every message published to the SNS topic.
• Messages can also be published to an SNS topic to invoke a Lambda function subscribed to the topic. See the code given below:

snsClient.publish(topicArn, "example_message");

Conclusion 

This article discusses AWS SNS Lambda and explains how to use trigger functions and events in detail.

visit our website to explore hevo

Hevo Data, a No-code Data Pipeline provides you with a consistent and reliable solution to manage data transfer between a variety of sources and a wide variety of Desired Destinations, with a few clicks. Hevo Data with its strong integration with 150+ sources (including 40+ free sources) allows you to not only export data from your desired data sources & load it to the destination of your choice, but also transform & enrich your data to make it analysis-ready so that you can focus on your key business needs and perform insightful analysis.

Want to take Hevo for a spin? Sign Up for a 14-day free trial and experience the feature-rich Hevo suite first hand. You can also have a look at the unbeatable pricing that will help you choose the right plan for your business needs.