Using AWS Step Functions To Schedule Or Delay SNS Message Publication

with no AWS Lambda function required

A co-worker at Archer asked if there was a way to schedule messages
published to an Amazon SNS topic.

I know that scheduling messages to SQS queues is possible to some
extent using the DelaySeconds message timer, which
allows postponing visibility in the queue up to 15 minutes, but SNS
does not currently have native support for delays.

However, since AWS Step Functions has built-in integration with SNS,
and since it also has a Wait state that can schedule or delay
execution, we can implement a fairly simple Step Functions state
machine that puts a delay in front of publishing a message to an SNS
topic, without any AWS Lambda code.

Overview

This article uses an AWS CloudFormation template to create a sample
AWS stack with one SNS topic and one Step Functions state machine with
two states.

AWS architecture diagram

This is the CloudFormation template, if you’d like to review it:

CloudFormation template: aws-sns-delayed

Here is the Step Functions state machine definition from the above
CloudFormation template:

{ "StartAt": "Delay", "Comment": "Publish to SNS with delay", "States": { "Delay": { "Type": "Wait", "SecondsPath": "$.delay_seconds", "Next": "Publish to SNS" }, "Publish to SNS": { "Type": "Task", "Resource": "arn:aws:states:::sns:publish", "Parameters": { "TopicArn": "${SNSTopic}", "Subject.$": "$.subject", "Message.$": "$.message" }, "End": true } }
}

The “Delay” state waits for “delay_seconds” provided in the input to
the state machine execution (as we’ll see below).

The “Publish to SNS” task uses the Step Functions integration with SNS
to call the publish API directly with the parameters listed, some of
which are also passed in to the state machine execution.

Now let’s take it for a spin!

Preparation

Clone the demo repo from GitHub to get the CloudFormation template:

git clone git@github.com:alestic/aws-sns-delayed.git
cd aws-sns-delayed

Select your email address and a stack name for this demo:

email=YOUREMAIL@example.com
stack_name=sns-delayed-demo

Launch Stack

Deploy the CloudFormation stack:

stack_parameters="NotificationEmail=$email"
aws cloudformation deploy \ --stack-name "$stack_name" \ --template-file template.yaml \ --capabilities CAPABILITY_IAM \ --parameter-overrides "$stack_parameters"

The deploy command can be re-run after you edit template.html and
want to deploy the changes.

Get the Step Function State Machine and SNS topic from the running
stack outputs:

state_machine=$(aws cloudformation describe-stacks \ --stack-name "$stack_name" \ --output text \ --query 'Stacks[][Outputs][][?OutputKey==`StepFunctionsStateMachine`][OutputValue]')
echo state_machine=$state_machine sns_topic=$(aws cloudformation describe-stacks \ --stack-name "$stack_name" \ --output text \ --query 'Stacks[][Outputs][][?OutputKey==`SNSTopic`][OutputValue]')
echo sns_topic=$sns_topic

Publish Messages

Before you go any further, make sure you go to your email and confirm
your subscription to the SNS topic. If you don’t do this, you won’t
get to see any of the test messages published to the topic below.

Test 1: Post directly to the SNS Topic, circumventing the Step Functions state
machine:

aws sns publish \ --topic-arn "$sns_topic" \ --subject "Test message published directly to SNS topic" \ --message "hello, world"

Test 2: Post through the Step Functions state machine, with a delay specified
in seconds:

execution_input='{ "delay_seconds": 60, "subject": "Test message delayed through AWS Step Functions", "message": "hello, world"
}' aws stepfunctions start-execution \ --state-machine-arn "$state_machine" \ --input "$execution_input"

That’s it! Check your email for the test messages.

You can also go to the AWS Step Functions
Console and watch your state machjine executing
in real time. Here is a graphic representation of the state machine as
displayed in the AWS Console, shown while an execution is in the Delay
state:

AWS Step Functions state machine

Cleanup

Once you have finished playing with this sample stack, you can delete it:

aws cloudformation delete-stack \ --stack-name "$stack_name"

This cleans up everything… eventually.

If you have created Step Functions state machine executions that have
very long delays, then the state machine will be put in a “DELETING”
status, and won’t actually disappear until every execution has entered
its next state transition.

If this doesn’t feel clean to you, it is possible to manually stop
each running execution, allowing the state machine to finally be
deleted.

Adaptations

Absolute Timestamps

The above example uses the SecondsPath field in the Wait state,
which delays for the specified number of seconds. To instead schedule
the message for publishing at a specific date/time in the future,
replace the line:

"SecondsPath": "$.delay_seconds",

with:

"TimestampPath": "$.delay_timestamp",

and change the execution_input to include the absolute timestamp
when you want the SNS message to be published, like so:

execution_input='{ "delay_timestamp": "2020-01-01T00:00:00Z", "subject": "Happy New Year!", "message": "Peace to you and yours in this new year."
}'

I considered adding extra Step Function states to detect whether a
relative or absolute delay was specified in the input, then branching
to the appropriate type of Wait state, but adding 50% in Step
Functions cost didn’t seem worth it.

Other SNS Message Formats

This example publishes a simple text message to the SNS topic along
with a Subject value to make emails look prettier.

SNS publish supports other message types including
structured messages where different formatted values are sent to
different types of subscribers. You can modify the “Publish to SNS”
task parameters in the state machine definition to fit your
application’s needs.

Other SNS Topics

This sample CloudFormation template creates its own SNS topic, but you
could remove it and parameterize the ARN in the template to put a
delay in front of another, pre-existing SNS topic.

You could even add an SNS topic ARN parameter to the Step Functions
state machine, so that you can schedule or delay messages to arbitrary
SNS topics. In order for this to work, though, you would need to
change the IAM role so that the state machine can publish to those
topics.

Delaying Other AWS Activities

SNS is not the only resource with built-in AWS Step Functions
integration support. You can use this same approach to schedule or
delay operations with DynamoDB, AWS Batch, Amazon ECS, Fargate, SQS,
AWS Glue, SageMaker, and of course, AWS Lambda.

CAVEATS

Size – The message limit for SNS is currently 256 KB. The message
limit for Step Functions is currently 32 KB, so you can’t
use this approach for all SNS applications.

Cost – A million SNS publishes currently costs about $0.25
depending on the region, plus delivery charges, if any. A million Step
Function state transitions currently costs $25, but this state machine
uses 2 state transitions per post, so the resulting cost will be
around $50 for a million delayed/scheduled SNS messages.

Time – Step Function state machine executions must complete within 1
year, including any Wait steps, so this method cannot be used to
schedule SNS posts more than a year in advance.

Limits tend to increase and costs tend to decrease over time with AWS,
so look up the current values before making decisions.

AWS Console “Task Timer” Sample Project

The AWS Step Functions Developer Guide describes a Task
Timer project that is available in the AWS Console Step
Functions Sample Projects.

The Task Timer sample state machine uses an AWS Lambda function, so
it’s a great example for how to drive AWS Lambda with Step Functions.

However, since Step Functions recently announced the ability to
integrate directly with SNS, we don’t need to complicate this
functionality with Lambda code. Step Functions can do all the work
with no function to write or maintain, as demonstrated above.

Original article and comments: https://alestic.com/2019/05/aws-delayed-sns-step-functions/

Spread the love

Posted by News Monkey

blank