How to run cron jobs (on a schedule) in AWS Lambda - Python version
In this article, we're going to discuss how to run the AWS Lambda function periodically or in a schedule using Python.
The typescript version of this article is available here.
As you may know, we need some form of the trigger to call the lambda. For example, you can upload an object to S3 and this would create an event which in turn would trigger the lambda.
Likewise, if you want to run a lambda function on a cron schedule - you can use the EventBridge service to create a rule which in turn will create a trigger on a schedule.
Primarily, there are 2 ways to create a schedule using EventBridge
- Rate expression
- Cron expression
Below is the high-level flow of triggering a lambda function on a schedule.
I'm going to use AWS CDK to create all the necessary resources. You can execute the below command to create a new AWS CDK application.
mkdir cron-lambda
cd cron-lambda
cdk init app --language=pythonIn the below code snippet, we're creating a lambda function
lambda_fn = _lambda.Function(self, 'lambda_fn',
runtime=_lambda.Runtime.PYTHON_3_8,
function_name='lambda_fn',
timeout=Duration.seconds(180),
memory_size=256,
code=_lambda.Code.from_asset(
'lambdas'),
handler='lambda.handler')The corresponding lambda code( lambda.py ) in lambdas folder is pasted below
def handler(event, context):
print('in python handler')
print(event)
There is nothing special with this lambda code. We're just printing the event statement. You can see this log in the cloudwatch service.
After creating the lambda function, we need to do 2 things
- Create event rule using either rate expression or cron expression
- Add lambda function as a target to the created event rule
Rate expressions
Rate expressions are a simpler format for representing schedules. The rate expression is a string value that follows the format
rate(value unit)
value should be a positive number whereas the unit can be any of the below values
- minute
- minutes
- hour
- hours
- day
- days
If the value is equal to 1, then the unit must be singular. If the value is greater than 1, the unit must be plural
Examples:
rate(1 minute) : This expression will create a schedule for every minute
rate(5 minute) : This expression will create a schedule for every 5 minutes
rate(1 hour) : This expression will create a schedule for every hour
rate(5 hours) : This expression will create a schedule for every 5 hours
rate(1 day) : This expression will create a schedule for everyday
rate(5 days) : This expression will create a schedule for every 5 days
In the below code snippet, we've created an event rule using rate expression - to create a schedule for every minute.
every_min_rule = events.Rule(self, 'every_min_rule',
schedule=events.Schedule.expression('rate(1 minute)'))
every_min_rule.add_target(events_targets.LambdaFunction(lambda_fn))You can deploy the stack using the below command
cdk deployOnce you deploy your stack, the lambda function will be executed every minute and you can see the logs for the same in cloudwatch.
Cron expressions
Cron expressions are most commonly used format in both legacy and modern systems for running background jobs and these cron expressions are being used in UNIX & Linux environments for quite some time.
Cron expression has below fields
minutes: This represents the minutes value and can be from 0-59
hours: This represents the hours value and can be from 0-23
day-of-month: This represents the day of the month and can be from 1-31
month: This represents the month of the year, either from 1-12 or from JAN-DEC
day-of-week: This represents the day of the week, either from 1-7 or from SUN-SAT
year: This represents the year
Restriction: We'll not be able to specify the value of day-of-month and day-of-week in the same expression. If you specify the value (or a * in one of these fields, you must specify ? in the other.
If you're using CDK, there are 2 ways to define cron expressions
- string format
- object format
Cron expression in string format:
This is the classical format and this string follows the below format
cron(minutes hours day-of-month month day-of-week year)
We've put * for every field except for the field day-of-week as per the above restriction
every_min_cron_rule = events.Rule(self, 'every_min_cron_rule',
schedule=events.Schedule.expression('cron(* * * * ? *)'))
every_min_cron_rule.add_target(
events_targets.LambdaFunction(lambda_fn))The above code will create a schedule for every minute. When you access the EventBridge service in AWS console, you'll be able to see the next 10 trigger timings.
Examples:
cron(0 0 * * ? *) : This expression will make the event trigger everyday at midnight, as we've specified the value for minutes and hours as 0.
cron(0 0 1 * ? *) : This expression will make the event trigger on 1st of every month at midnight. This expression is similar to the previous expression except we've mentioned day-of-month as 1
Cron expression in object format:
AWS CDK provides an additional option to express your cron expression - in object format. Instead of providing the cron expression in string format, which may be error-prone - you can provide it in object format.
AWS CDK provides an additional option to express your cron expression - in object format. Instead of providing the cron expression in string format, which may be error-prone - you can provide it in object format.
every_min_cron_event_rule = events.Rule(self, 'every_min_cron_event_rule',
schedule=events.Schedule.cron(
minute='*',
hour='*',
day='*',
month='*',
year='*',
))The fields of the cron expression that we discussed earlier become the properties of the object that we pass.
Below is the complete code
from aws_cdk import (
Duration,
Stack,
aws_lambda as _lambda,
aws_events as events,
aws_events_targets as events_targets,
)
from constructs import Construct
class CronLambdaStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
lambda_fn = _lambda.Function(self, 'lambda_fn',
runtime=_lambda.Runtime.PYTHON_3_8,
function_name='lambda_fn',
timeout=Duration.seconds(180),
memory_size=256,
code=_lambda.Code.from_asset(
'lambdas'),
handler='lambda.handler')
every_min_rule = events.Rule(self, 'every_min_rule',
schedule=events.Schedule.expression('rate(1 minute)'))
every_min_rule.add_target(events_targets.LambdaFunction(lambda_fn))
every_min_cron_rule = events.Rule(self, 'every_min_cron_rule',
schedule=events.Schedule.expression('cron(* * * * ? *)'))
every_min_cron_rule.add_target(
events_targets.LambdaFunction(lambda_fn))
every_min_cron_event_rule = events.Rule(self, 'every_min_cron_event_rule',
schedule=events.Schedule.cron(
minute='*',
hour='*',
day='*',
month='*',
year='*',
))
every_min_cron_event_rule.add_target(
events_targets.LambdaFunction(lambda_fn))
Let me know your thoughts on the comments.