How to run cron jobs (on a schedule) in AWS Lambda

How to run cron jobs (on a schedule) in AWS Lambda

In this article, were going to discuss about how to run AWS Lambda function periodically or in a schedule.

As you may know, we need some form of trigger to call the lambda. For example, you can upload an object to S3 and this would create an event which in turn would trigger the lambda.

Likewise, if you want to run a lambda function on a cron schedule - you can use EventBridge service to create a rule which in turn will create trigger on a schedule.

Primarily, there are 2 ways to create a schedule using EventBridge

  • Rate expression
  • Cron expression

Below is the high level flow of triggering a lambda function on a schedule.

I'm going to use AWS CDK to create all the necessary resources. You can execute below command to create new AWS CDK application.

mkdir cron-lambda
cd cron-lambda
cdk init app --language=typescript
Creating AWS CDK application

In the below code snippet, we're creating a lambda function

export class AwsCdkCronLambdaStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const nodeJsFnProps: NodejsFunctionProps = {
      bundling: {
        externalModules: [
          'aws-sdk', // Use the 'aws-sdk' available in the Lambda runtime
        ],
      },
      runtime: Runtime.NODEJS_16_X,
    };

    const rateExprLambda = new NodejsFunction(this, 'rateExprLambda', {
      entry: path.join(__dirname, '../lambdas', 'rate-expr-lambda.ts'),
      ...nodeJsFnProps,
      functionName: 'rateExprLambda',
    });
   }
  }

The corresponding lambda code( rate-expr-lambda.ts )  in lambdas folder is pasted below

export const handler = async (event: any = {}): Promise<any> => {
  console.log(
    'This handler will run on a schedule rate expression:',
    JSON.stringify(event)
  );
};

There is nothing special with this lambda code. We're just printing a statement (along with the stringified event) in console. You can see this log in cloudwatch service.

After creating lambda function, we need to do 2 things

  • Create event rule using either rate expression or cron expression
  • Add lambda function as target to the created event rule

Rate expressions

Rate expressions are simpler format for representing schedules. The rate expression is a string value which follows the format

rate(value unit)

value should be a positive number whereas the unit can be any of the below values

  • minute
  • minutes
  • hour
  • hours
  • day
  • days

If the value is equal to 1, then the unit must be singular. If the value is greater than 1, the unit must be plural

Examples:

rate(1 minute) :  This expression will create a schedule for every minute

rate(5 minute) :  This expression will create a schedule for every 5 minutes

rate(1 hour) :  This expression will create a schedule for every hour

rate(5 hours) :  This expression will create a schedule for every 5 hours

rate(1 day) :  This expression will create a schedule for every day

rate(5 days) :  This expression will create a schedule for every 5 days

In the below code snippet, we've created event rule using rate expression - to create schedule for every minute.

    const everyMinRateER = new events.Rule(this, 'everyMinRateER', {
      schedule: events.Schedule.expression('rate(1 minute)'),
    });

    everyMinRateER.addTarget(new targets.LambdaFunction(rateExprLambda));

You can deploy the stack using the below command

cdk deploy

Once you deploy your stack, the lambda function will be executing every minute and you can see the logs for the same in cloudwatch.

Cron expressions

Cron expressions is most commonly used format in both legacy and modern systems for running background jobs and this cron expressions are being used in UNIX & Linux environments for quite some time.

Cron expression has below fields

minutes: This represents the minutes value and can be from 0-59

hours: This represents the hours value and can be from 0-23

day-of-month: This represents the day of the month and can be from 1-31

month: This represents the month of the year, either from 1-12 or from JAN-DEC

day-of-week: This represents the day of the week, either from 1-7 or from SUN-SAT

year: This represents the year

Restriction: We'll not be able to specify the value of  day-of-month and day-of-week in the same expression. If you specify the value (or a * in one of these fields, you must specify ? in the other.

If you're using CDK, there are 2 ways to define cron expressions

  • string format
  • object format

Cron expression in string format:

This is the classical format and this string follows the below format

cron(minutes hours day-of-month month day-of-week year)

We've put * for every field except for the field day-of-week as per the above restriction

  const everyMinCronER = new events.Rule(this, 'everyMinCronER', {
      schedule: events.Schedule.expression('cron(* * * * ? *)'),
    });

  everyMinCronER.addTarget(new targets.LambdaFunction(lambda));

Above code will create schedule for every minute. When you access EventBridge service in aws console, you'll be able to see the next 10 trigger timings.

Examples:

cron(0 0 * * ? *) : This expression will make the event to trigger everyday at midnight, as we've specified the value for minutes and hours as 0.

cron(0 0 1 * ? *) : This expression will make the event to trigger at 1st of every month at midnight. This expression is similar to the previous expression except we've mentioned day-of-month as 1

Cron expression in object format:

AWS CDK provides additional option to express your cron expression - in object format. Instead of providing the cron expression in string format, which may be error prone - you can provide it in object format.

 const everyMinCronER = new events.Rule(this, 'everyMinCronER', {
      schedule: events.Schedule.cron({
        minute: '*',
        hour: '*',
        day: '*',
        month: '*',
        year: '*',
      }),
    });

   everyMinCronER.addTarget(new targets.LambdaFunction(lambda));

The fields of the cron expression that we've discussed earlier become the properties of the object that we pass.

Below is the complete code

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as path from 'path';
import {
  NodejsFunction,
  NodejsFunctionProps,
} from 'aws-cdk-lib/aws-lambda-nodejs';
import * as targets from 'aws-cdk-lib/aws-events-targets';
import * as events from 'aws-cdk-lib/aws-events';
import { Runtime } from 'aws-cdk-lib/aws-lambda';

export class AwsCdkCronLambdaStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const nodeJsFnProps: NodejsFunctionProps = {
      bundling: {
        externalModules: [
          'aws-sdk', // Use the 'aws-sdk' available in the Lambda runtime
        ],
      },
      runtime: Runtime.NODEJS_16_X,
    };

    const lambda = new NodejsFunction(this, 'rateExprLambda', {
      entry: path.join(__dirname, '../lambdas', 'lambda.ts'),
      ...nodeJsFnProps,
      functionName: 'rateExprLambda',
    });

    const everyMinRateER = new events.Rule(this, 'everyMinRateER', {
      schedule: events.Schedule.expression('rate(1 minute)'),
    });

    everyMinRateER.addTarget(new targets.LambdaFunction(lambda));

    const everydayCronER = new events.Rule(this, 'everydayCronER', {
      schedule: events.Schedule.expression('cron(0 0 * * ? *)'),
    });

    everydayCronER.addTarget(new targets.LambdaFunction(lambda));

    const everyMinCronER = new events.Rule(this, 'everyMinCronER', {
      schedule: events.Schedule.cron({
        minute: '*',
        hour: '*',
        day: '*',
        month: '*',
        year: '*',
      }),
    });

    everyMinCronER.addTarget(new targets.LambdaFunction(lambda));
  }
}

Let me know your thoughts on the comments.