logo

Documentation

Introduction

Classes

Connectors

Deployment

AWS Lambda Connector

reshuffle-aws-connectors

Code | npm | Code sample

npm install reshuffle-aws-connectors

Reshuffle AWS Lambda Connector

This Reshuffle connector can be used to access AWS Lambda. It is implemented using Amazon's Lambda SDK.

The following example creates, invokes and deletes a simple Lambda function:

const crypto = require('crypto')
const { Reshuffle } = require('reshuffle')
const { AWSLambdaConnector } = require('reshuffle-aws-connectors')

;(async () => {
  const app = new Reshuffle()

  const awsLambdaConnector = new AWSLambdaConnector(app, {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    region: process.env.AWS_DEFAULT_REGION,
  })

  const funcName = `function-${crypto.randomBytes(8).toString('hex')}`

  console.log('Creating Lambda function:', funcName)
  await awsLambdaConnector.createFromCode(funcName, `
    exports.handler = async (event) => ({
      statusCode: 200,
      body: JSON.stringify(event),
    })
  `)

  const req = { foo: 'bar' }
  const res = await awsLambdaConnector.invoke(funcName, req)
  console.log('Lambda response:', req, '->', res)

  console.log('Deleting Lambda function')
  await awsLambdaConnector.delete(funcName)
})()

Table of Contents

Configuration Configuration options

Connector events:

queueComplete Queue processing is complete

Connector actions:

command Run a CLI command on Lambda

create Create a new Lambda function

createFromBuffer Create a new Lambda function from buffer

createFromCode Create a new Lambda function from code string

createFromFile Create a new Lambda function from file

createInFolder Create a new Lambda function by collecting files in a folder

delete Delete a Lambda function

enqueue Process a queue of tasks in Lambda functions

getFunctionInfo Get detailed function information

invoke Execute a Lambda function

listFunctions List deployed functions

updateCode Update Lambda function code

updateCodeFromBuffer Update Lambda function code from buffer

updateCodeFromCode Update Lambda function code from code string

updateCodeFromFile Update Lambda function code from file

updateCodeInFolder Update Lambda function code by collecting files in a folder

SDK:

sdk Get direct SDK access

Configuration options
const app = new Reshuffle()
const awsLambdaConnector = new AWSLambdaConnector(app, {
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  region: process.env.AWS_DEFAULT_REGION,
})

Connector events

Queue Complete event

Example:

awsLambdaConnector.on(
  { type: 'QueueComplete' },
  async (event, app) => {
    console.log(event.qid)
    console.log(event.payloads)
    console.log(event.resolutions)
  }
)

This event is fired when a processing queue has completed processing all its payloads. See the enqueue action for details.

Connector actions

Command action
(
  functionName: string,
  executable: string,
  command: string,
  files: string | string[] = [],
  options: Options = {},
) => any

Usage:

const mediaInfo = await awsLambdaConnector.command(
  'reshuffle-command-mediainfo',
  'https://<my-server>/mediainfo'
  `mediainfo --output=JSON <my-video>`,
  's3://<my-bucket>/<my-video>',
)

Use AWS Lambda to run a CLI command. This action invokes the Lambda function named functionName, loads the files with the specified urls into a temporary folder and runs the specified CLI command. The standard output from the CLI command is collected and returned to the user. At this point, the action does not support collection of output files generated by the CLI command.

URLs for executable and files can be either HTTP(S) URLS or S3 URLS. Note that the executable name in command must match the filename in the specified by the executable URL.

If needed, the action creates the Labda function and loads executable into the Lambda container. The executable needs to be compatible with the AWS Lambda Linux runtime environment. If a Lambda function with this name exists that was not previsouly deployed by the connector, deployment fails. If a Lambda exists that was deployed by the connector, then deployment is skipped unless options.force is true.

The options are the same as the ones used by create with the addition of the force flag mentioned above.

Create action

Definition:

interface Payload {
  code?: string
  filename?: string
  buffer?: Buffer
}

(
  functionName: string,
  payload: Payload,
  options: object = {},
) => object

Usage:

const functionInfo = await awsLambdaConnector.create(
  'toLowerUpperString',
  { code: `
    exports.handler = async (event) => {
      const str = event.str || 'Hello, world!'
      return {
        statusCode: 200,
        body: JSON.stringify({
          lower: str.toLowerCase(),
          upper: str.toUpperCase(),
        }),
      }
    }
  ` },
)

Create a new Lambda function with the given functionName. The code for the newly created function can be specified in one of three ways:

  • buffer a NodeJS buffer with zipped content (a package like JSZip can help)
  • code a string with code
  • filename a name of a file containing code

The optional options object may contain the following properties:

  • env - Environment variables
  • memorySize - Container memory size in MB (defaults to 256)
  • roleName - Function execution role (defaults to lambda_basic_execution)
  • runtime - Runtime environment (defaults to nodejs12.x)
  • tags - Tags
  • timeout - Execution timeout in seconds (default to 3)

The created function can be invoked using the invoke action, or tied to a myriad of AWS supported events.

We note that the AWS SDK provides many more options for creating and configuring Lambda functions. For example, functions can be deployed from an S3 zip file, allowing multiple files and dependencies to be deployed simultaneously. to leverage these capbilities, you can use the sdk action to gain direct access to the SDK and use its createFunction method directly.

Create From Buffer action

Definition:

(
  functionName: string,
  buffer: Buffer,
  options: object = {},
) => object

Usage:

const zip = new require('JSZip')()
// ... add files ...
const buffer = await zip.generateAsync({ type: 'nodebuffer' })
const functionInfo = await awsLambdaConnector.createFromBuffer(
  'toLowerUpperString',
  buffer,
)

Create a new Lambda function with the given functionName to execute the code in the specified buffer. See create above for more details.

Create From Code action

Definition:

(
  functionName: string,
  code: string,
  options: object = {},
) => object

Usage:

const functionInfo = await awsLambdaConnector.createFromCode(
  'toLowerUpperString',
  `
  exports.handler = async (event) => {
    const str = event.str || 'Hello, world!'
    return {
      statusCode: 200,
      body: JSON.stringify({
        lower: str.toLowerCase(),
        upper: str.toUpperCase(),
      }),
    }
  }
  `,
)

Create a new Lambda function with the given functionName to execute the code in code. See create above for more details.

Create From File action

Definition:

(
  functionName: string,
  filename: string,
  options: object = {},
) => object

Usage:

const functionInfo = await awsLambdaConnector.createFromFile(
  'toLowerUpperString',
  './toLowerUpperString.js',
)

Create a new Lambda function with the given functionName to execute the code inside the file filename. See create above for more details.

Create In Folder action

Definition:

(
  functionName: string,
  fileHandler: async (folder => Folder) => Promise<void>,
  options: object = {},
) => object

Usage:

const functionInfo = await awsLambdaConnector.createInFolder(
  'toLowerUpperString',
  async folder => {
    await folder.copy('index.js', `${__dirname}/toLowerUpperString.js`)
  }
)

Create a new Lambda function with the given functionName by setting up files in a folder. The folderHandler object receives a single folder object that provides the following methods:

  • copy(targetName: string, sourcePath: string): Promise Copy a file into the folder
  • exec(cmd: string): Promise<{ error?: Error, stdout: string | Buffer, stderr: string | Buffer }> Run a command inside the folder
  • contains(filename: string): Promise Check if the folder contains a file
  • write(targetName: string, data: string): Promise Create a file inside the folder with the specified data

See create above for more details.

Delete action

Definition:

(
  functionName: string,
) => void

Usage:

await awsLambdaConnector.delete('toLowerUpperString')

Delete the Lambda function with the name functionName. Be careful, this action is not reversible and will delete any function, not just ones created by this connector.

Enqueue action

Definition:

(
  functionName: string,
  payload: any|any[],
  maxConcurrent: number = 100,
) => string

Usage:

const qid = await awsLambdaConnector.enqueue(
  'toLowerUpperString',
  [
    { str: 'Alpha' },
    { str: 'Beta' },
    { str: 'Gamma' },
  ],
)

Asynchronously process a series of tasks with the Lambda function named functionName. The payload is an array of elements, each would be passed in turn as an input to the Lambda function. If payload is scalar, only a single invocation will ensue.

The maxConcurrent argument can be used to limit the number of simultaneous invocaions of the Lambda function, with a hard limit of 100 per queue. Currently the connector does not enforce a global limit on the number of functions it invokes through this action.

When all payloads have been processed, the action triggers a queueComplete event with the queue ID, the payloads array and a resolutions array in the event object. Each resolution is either the value returned by the Lambda function or an Error object if the invocation failed.

Get Function Info action

Definition:

(
  functionName: string,
) => any

Usage:

const info = await awsLambdaConnector.getFunctionInfo(
  'toLowerUpperString',
)

Get detailed information about the specified function.

Invoke action

Definition:

(
  functionName: string,
  requestPayload: any = {},
) => any

Usage:

const { lower, upper } = await awsLambdaConnector.invoke(
  'toLowerUpperString',
  { str: 'My Awesome String' },
)

Invoke the Lambda function with the name functionName, passing it the payload provided in requestPayload. The payload can be any JSON serializable JavaScript object.

The invoke action returns the response payload returned by the Lambda function. In case of an error during invocation or execution of the function, this action throws an error.

List Functions action

Definition:

() => any[]

Usage:

const list = await awsLambdaConnector.listFunctions()

Get information about deployed Lambda functions.

Update Code action

Definition:

(
  functionName: string,
  payload: Payload,
  options: object = {},
) => object

Usage:

const functionInfo = await awsLambdaConnector.updateCode(
  'toLowerUpperString',
  { code: `
    exports.handler = async (event) => {
      const str = event.str || 'Hello, beautiful world!!'
      return {
        statusCode: 200,
        body: JSON.stringify({
          lower: str.toLowerCase(),
          upper: str.toUpperCase(),
        }),
      }
    }
  ` },
)

Update the code of an existing Lambda function with the given functionName. The code can be specified in one of three ways, as defined in create above.

Update Code From Buffer action

Definition:

(
  functionName: string,
  buffer: Buffer,
) => object

Usage:

const zip = new require('JSZip')()
// ... add files ...
const buffer = await zip.generateAsync({ type: 'nodebuffer' })
const functionInfo = await awsLambdaConnector.updateCodeFromBuffer(
  'toLowerUpperString',
  buffer,
)

Update an existing Lambda function with the given functionName to execute the code in the specified buffer. See create above for more details.

Update Code From Code action

Definition:

(
  functionName: string,
  code: string,
) => object

Usage:

const functionInfo = await awsLambdaConnector.updateCodeFromCode(
  'toLowerUpperString',
  `
  exports.handler = async (event) => {
    const str = event.str || 'Hello, beautiful world!!'
    return {
      statusCode: 200,
      body: JSON.stringify({
        lower: str.toLowerCase(),
        upper: str.toUpperCase(),
      }),
    }
  }
  `,
)

Update an existing Lambda function with the given functionName to execute the code in code. See create above for more details.

Update Code From File action

Definition:

(
  functionName: string,
  filename: string,
) => object

Usage:

const functionInfo = await awsLambdaConnector.createFromFile(
  'toLowerUpperString',
  './toLowerUpperString.js',
)

Update an existing Lambda function with the given functionName to execute the code inside the file filename. See create above for more details.

Update Code In Folder action

Definition:

(
  functionName: string,
  fileHandler: async (folder => Folder) => Promise<void>,
) => object

Usage:

const functionInfo = await awsLambdaConnector.updateCodeInFolder(
  'toLowerUpperString',
  async folder => {
    await folder.copy('index.js', `${__dirname}/toLowerUpperString.js`)
  }
)

Update an existing Lambda function with the given functionName to execute the code set up in a folder. See createInFolder above for more details.

SDK

SDK action

Definition:

(
  options ?: object,
) => object

Usage:

const lambda = await awsLambdaConnector.sdk()

Get the underlying SDK object. You can specify additional options to override or add to the required fields in the connector's configuration.