Skip to content

Instantly share code, notes, and snippets.

@cmawhorter
Last active February 28, 2024 16:28
Show Gist options
  • Star 20 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save cmawhorter/3cf67164414c4a97fcd8 to your computer and use it in GitHub Desktop.
Save cmawhorter/3cf67164414c4a97fcd8 to your computer and use it in GitHub Desktop.
Solution to AWS Lambda node.js UnrecognizedClientException "The security token included in the request is invalid."

Troubleshooting AWS unauthorized errors in lambda requests

This is mainly for node.js but might apply to other environments. Unsure.

If you are running a AWS Lambda function that calls another AWS service and getting an error about invalid tokens or other access denied errors, do this:

Check IAM

The role assigned to your lambda function will need permission to perform the actions. Check IAM and make sure the role has all the permissions.

I usually give Full Access to the service in question (e.g. DynamoDB) if I suspect a problem here. I then work backward toward the granular rules. (Sometimes services require more permissions than you expect.)

Verify your settings and incoming data

Example below. Check the logs for results.

module.exports.handler = function(event, context) {
  // incoming data ok?
  console.log('event', JSON.stringify(event, null, 2));
  // env vars ok? (mostly set by AWS lambda, but might have some of yours)
  console.log('env', JSON.stringify(process.env, null, 2));
  // anything weird with the AWS service instance?
  console.log('dynamodb client', JSON.stringify(db._dynamodbClient));
  // what about the service config?
  console.log('dynamodb service', JSON.stringify(db._dynamodbClient.service, null, 2));
  // ...

Beware env credentials

This has bit me a couple times now and the solution isn't immediately obvious. Lambda automatically generates temporary aws credentials for the lambda function to use. These temporary credentials get added to process.env by Lambda

For some reason though, if you try to set these manually when creating a dynamodb client for example, it won't work.

It seems like best practice here is to always store credentials in environment and let the AWS sdk detect them.

Example:

This will work on local but fail on lambda:

var dynamodb = new AWS.DynamoDB.DocumentClient({
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  region: process.env.AWS_DEFAULT_REGION,
});

This works on lambda AND local (as long as your env vars are named properly):

var dynamodb = new AWS.DynamoDB.DocumentClient({ region: process.env.AWS_DEFAULT_REGION });

I'm not exactly sure why this works, but even moving that creation inside the handler scope doesn't fix it, so it doesn't seem to be a race with client creation and process.env being ready.

Edit: @epiphone might have the reason in his comment below. this could be session token being required in prod but not local. e.g. adding sessionToken: process.env.AWS_SESSION_TOKEN to your params. NOTE: if you're using cognito with an assumed identity this might also be the issue since cognito requests require the session token IIRC.

AWS.config.update()

I see a lot of people still landing on this gist and referencing AWS.config.update(). DO NOT USE THIS. ANYWHERE. It's cancer. Instead use the constructor of the service to pass this info in. e.g. new AWS.Lambda({ region: '...' })

The reason it's horrible is it's a race condition built directly into the mediocre (and that's being generous) aws-sdk js v2 client.

update() only impacts things created after it is called and not any clients created before. And then (unless it's changed) it's still possible to override these settings directly on the constructor. tl;dr it's cancer and it should be avoided.

@forzagreen
Copy link

I still have the problem.
My lambda works correctly for a moment (sometimes 2 hours, sometimes 10 min), then it stops again with the error message:
[UnrecognizedClientException: The security token included in the request is invalid]
In another lambda (with less requests to DynamoDB), it's always working well.

@galxy25
Copy link

galxy25 commented Feb 24, 2017

To resolve issues with not being able to specify aws credentials for the client(like if you want to assume a role to call cross account)

var creds = new AWS.Credentials({accessKeyId: access_id, secretAccessKey: access_key, sessionToken: session_token});
var lambda = new AWS.Lambda({credentials: creds, signatureCache: false});

@ugenlik
Copy link

ugenlik commented May 31, 2017

wow alexa skill kit and lambda was working on my local but not on lambda following your advise , I have changed my dynamodb setting from

AWS.config.update({
        credentials: {
            accessKeyId: process.env.AWS_ACCESS_KEY_ID,
            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
        },
        region: process.env.AWS_DEFAULT_REGION
    });

to

AWS.config.update({
        region: process.env.AWS_DEFAULT_REGION
    });

and it worked. took me hours to find this post , thanks

@Tqader
Copy link

Tqader commented Jan 10, 2018

I am using :

AWS.config.update({
region: process.env.AWS_DEFAULT_REGION
});

Gut still getting UnrecognizedClientException "The security token included in the request is invalid." periodically. Even tried setting the DynamoDB retry options as:
var dynamodb = new AWS.DynamoDB({maxRetries: 5, retryDelayOptions: {base: 2000} });
var docClient = new AWS.DynamoDB.DocumentClient({service : dynamodb});

Seems like the Lambda updates the temporary aws credentials periodically and the AWS.DynamoDB.DocumentClient calls fail as this is happening. Has somebody run into this situation and what was the fix? All help will be appreciated.

@tfrancois
Copy link

Has anyone solved this issue? I'm having the same problem as well. Can't access my DynamoDB.DocumentClient in Lambda even though I've made sure the correct execution role policy is set in the Lambda console. The issue is driving me bonkers.

@epiphone
Copy link

This will work on local but fail on lambda:
var dynamodb = new AWS.DynamoDB.DocumentClient({
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: process.env.AWS_DEFAULT_REGION,
});

Try adding sessionToken: process.env.AWS_SESSION_TOKEN to the DocumentClient constructor params.

@cmawhorter
Copy link
Author

thanks @epiphone! added your note to the doc since that's likely the root cause. cool, thanks!

@zhaozhongming
Copy link

I have a lambda which accesses the Neptune , same error, if I switch to hardcode my admin key and secret, it works, however when I use the key and secret provided by lambda (process.env.*), it says The security token included in the request is invalid, I am sure I have all the policy attached to the lambda execute role, I am stuck now.

@alex-hladun
Copy link

Thank you so much! Trying to over-configure with AWS.Config.Update when i should've just let it do its thing

@zach-sim
Copy link

This, as the second result on my google search was so much more helpful than the first result in solving my bug.

image

@alnaranjo
Copy link

I've been trying to fix this for days. Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment