This is mainly for node.js but might apply to other environments. Unsure.
If you are running a AWS Lambda function that calls another AWS service and getting an error about invalid tokens or other access denied errors, do this:
The role assigned to your lambda function will need permission to perform the actions. Check IAM and make sure the role has all the permissions.
I usually give Full Access to the service in question (e.g. DynamoDB) if I suspect a problem here. I then work backward toward the granular rules. (Sometimes services require more permissions than you expect.)
Example below. Check the logs for results.
module.exports.handler = function(event, context) {
// incoming data ok?
console.log('event', JSON.stringify(event, null, 2));
// env vars ok? (mostly set by AWS lambda, but might have some of yours)
console.log('env', JSON.stringify(process.env, null, 2));
// anything weird with the AWS service instance?
console.log('dynamodb client', JSON.stringify(db._dynamodbClient));
// what about the service config?
console.log('dynamodb service', JSON.stringify(db._dynamodbClient.service, null, 2));
// ...This has bit me a couple times now and the solution isn't immediately obvious. Lambda automatically generates temporary aws credentials for the lambda function to use. These temporary credentials get added to process.env by Lambda
For some reason though, if you try to set these manually when creating a dynamodb client for example, it won't work.
It seems like best practice here is to always store credentials in environment and let the AWS sdk detect them.
Example:
This will work on local but fail on lambda:
var dynamodb = new AWS.DynamoDB.DocumentClient({
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: process.env.AWS_DEFAULT_REGION,
});This works on lambda AND local (as long as your env vars are named properly):
var dynamodb = new AWS.DynamoDB.DocumentClient({ region: process.env.AWS_DEFAULT_REGION });I'm not exactly sure why this works, but even moving that creation inside the handler scope doesn't fix it, so it doesn't seem to be a race with client creation and process.env being ready.