Skip to content

Instantly share code, notes, and snippets.

@dmattia
Last active March 19, 2024 16:35
Show Gist options
  • Star 13 You must be signed in to star a gist
  • Fork 11 You must be signed in to fork a gist
  • Save dmattia/0d17696bad1dffd90ec7c899e0343955 to your computer and use it in GitHub Desktop.
Save dmattia/0d17696bad1dffd90ec7c899e0343955 to your computer and use it in GitHub Desktop.
A less verbose terragrunt
/**
* Wrapper around terragrunt to display output succinctly on Atlantis.
*
* Terragrunt is notoriously verbose, which can cause Atlantis to output
* hundreds of comments on single PRs, which can be annoying.
*
* This script will output just the final plan for resources to update on
* successful terragrunt runs, but will output all terragrunt output on
* errors.
*/
const shell = require('shelljs');
const path = require('path');
const { PLANFILE } = process.env;
const logger = console;
/**
* A map of terraform field names to mask the output of to a Github
* Issue explaining why that field is masked
*/
const maskMap = {
// Sensitive values from aws_cognito_identity_provider
client_id:
'https://github.com/terraform-providers/terraform-provider-aws/issues/9934',
client_secret:
'https://github.com/terraform-providers/terraform-provider-aws/issues/9934',
};
/**
* A map of patterns for terragrunt modules that we would expect to fail to a Github
* Issue explaining why those modules are expected to fail.
*
* When these modules fail, we will display a message clearly stating that
* this is an expected behavior
*/
const expectedFailingModules = {
'some/terragrunt/module':
'https://github.com/some-org/some-repo/issues/1234',
};
/**
* Masks any blocklisted field names in the terraform output.
*
* Ideally, PRs would be sent to mark these fields as sensitive in
* the terraform provider itself, but this works as a temporary measure
* while fields those PRs are in review
*
* @param output - The original plan output
* @returns the plan output with sensitive values removed
*/
function maskSensitiveValues(output) {
return Object.keys(maskMap).reduce(
(out, fieldName) =>
out.replace(
new RegExp(`("${fieldName}" *=) ".*"`, 'g'),
(_, match) =>
`${match} This field is sensitive and cannot be shown in PRs`,
),
output,
);
}
/**
* Promisifies shelljs.exec
*
* @param {string} command - Command to execute in the local shell
* @returns The resolved command
*/
async function run(command) {
return new Promise((resolve) => {
shell.exec(command, { silent: true }, (code, stdout, stderr) => {
resolve({ code, stdout, stderr });
});
});
}
/**
* Runs a plan via terragrunt. Output is only shown on error
*/
async function runPlan() {
const wasExpectedToFail = Object.keys(
expectedFailingModules,
).some((pattern) => new RegExp(pattern).test(shell.pwd()));
if (wasExpectedToFail) {
logger.log(
`Atlantis does not currently support the module in ${shell.pwd()}. Please run this module locally`,
);
shell.touch(PLANFILE);
return;
}
const { code, stderr } = await run(
`terragrunt plan -no-color -out=${PLANFILE}`,
);
if (code !== 0) {
logger.log(stderr);
throw Error(`Failed to run plan in ${shell.pwd()}`);
}
}
/**
* Prints a representation of the terraform plan output to the console
*/
async function printPlanFile() {
const { dir, base } = path.parse(PLANFILE);
shell.cd(dir);
const { stdout } = await run(`terragrunt show -no-color ${base}`);
logger.log(maskSensitiveValues(stdout));
}
/**
* Runs an apply via terragrunt. Output is only shown on error
*/
async function runAndPrintApply() {
const { code, stdout, stderr } = await run(
`terragrunt apply -no-color ${PLANFILE}`,
);
if (code !== 0) {
logger.log(stderr);
throw Error(`Failed to run apply in ${shell.pwd()}`);
} else {
logger.log(stdout);
}
}
/**
* Main function
*/
async function main() {
const args = process.argv.slice(2);
const command = args[0];
if (command.toString().trim() === 'apply') {
await runAndPrintApply();
} else {
await runPlan();
await printPlanFile();
}
}
/**
* Run the program, exiting with a status code of 1 on any error
*/
main().catch((err) => {
logger.error(err);
process.exit(1);
});
@rverma-jm
Copy link

hey can you share how you used this. Is this something you included in repo.yaml?

@dmattia
Copy link
Author

dmattia commented Apr 15, 2020

Yep, we use it in repo.yaml.

Our plan step is overridden to have run="node ~/terragroan.js", and then our apply is overridden to just be "terragrunt apply -no-color plan.out"

@jghward
Copy link

jghward commented Apr 28, 2020

Thanks for this, exactly what I'm looking for. I was wondering if you had used this with GitLab as I'm having trouble getting it to work with the repo.yaml settings you suggested above. The plan works fine but the subsequent apply fails with Unable to update commit status: POST https://gitlab[redacted]: 400 {message: Cannot transition status via :enqueue from :pending (Reason(s): Status cannot transition via "enqueue”)}

@dmattia
Copy link
Author

dmattia commented May 4, 2020

Unfortunately I have only used this with Github, and have not seen that error :(

What does your apply command look like?

@jghward
Copy link

jghward commented May 4, 2020

No problem, I ended up writing a bash version of your script and was getting the same error.. I eventually fixed it by using the $PLANFILE environment var that Atlantis sets instead of hardcoding 'plan.out' and it started working. Thanks!

@angeloskaltsikis
Copy link

@dmattia i want to thank you about this NodeJS wrapper & of course for the amazing https://github.com/transcend-io/terragrunt-atlantis-config.
I have created a fork of your gist which support some more features. To be exact:

  • Support multiple planfiles supplied by Atlantis (Makes atlantis plan/apply work for all cases (general command or with directory)
  • Fixes bug with some planfile with blank Atlantis output (fixed by using terragrunt show instead of terraform show)
  • Support terragrunt apply less verbose output 🎉

Please let me know if you managed to use it with success.

@mqmr
Copy link

mqmr commented Jun 29, 2020

Hello,

Thanks for sharing this wrapper, it's very clever way to reduce terragrunt's verbosity.
Could you please tell how you handle output of terragrunt {plan,apply}-all commands?

Thank you.

@dmattia
Copy link
Author

dmattia commented Jun 29, 2020

@angeloskaltsikis I started using your changes, they're fantastic! I never knew terragrunt show existed, but that alone is a huge improvement. Thank you very much!

@mqmr unfortunately, I don't have a lot of experience with {plan,apply}-all commands, as I tend to only use terragrunt on a single module at a time.

@angeloskaltsikis
Copy link

@dmattia Thats great to hear. So glad we could give you an enhancement to your original work, because as i mentioned it helped us a lot 🚀

@mqmr Even if we use terragrunt plan-all massively and sometimes the apply-all as well its a bit scary to use those for one repo with multiple module configurations from Atlantis.
As a result we haven't created any function for {plan,apply}-all commands yet.
Having said that i am really interested on finding/creating a way to run the plan commands on parallel (possibly the apply as well) if possible (similar to plan-all but only for the files that having changes) but i still haven't decided on which is the best way to do that. There is an atlantis parallel_plan & parallel_apply feature released with Atlantis v0.13.0.
However this is only supported if you are using Terraform workspaces, which if you are using Terragrunt, there is no reason to do so.

May i ask, is your usecase of using the wrapper above with Atlantis or you are you interested in using it in another system? (I am asking that other than the $PLANFILE supplied by Atlantis the whole thing can be easily used even locally for less verbose outputs).
Thanks for kickstarting this conversation as it is good for @dmattia to check it as well as he may have some input too!

@mqmr
Copy link

mqmr commented Jul 14, 2020

Even if we use terragrunt plan-all massively and sometimes the apply-all as well its a bit scary to use those for one repo with multiple module configurations from Atlantis.

Yes, that's true.

is your usecase of using the wrapper above with Atlantis or you are you interested in using it in another system?

I ended up with my own implementation of such wrapper, but the whole idea kept the same. For tg plan-all I use grep to remove from its output excessive information because I couldn't find a way how to use PLANFILE with tg show for several resources.

@tomasbackman
Copy link

tomasbackman commented Dec 8, 2020

It was a little tricky at first to get this up and running, especially since the comments above with suggestions are a bit old and the script seem to have changed since then (most notably for example switched name from terragroan.js to terragrunt_light.js) and I started off trying without thinking too much.. =)

As a help for people trying to use this, we had to do the following to get this to work:
(Setup: Having terragrunt-atlantis-config running in a docker container and used by github actions)

In the repos.yaml (github action for atlantis)
Under plan:
run: terragrunt plan -no-color -out=$PLANFILE -> run: /atlantis/config/terragrunt_light.js plan
Under apply:
run: terragrunt apply -no-color $PLANFILE -> run: /atlantis/config/terragrunt_light.js apply

InDockfile:
Added:
RUN apk add --no-cache nodejs
RUN apk add --no-cache npm
RUN npm install shelljs --global
ENV NODE_PATH=/usr/lib/node_modules
COPY terragrunt_light.js /atlantis/config/

Now it runs beautifully and we are happy =)
It is a great script/help for when using terragrunt in atlantis!
@dmattia I feel this should be part of the big (and also great) https://github.com/transcend-io/terragrunt-atlantis-config or at least referenced from there.. =)

@kitos9112
Copy link

@dmattia thanks for sharing this small wrapper around Terragunt + Atlantis. I use it in combination with your terragrunt-atlantis-config to generate and validate atlantis.yaml files :) - I was wondering whether you're familiar with some tool to help visualise better the actual colours in the Atlantis output when using Terragrunt plan? I know the reasons why we use the no-colour CLI flag but not sure if another parsing to the Atlantis output could be performed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment