Repair symlinks in AWS codepipeline

buildspec-terraform-symlinks.yml

version: 0.2

phases:
  pre_build:
    commands:
      # Hack to fix the symlinks that Codepipeline breaks
      # Applies to .tf files that contain only 1 line
      - for file in *.tf ; do awk 'NR==2{exit}END{exit NR!=1}' "$file" && sh -c "cat $file | xargs -I R2 ln -sf R2 $file"; done

When codepipeline pulls the source for a git repo it collapses the symlinks, with each symlink file containing 1 line of text - the target of the symlink. Adding this command to the buildspec with find collapses symlinks and re-link them to their target.

To demonstrate how it works:

$ echo "SRC\nFILE" > src.tf && echo "./src.tf" > ln.tf

$ cat ln.tf 
./src.tf

$ for file in *.tf ; do awk 'NR==2{exit}END{exit NR!=1}' "$file" && sh -c "cat $file | xargs -I R2 ln -sf R2 $file"; done

$ cat ln.tf
SRC
FILE

The pattern matches *.tf files in the pwd, but can be easily extended to whatever target files are desired.

sources:
https://stackoverflow.com/questions/34168397/how-can-i-find-files-with-more-than-one-line-in-them-with-a-given-name
https://superuser.com/questions/638998/easiest-way-to-restore-symbolic-links-turned-into-text-files

Brilliant, I was spending hours to realize that codebuild bricks all symlinks (in our case package.json files referencing a package.json in the project root). Given our repo size we cannot just switch to the proposed solution of cloning the repo as cheap CodeBuild does not support shallow clones on branches. This is what I ended up doing:

      - find . -name 'package.json' -type f -exec bash -c 'if [ ! -L "{}" ]; then target=$(head -n 1 "{}"); [[ $target == /* || $target == .* || $target == */* ]] && { rm -f "{}"; ln -sf "$target" "{}"; }; fi' \;

In this script:

1. find . -name 'package.json' -type f: Finds files named package.json that are regular files.
2. -exec bash -c '...' \;: Executes the given bash script for each file found.
3. if [ ! -L "{}" ]; then ... fi: Checks if the file is not a symlink (-L checks if the file is a symlink).
4. target=$(head -n 1 "{}"): Reads the first line of the file to get the presumed target path.
5. [[ $target == /* || $target == .* || $target == */* ]]: Checks if the target path looks like a valid path.
6. { rm -f "{}"; ln -sf "$target" "{}"; }: Removes the regular file and creates a symlink with the same name pointing to the target path.

This command will only convert package.json files to symlinks if they are not already symlinks and if their content appears to be a valid path. As always, test this script in a controlled environment before using it in production to ensure it works as expected with your specific file setup.