Migrating Terraform State

Migrating Terraform Cloud State.md

Migrating Terraform Cloud State (the Super Janky Method™️)

The following is a semi-automated (or semi-manual, if you're a glass-half-empty person) way of generating terraform state mv commands to move existing resources to new paths.

It does not use a Terraform plan file, as this is often not conveniently available (e.g. when using Terraform Cloud).

Tested with Terraform 0.12, 0.13 and 0.14 output.

Note: if you're not using Terraform Cloud, you might want to try some of the proper tooling other people have built, such as https://github.com/mbode/terraform-state-mover.

Prerequisites

• A text editor or IDE that supports regex find and replace.

Instructions

1. Run terraform plan and generate the output.
2. Copy the changed-resources output to your text editor.
3. Find: ^\s\s# (.*) will be.*(\n(^$|\s\s[^#].*))*$
   Replace: $1
4. Move the lines around to create pairs of resources, where the old line is the first of the pair, and the new line is the second of the pair. Learning the keyboard shortcuts in your text editor for moving lines up and down will help here.
5. Find: ^(.*)\n(.*)$
   Replace: terraform state mv '$1' '$2' &&
6. Remove the final &&.

Example

1. The following plan is generated by terraform plan:

     # module.my_s3_bucket.aws_s3_bucket_public_access_block.main will be destroyed
     - resource "aws_s3_bucket_public_access_block" "main" {
         - block_public_acls       = true -> null
         - block_public_policy     = true -> null
         - bucket                  = "my-s3-bucket" -> null
         - id                      = "my-s3-bucket" -> null
         - ignore_public_acls      = true -> null
         - restrict_public_buckets = true -> null
       }
   
     # module.my_s3_bucket.aws_iam_policy.read_write will be destroyed
     - resource "aws_iam_policy" "read_write" {
         - arn    = "arn:aws:iam::1234567890:policy/s3-bucket/my-s3-bucket-read-write" -> null
         - id     = "arn:aws:iam::1234567890:policy/s3-bucket/my-s3-bucket-read-write" -> null
         - name   = "my-s3-bucket-read-write" -> null
         - path   = "/s3-bucket/" -> null
         - policy = jsonencode(
               {
                 - Statement = [
                     - {
                         - Action   = [
                             - "s3:PutObject",
                             - "s3:ListBucket",
                             - "s3:GetObject",
                             - "s3:DeleteObject",
                           ]
                         - Effect   = "Allow"
                         - Resource = [
                             - "arn:aws:s3:::my-s3-bucket/*",
                             - "arn:aws:s3:::my-s3-bucket",
                           ]
                         - Sid      = "ReadWrite"
                       },
                   ]
                 - Version   = "2012-10-17"
               }
           ) -> null
       }
   
     # module.my_awesome_supermodule.module.s3_bucket.aws_s3_bucket_public_access_block.main will be created
     + resource "aws_s3_bucket_public_access_block" "main" {
         + block_public_acls       = true
         + block_public_policy     = true
         + bucket                  = (known after apply)
         + id                      = (known after apply)
         + ignore_public_acls      = true
         + restrict_public_buckets = true
       }
   
     # module.my_awesome_supermodule.module.s3_bucket.aws_iam_policy.read_write will be created
     + resource "aws_iam_policy" "read_write" {
         + arn    = (known after apply)
         + id     = (known after apply)
         + name   = "my-s3-bucket-read-write"
         + path   = "/s3-bucket/"
         + policy = jsonencode(
               {
                 + Statement = [
                     + {
                         + Action   = [
                             + "s3:PutObject",
                             + "s3:ListBucket",
                             + "s3:GetObject",
                             + "s3:DeleteObject",
                           ]
                         + Effect   = "Allow"
                         + Resource = [
                             + "arn:aws:s3:::my-s3-bucket/*",
                             + "arn:aws:s3:::my-s3-bucket",
                           ]
                         + Sid      = "ReadWrite"
                       },
                   ]
                 + Version   = "2012-10-17"
               }
           )
       }
   

2. Doing a regex find and replace, where the find is ^\s\s# (.*) will be.*(\n(^$|\s\s[^#].*))*$ and the replace is $1, leaves us with the following:

   module.my_s3_bucket.aws_s3_bucket_public_access_block.main
   module.my_s3_bucket.aws_iam_policy.read_write
   module.my_awesome_supermodule.module.s3_bucket.aws_s3_bucket_public_access_block.main
   module.my_awesome_supermodule.module.s3_bucket.aws_iam_policy.read_write
   

3. If we shuffle those around, we now two pairs of resources:

   module.my_s3_bucket.aws_s3_bucket_public_access_block.main
   module.my_awesome_supermodule.module.s3_bucket.aws_s3_bucket_public_access_block.main
   module.my_s3_bucket.aws_iam_policy.read_write
   module.my_awesome_supermodule.module.s3_bucket.aws_iam_policy.read_write
   

4. Another regex find and replace, where the find is ^(.*)\n(.*)$ and the replace is terraform state mv '$1' '$2' &&, gives us the following:

   terraform state mv 'module.my_s3_bucket.aws_s3_bucket_public_access_block.main' 'module.my_awesome_supermodule.module.s3_bucket.aws_s3_bucket_public_access_block.main' &&
   terraform state mv 'module.my_s3_bucket.aws_iam_policy.read_write' 'module.my_awesome_supermodule.module.s3_bucket.aws_iam_policy.read_write' &&
   

5. Simply remove the final &&, and you're done! You now have a list of terraform state mv commands you can use.