Moving Concourse Pipelines

README.md

Moving Concourse Pipelines

Currently concourse does not support moving pipelines between teams via fly CLI. There is an issue for that here

The only way to do this is to make a few changes in the DB.

If you run the statement below you will see that 6 tables have the team_id column.

concourse=> select table_name                                                                                                                                                                                                                                                                                                                                                       from INFORMATION_SCHEMA.COLUMNS                                                                                                                                                                                                                                                                                                                                                 where column_name = 'team_id';;
    from INFORMATION_SCHEMA.COLUMNS
    where column_name = 'team_id';

If you describe the tables \d+ pipelines you can see the column is present. Run a quick select * from pipelines limit 5 to examine what the data is like.

If you do this for the rest of the tables, it becomes apparant that the only tables you need to edit are pipelines, and builds. This will move your pipeline definitions and your historic builds over correctly, and everything will carry on working.

If you want to move all your pipelines from team main to team new (lets say it has an id of 3) you would just run the following statements:

update pipelines set team_id = 3 where team_id = 1; 
update builds set team_id = 3 where team_id = 1;

This will move all of your pipelines that belong to team 1 to team 3.

However, you might want to split your pipelines into multiple teams so find all the pipeline names that share a prefix, using something like

select * from pipelines where name ~ 'shared';

or just find the ids if they dont follow a convention and run something like this

update builds set team_id = 2 where pipeline_id in (3,6,8,12,43,44,21,22);

You can run a similar statement for the pipelines too.

To finish the migration, you will also need to refresh the materialized views to reflect the new pipeline locations

refresh materialized view next_builds_per_job;
refresh materialized view transition_builds_per_job;
refresh materialized view latest_completed_builds_per_job;

I would like to confirm that this worked very well for us on Concourse 4.2.1, thanks! 💯

This worked for me on Concourse 4.2.2. Thank you! 👍 ❤️

Awesome!

Anyone know if this works for v5.1?

Confirmed that it also works for v5.1. Thanks for the solution!

Still working as of v5.8.0, thank you!

this worked for me too on 5.7.x
I don't know what @wendorf meant by "refresh the materialized views", I copied those commands to the postgres command line and it didn't like them.

This seems to still work in Concourse 7.

Worked on 7.8.
The 'team_id' column was present in the following tables: pipelines, containers, builds, volumes, and workers.
Materialized views were removed in 2018, so you can ignore the advice from @wendorf.