Skip to content

Instantly share code, notes, and snippets.

@willejs
Last active January 4, 2023 09:39
Show Gist options
  • Save willejs/7a58cd74f1b5da7d1a5bb83f2dbf8d2e to your computer and use it in GitHub Desktop.
Save willejs/7a58cd74f1b5da7d1a5bb83f2dbf8d2e to your computer and use it in GitHub Desktop.
Moving Concourse Pipelines

Moving Concourse Pipelines

Currently concourse does not support moving pipelines between teams via fly CLI. There is an issue for that here

The only way to do this is to make a few changes in the DB.

If you run the statement below you will see that 6 tables have the team_id column.

concourse=> select table_name                                                                                                                                                                                                                                                                                                                                                       from INFORMATION_SCHEMA.COLUMNS                                                                                                                                                                                                                                                                                                                                                 where column_name = 'team_id';;
    from INFORMATION_SCHEMA.COLUMNS
    where column_name = 'team_id';

If you describe the tables \d+ pipelines you can see the column is present. Run a quick select * from pipelines limit 5 to examine what the data is like.

If you do this for the rest of the tables, it becomes apparant that the only tables you need to edit are pipelines, and builds. This will move your pipeline definitions and your historic builds over correctly, and everything will carry on working.

If you want to move all your pipelines from team main to team new (lets say it has an id of 3) you would just run the following statements:

update pipelines set team_id = 3 where team_id = 1; 
update builds set team_id = 3 where team_id = 1;

This will move all of your pipelines that belong to team 1 to team 3.

However, you might want to split your pipelines into multiple teams so find all the pipeline names that share a prefix, using something like

select * from pipelines where name ~ 'shared';

or just find the ids if they dont follow a convention and run something like this

update builds set team_id = 2 where pipeline_id in (3,6,8,12,43,44,21,22);

You can run a similar statement for the pipelines too.

@wendorf
Copy link

wendorf commented Jun 8, 2018

To finish the migration, you will also need to refresh the materialized views to reflect the new pipeline locations

refresh materialized view next_builds_per_job;
refresh materialized view transition_builds_per_job;
refresh materialized view latest_completed_builds_per_job;

@marco-m
Copy link

marco-m commented Nov 22, 2018

I would like to confirm that this worked very well for us on Concourse 4.2.1, thanks! 💯

@flavorjones
Copy link

This worked for me on Concourse 4.2.2. Thank you! 👍 ❤️

@edwardstudy
Copy link

Awesome!

@infews
Copy link

infews commented May 9, 2019

Anyone know if this works for v5.1?

@Sophichia
Copy link

Confirmed that it also works for v5.1. Thanks for the solution!

@mdelillo
Copy link

mdelillo commented Apr 3, 2020

Still working as of v5.8.0, thank you!

@speculatrix
Copy link

this worked for me too on 5.7.x
I don't know what @wendorf meant by "refresh the materialized views", I copied those commands to the postgres command line and it didn't like them.

@artburkart
Copy link

artburkart commented Dec 3, 2021

This seems to still work in Concourse 7.

@wailashi
Copy link

wailashi commented Nov 7, 2022

Worked on 7.8.
The 'team_id' column was present in the following tables: pipelines, containers, builds, volumes, and workers.
Materialized views were removed in 2018, so you can ignore the advice from @wendorf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment