| Full Thread | Original Author | Original document |
|---|---|---|
| @tdpauw - thread link | @nico_mommaerts - tweet link | Pastebin link |
- performance, Jenkins is file based and with several 1000's of jobs startup time and loading of views with lots of jobs is horrible.
- avoid plugin hell, each team has his own specific workflows and uses different plugins. By isolating sets of plugins per team there is less chance of conflict between them
- possibility to authorize teams on their own jenkins, folder/job authorization also has serious performance issues in Jenkins with a large number of jobs. While we only authenticate developers and do not use authorization (everyone can see/start everyons jobs), we have had audit recommendations in the past about that. Depends on the industry you are in of course.
- easier lifecycle management: it's difficult to impossible to make sure all jobs and the plugins they use still function as they should when upgrading a Jenkins instance. By having multiple instances the impact on the organisation is less when we upgrade one by one. It's also easier to choose the right time to do an upgrade, it doesn't have to be convenient for the entire IT org at once. Usually each upgrade also means the dev teams need to change some of their jobs, so they have to have time to do that.
- not a single point of failure, Jenkins is not the most stable system out there
- obviously you have to maintain more instances, and they all start diverging from each other really quickly, plugin wise. (if you choose to allow that)
- the flipside of the pro of plugin management, we have a few custom plugins and sometimes need to roll out bugfixes for third party plugins. So those need to be in sync across all instances. We do use Jenkins Enterprise and have our own Update Center, that makes it a bit easier.
- license cost because we use Enterprise. A bit more resources but not enough to notice. Depends on the size of your organisation of course, if that cost matters or not.
Note that while we have several instances, we still manage them centrally in one team. So not sure if you would interpret that as decentralized.
Unfortunately due to choices of the customer, we do not focus on checks like quality and security. We have a centralized Sonar but do not enforce using it, nor follow the results. However, for our largest Jenkins instance we have a framework setup so that developers can't create jobs themselves. All jobs are created with the jobdsl plugin. Job definitions are thus in scm, and all jobs for a given team are created the same (get a Sonar job etc). You could do that for the entire organisation, give developers access to a declarative part of the jobsdsl framework so they can add/remove jobs themselves, and modify certain parts of it. But at the same time they cannot modify how the jobs are created, or what Sonar profile is used etc.
I see trust being mentioned among your replies. Not sure how I feel about that. For us trust is not a strategy :) Even with mature teams, the goals of a dev team are not the same as for example a deployment team. Hard to avoid them starting to optimize locally. Again, depends on the size of your org and industry. I do know Netflix gives each of their teams total freedom in CI and CD, but I also don't think there many companies out there that can get away with that.
Given the heavy security focus in the finance industry, deployment infrastructure cannot be owned, shared or used with the dev teams. Seperation of responsibilities and all of that. So that was a no-brainer for us.