"Vendoring" is a vile anti-pattern
What is "vendoring"?
From a comment on StackOverflow:
Vendoring is the moving of all 3rd party items such as plugins, gems and even rails into the /vendor directory. This is one method for ensuring that all files are deployed to the production server the same as the dev environment.
The activity described above, on its own, is fine. It merely describes the deployment location for various resources in an application.
However, many programmers have begun to commit the result of the above procedure to their own source code repositories.
That is to say, they copy all of the files at some version from one version control repository and paste them into a different version control repository.
You should have flinched reading the previous description. This practice is more vile than extending existing code by duplicating whole functions instead of using inheritance and abstraction, and for the same reasons.
Why is "vendoring" bad?
Extracting code from a version control repository to be archived in a compressed file or stored in a particular directory for deployment is not bad.
Extracting code from a version control repository to be re-comitted to a different version control repository is evil.
When you copy code between repositories:
- all history, branch, and tag information is lost
- pulling updates is impossible
- it invites modification, divergence, and unintentional forks
- it wastes space
- it is excruciatingly tedious to discover later which version of the code was the canonical source of the copied version, unless the person doing it went out of their way to document that information
What you should do instead
Use git submodules
Git's submodule mechanism stores a URL and a commit hash in your repository, which itself is under version control.
(TODO: explain more, examples of work-alikes from different VCSs)
Use an approximation of git submodules
If you can't use git submodules, use a script that deploys your third-party resources at the appropriate time, from a canonical source.
(TODO: describe more, provide examples)
None of those suggestions work for me
If you have a situation where it seems like "vendoring" is really the best way to deploy your code, contact me, call me an idiot, and describe why. I'm sure there's a better way, and where there's not, it's a bug. I hope to eventually document all such situations to prevent people from falling into this bad habit.
I wrote this four years ago in a fit of frustration because i kept encountering instances of "doing it wrong" -- what @capnslipp calls "willy-nilly by-hand vendoring," complete with local modifications to the vendored code, not in a forked repo, that he says is 'B.S." that you don't find in the real world. One such occurrence of this essentially locked my team's primary application to an outdated version with security holes, since we couldn't easily upgrade its dependencies. I was horrified, and since I made the noise about it, I was assigned the ticket to clean up the mess.
I couldn't find any tools at that time that would manage the problem in an automated way (for the language we were using), but I did see many instances of questionable vendoring practices being used in open source projects. I became concerned that if people didn't find a different mechanism or at least do it with some kind of proper record-keeping, I'd keep encountering that sort of mess.
One of the reasons this remained an obviously unfinished gist rant and not properly published anywhere is because I never found the time to do that exhaustive survey of software development dependency managers which might (in 2014?) have quelled my fears. My career went a different direction and I haven't had to deal with the issue since then, so I promptly forgot this existed.
I can't blame @capnslipp too much for the combative attitude and casting aspersions on me, since I set the mood with that antagonistic title and strongly-worded assertions. I regret the tone I used. This whole thing probably could have been: "don't do vendoring by-hand; use a dependency-tracking tool instead, and don't touch the vendor folder."
I was recently thinking to update this gist to that end, and include the helpful advice that people have commenters have given over the years. But if most shops are now aware of and following vendoring best-practices it's probably better just to delete it.