This assessment is performed with respect to the requirements of the .NET community open source projects which I maintain. It is restricted to the free tier provided by each system.
Only the drawbacks are listed, which makes the assessment sound a bit negative, but it's relative to a baseline functional expectation:
- Linux and Windows
- Fast provisioning
- Concurrent builds
- Common build properties surfaced as env vars (e.g build number)
- Readable, copyable, and line-linkable log output, supporting ANSI colour codes and code page 437
Plus generally expected things like "a good UI".
- Appveyor
- very slow provisioning, especially of Linux agents
- does not run concurrent builds
- does not support code page 437 on Windows without workarounds
- Azure DevOps
- console output is utterly broken, in too many ways to list
- navigation (and UI in general) is painful
- Circle CI
- doesn't support Windows
- console output:
- is not retained - it's only shown if the page was opened while the build was running
- is not line-linkable
- often has an extra blank line between every line
- Cirrus CI
- doesn't surface a build number
- console output does not show any whitespace in lines with ANSI colour codes
- GitHub CI
- doesn't surface a build number
- console output is not copyable - try selecting a section of text larger than the visible buffer!
- GitLab CI
- doesn't support Windows
- console output is not line-linkable
- no further investigation
- Travis CI
- only has partial Windows support
- slow provisioning
- only makes env vars available after parsing the YAML file, which is a complete PITA