Skip to content

Instantly share code, notes, and snippets.

@nilleb

nilleb/relengprinciples.md

Last active May 18, 2017 19:40
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nilleb/f7e9211823b28427ee22a078575ff627 to your computer and use it in GitHub Desktop.
Save nilleb/f7e9211823b28427ee22a078575ff627 to your computer and use it in GitHub Desktop.
Download ZIP
Release engineering principles
Raw
relengprinciples.md

This documents details my release engineering principles. They have been forged through experience, no need to say. :-)

The context is: you want to build your software, and eventually distribute it. You are not alone, and even if, you're not immortal. If you want your creation to live after you,

You need

  • An objective, repeatable, exhaustive process.
    • No hidden knowledge.
    • No environmental prerequisites.
    • In three years, you should be able to get the same binaries you get today.
  • To trace what's going on, at the most fine grained detail level.
    • You need a perishable storage for such information.
    • You need to be able to extract reusable patterns from such logs.
  • Collaboration.
    • Write your softwares as if they were going to be published on github.
    • Open their code to the greatest number.
    • Use platforms that allow you to receive modification requests and to track bug issues.
  • Software quality follow-up
    • Improve yourself
    • Improve your code
    • Improve your deliverables
  • To identify the value added deliverables
    • The build process outputs a lot of byproducts. A small percentage are really useful.
    • Tell everybody which deliverables really matter.
  • To share knowledge
    • Knowledge is power. Abuse and concentration of power are bad for your community.
  • To avoid duplication
    • Even the littlest duplication is a loss of time
    • Boilerplating is dangerous: don't multiply almost-identical copies of the same file
    • Prefer templates to boilerplates
  • To keep distinct sources and binaries

You get

  • Your software is open to contribution (open-source != contribution)
  • A traceable process
    • You can optimize the build process
  • A diagnostic of your software quality
    • You can optimize your software

The roles

So far, you can glimpse several roles in your release engineering pipeline. Let me introduce them:

  • a build process,
  • a PMP (Project Management Platform)
  • a SCM (Source Control Management),
  • a SPM (Software Production Platform),
  • a SQM (Software Quality Management Platform)
  • a AMP (Artifacts Management Platform).

About the PMP

Well, I don't want to talk about this subject: in my experience, PMP softwares are lousy.

About the SCM

Easy choice: as of 2016, there's only one SCM. :-)

About the SPM

The ideal SPM

  • recognizes the build process entry point (no need to tell)
  • recognizes logs and quality indicators collected during the build process (no need to tell)
  • sends the value added deliverables to the APM
  • offers a set of reports about the collected data
  • lets the users recognize (failure) patterns
  • uses those patterns in future builds and helps users adopt the best strategy to fix those failures

As of 2016, you've many choices. According to my experience no ideal SPM exists. Every one has its features and trade/offs. Perhaps I will write about them, one day.

About the SQM

The ideal SQM

  • offers dashboards about the current state of a project
  • offers indicators about the evolution of the quality indicators over a period of time
  • integrates with the project management tools
  • lets you aggregate or drill down indicators

As of 2016, you have many choices in this field, too. I have a preference, but I am not going to disclose it.

About the APM

The ideal APM

  • offers you the possibility to host a deliverable
  • lets you decide whether a deliverable should be visible or not, and to whom
  • lets you map metadata to binaries (traceability)
  • lets you 'promote' a deliverable, changing its availability
  • offers a REST API to do all of the above

As of 2016, you've many choices for the APM, too. Pretty good ones exist.

How to obtain a working release process

You're probably lost at this point. You haven't even started writing a line of code, and you need already a bunch of servers to cover the roles. Never mind, and let's go back to the most important one: the build process.

  • Build process must be idempotent°
    • avoid all types of binaries modification (e.g. time dependent version increases)
    • ° (distribution issue) if your OS requires signature verification, at some point you will have to timestamp your binaries.
  • Build processes must be stored along with the sources
    • SCMs are good at storing history
    • who's responsible for storing build processes?
    • who keeps track of their modifications? how?
    • how can you roll back a build process?
  • Everyone° must be able to build the sources, everywhere
    • clearly identify dependencies and requirements
    • in your production perimeter, make sure that every dependency has been cached
      • the build process must be aware of these dependencies, and inform the user about their presence
      • your project dependencies tree must be exhaustive
    • no floating-version dependency
    • don't let anybody change one of your dependencies without notice
      • ° (distribution issue) if you're signing your binaries, your certificate contains sensitive information. Replace your 'production' certificate with a fake one, in the general workflow. Allow the user to switch the certificate in the easiest way possible.
  • Allow the user to interact with the build workflow by means of a 'configuration'
    • e.g. where to get the signing certificate
    • this should be a build host/user related configuration
    • allow the user to use several 'configuration providers' (e.g. command line, json file, ..)
  • Version numbers matter
    • have a look at the Semantic Versioning
    • if you don't like semantic versioning, try however to give the version number a meaning
  • Help your customer to get the build failure reasons right
    • take the time to explain build failures
    • use the best technology to recognize failure patterns and give hints
  • Metadata matters
    • identify the origin of your binaries
    • make this metadata available for everyone
  • Formalize the deployment process in the same way you did with the build process
    • everyone speaks about Continuous Deployment
    • automation is the key
  • While developing software
    • Keep your quality under control (linters)
    • Avoid duplications
    • Unit test your software (in a reasonable way)
    • Respect conventions
    • Adopt SOLID, KISS, DRY principles
    • Read "Clean code" by Robert C. Martin or
    • Use comments to document code only when there's no alternative
    • Architecture is important
      • Refactor your code to get a dependency tree instead of a monolith
      • Compilers and build tools can help you improve the build performance. Let them help you.
  • While managing a software lifecycle
    • Provision the right amount of time to each phase of the software life
      • Defining a build process for your software is a phase of its life
      • Testing your software is a phase of its life
      • Managing software quality is a phase of its life
      • Deploying it and supporting it is a .. well, you got it ;-)
    • The software production process covers all of them, because of its aspects
    • Evaluate the project dependencies (thirdparties?)
      • Take care to software licenses
      • Take care to effective costs
      • When two hypothetical dependencies are equivalent, prefer the open-source one

The other roles

Well, I've listed exhaustively all the roles in a release engineering process. But nowadays, you don't need to have your private release engineering environment. If you have a look at the Internet, and at the services it offers, you will be able to complete the checklist in a few hours.

As of 2016

This is my list! You don't have to take it blindly. Google is your friend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment