Skip to content

Instantly share code, notes, and snippets.

@cubanx
Last active August 29, 2015 14:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cubanx/bef1e763ae4abc378b7f to your computer and use it in GitHub Desktop.
Save cubanx/bef1e763ae4abc378b7f to your computer and use it in GitHub Desktop.
Prod deployment workflow
#When something goes wrong...
###Deploy `master` to UK, find a bug at some point during the week.
This scenario is the one we're hoping will happen the most of the least likely scenarios :) (Yeah, kinda confusing sentence, but read it again)
* `DARLENE` is deployed to UK on Friday and it's master as of midnight Thursday.
* `CHARLIE` is the lagged release deployed to US on the same Friday.
* `EDNA` is the release that started the moment the first commit went in after midnight on Thursday on master
```flow
st=>start: DARLENE is in UK
find=>operation: Find bug in DARLENE on UK (Note, bug does not exist in CHARLIE)
fix=>operation: Fix bug in master, currently EDNA.
season=>operation: Season EDNA in UK
dmuk=>operation: Deploy EDNA to UK
dmus=>operation: Deploy EDNA to US
seasoned=>condition: Seasoned enough?
e=>end
st->find->fix->dmuk->season->seasoned->dmus->e
seasoned(yes)->dmus
seasoned(no)->season
```
In this scenario, seasoning will most likely be different than the 7 days, but we'll play it by ear.
###Deploy `master` to UK, good for a week, then find bug in US
This scenario feels like it should happen a lot less than the previous one. I mean, we ran it in UK for a week, found a bug in US and it HAS to be fixed now? Seems like a party foul, but I could see it happening either in LMS because we don't have that many customers of that product in UK, or it might happen due to volume.
* `BERTHA` is master as of midnight Thursday deployed to UK on Friday
* `ALICIA` is the lagged release deployed to US on Friday
* `us-release` is the super short lived release branch we need for this scenario
* `CHARLIE` is the release that started the moment the first commit went in after midnight on Thursday on master
```flow
st=>start: BERTHA in UK, ALICIA is in US
find=>operation: Find bug in ALICIA in US
fix=>operation: Fix bug in master which is BERTHA
delta=>operation: Create us-release (Delta of ALICIA) branch and cherry pick change to master
season=>operation: Season in UK
wait=>operation: Wait until Friday
dmuk=>operation: Deploy BERTHA to UK
dmusr=>operation: Deploy us-release to US
dmus=>operation: Deploy BERTHA to US
seasoned=>condition: Seasoned enough?
e=>end
st->find->fix->delta->dmusr->wait->dmuk->season->seasoned->e
seasoned(yes)->dmus->e
seasoned(no)->season
```
There's a couple wrinkle in this scenario I'll attempt address:
**What if the fix on us-release has to go to UK immediately as well?**
Ok, you mean to tell me we didn't find it for a week on UK, but now, all of a sudden, it's an emergency that has to go to US? Seriously?
**Yes, seriously.**
*sigh* fine. Then either we do the fix on master (take `CHARLIE` at this point) and deploy that or we create a `uk-release` branch off of `BERTHA` and release `CHARLIE` when we feel good about it.
#When something goes wrong...
###Deploy `master` to UK, find a bug at some point during the week.
This scenario is the one we're hoping will happen the most of the least likely scenarios :) (Yeah, kinda confusing sentence, but read it again)
* `DARLENE` is deployed to UK on Friday and it's master as of midnight Thursday.
* `CHARLIE` is the lagged release deployed to US on the same Friday.
* `EDNA` is the release that started the moment the first commit went in after midnight on Thursday on master
```flow
st=>start: DARLENE is in UK
find=>operation: Find bug in DARLENE on UK (Note, bug does not exist in CHARLIE)
fix=>operation: Fix bug in master, currently EDNA.
season=>operation: Season EDNA in UK
dmuk=>operation: Deploy EDNA to UK
dmus=>operation: Deploy EDNA to US
seasoned=>condition: Seasoned enough?
e=>end
st->find->fix->dmuk->season->seasoned->dmus->e
seasoned(yes)->dmus
seasoned(no)->season
```
In this scenario, seasoning will most likely be different than the 7 days, but we'll play it by ear.
###Deploy `master` to UK, good for a week, then find bug in US
This scenario feels like it should happen a lot less than the previous one. I mean, we ran it in UK for a week, found a bug in US and it HAS to be fixed now? Seems like a party foul, but I could see it happening either in LMS because we don't have that many customers of that product in UK, or it might happen due to volume.
* `BERTHA` is master as of midnight Thursday deployed to UK on Friday
* `ALICIA` is the lagged release deployed to US on Friday
* `us-release` is the super short lived release branch we need for this scenario
* `CHARLIE` is the release that started the moment the first commit went in after midnight on Thursday on master
```flow
st=>start: BERTHA in UK, ALICIA is in US
find=>operation: Find bug in ALICIA in US
fix=>operation: Fix bug in master which is BERTHA
delta=>operation: Create us-release (Delta of ALICIA) branch and cherry pick change to master
season=>operation: Season in UK
wait=>operation: Wait until Friday
dmuk=>operation: Deploy BERTHA to UK
dmusr=>operation: Deploy us-release to US
dmus=>operation: Deploy BERTHA to US
seasoned=>condition: Seasoned enough?
e=>end
st->find->fix->delta->dmusr->wait->dmuk->season->seasoned->e
seasoned(yes)->dmus->e
seasoned(no)->season
```
There's a couple wrinkle in this scenario I'll attempt address:
**What if the fix on us-release has to go to UK immediately as well?**
Ok, you mean to tell me we didn't find it for a week on UK, but now, all of a sudden, it's an emergency that has to go to US? Seriously?
**Yes, seriously.**
*sigh* fine. Then either we do the fix on master (take `CHARLIE` at this point) and deploy that or we create a `uk-release` branch off of `BERTHA` and release `CHARLIE` when we feel good about it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment