I wake up and do my morning routine of checking the weather (because the Carrot Weather app will chastise me if I don't) then scan email, RSS feeds and news headlines.
Then breakfast. Today that was eggs with spinach. While eating I read the comic book series I'm in the middle of, Gideon Falls (really fantastic) then play my daily games: Star Realms, then Backgammon against the AI opponents.
I see I have message from Gillian Wenhold, Jon Z, and Neil, but nothing looks urgent so I don't reply to any of them immediately.
I see a message from Percona who is working with Igor to assist during some maintenance going on where we need to take the Gotham db servers down. That reminds me to check the status of another ticket, one to restore some home data for an issue I'm working on with Kraig where some code allowed some old home/owner info to get messed up. They have not restored the data, so I ask Fernando from Percona if he can take care of that around the other in-progress task.
Now I read the next very short chapter in the Google SRE book so I'm ready for the book club on Monday (was originally going to be today).
After my shower, it's coffee time. I've gotten a new message from Hayden who's thanking me for spending some time with him answering a question yesterday and asks if it's worth him turning it into a Stack Overflow post. It is, so I tell him that's a great idea.
Coffee's brewing, I let Gillian know I'll look into her query. She's going to be working on a CRUD page for state_coas but we don't have that data in our data dumps, which makes it hard to develop for. I tell her I'll look at the data for that in a bit and get back to her.
Jon Z. is running into a problem with the upgrade_playbook_ui script I added this past week. His issue is unrelated but it's because of the graphql-pro thing with some newer components and the answer was on Stack Overflow (search for "vendor/bundle"). Jon confirmed that it worked!
Finally at my desk!
Looking into Gillian's issue, the table she mentions is puny (20 rows of 3 columns) of data that's very obviously not sensitive. I give her an export to get her going. Leaving any updates to our data dumps up to her and her team.
I check in with Percona who's confirming things about the data we need restored and indicates they'll work on it after the other task is complete.
During all that, Hayden's already posted the Stack Overflow and is asking me to review it.
I see that Carlos has sent me a Connect message now, but plan to review Hayden's post first, when I see a mention from Kraig in System Operations which overriddes anything else going on. Turns out it was for me and Ben Langfeld related to a deployment issue that Ben's already on top of. I review what's going on, see that Ben's handling it and move back to Hayden's post.
I spend some time on feedback with Hayden. During which I'm answering some questions from Fernando from Percona related to the data restore.
Now to see what Carlos is asking. He apparently was helping out Jon Z. on his own and is posting an update to the script for updating playbook. It was stuff I was planning to do anyway, so I approved.
I look at the clock and realize I'm 2 minutes late for Heroes for Hire standup/retro for this week, so I join that.
Our regularly scheduled Ninja, Brandon, is on PTO so we talk through some confusion with Kelly/Bryan helping us out today. I see Kraig on the call, which makes me suspicious because this usually means he wants something from me.
During standup Percona indicates they've started restoring data and I confirm that I see it. Since he's on our standup, I inform Kraig. Jillian Amoroso informs me of a post-mortem The Incredibles is having after our retro that I should be a part of since it's the reason Percona is restoring this data.
Retro time, I need to pay attention, so shutting out all distractions until it's over.
And, it's done. A great retro, especially with the updated process we're using that Jillian had suggested.
Quick check with Percona who informs me that the restore is complete. Reached out to Kraig to confirm that he can see the data so I can give him access if he needs it.
We rushed to the end of retro to dive into the post-mortem for the Incredibles issue, 2021-09-23-TrainingDataHomeCorruption. So now paying attention to that. The issue was rooted in a misunderstanding of what a Fake Lead is, which should have been removed from Nitro long ago.. Some call it a "training" lead which is confusing.
Postmortem complete. A lot learned. A root cause in this case was lack of documentation.
Syncing up with Kraig to see what we want to do with the data cleanup that's related to that postmortem then grabbing a quick lunch. But I remembered that Bryan Reed was testing a story for me and wanted to follow up with him about the Gallery Metrics page that I fixed an issue on. I confirmed it working and now looking for his approval again. Hopefully I can merge the story when I return from lunch.
And I'm back. I stepped outside for a little bit to get some fresh air. And watched an episode of Star Wars Visions since the episodes are about fifteen minutes. Short. Also, got a call from my wife who's been out and wanted to know if I want a bubble tea. I skipped one for today.
Bryan acknowledged that my story was working as expected and approved it. Merged. Will check it later in production and inform Ben Klang, who brought the issue to me.
Check email and scan Sentry and Github for anything I think needs my attention. The only thing that's interesting is CHMN-252 because it deals with Power Story and Tim wants me to keep an eye on it. Nothing looks off here, so I'll leave it to the team and their ninjas at this time.
Quick scan of Connect, but things seem quiet which is common on Fridays because teams are in retro or focused on getting stuff out for the week.
My wife arrives home and surprises me a root beer lollipop she got from See's Candies when she was out. They give free samples, so if she's near, she's getting her free sample. I'm now glad I didn't take her up on her offer for the bubble tea, as well.
One hour until demos, so I see if Kraig's back and start to look into the data cleanup.
I post my story thus far in the Heroes for Hire room in case someone's interested in seeing the progression of my day.
Jumping into the Heroes for Hire zoom room to look at the data cleanup with Kraig. Out of a potential 161 homes affected, 52 homes have address info that's off, and there are only 6 owners with changed owner info.
We decide that with QA getting refreshed this weekend and Kraig not wanting to make significant data changes on a Friday that we wait until Monday to start fixing data. That way we can test the corrections in QA prior to updating production data. All the relevant data is for very old homes most of it not updated in many years so risk of them contacting us over the weekend seems very low.
I tell Kraig that for the rest of today I will work on collecting the data we'll need to change and get it staged into some update scripts that we can use to hit the ground running on Monday morning. Before signing off, he shares his recent beer purchase, a pack of "Haunted House" beer that he's excited about having later.
It's now about twenty minutes until demos. I'll work on pulling that data together. I generally sign into the demo zoom room five to ten minutes early so I can enjoy whatever music Ben's playing. Instead of my pre-demos plan though, I get distracted by my Twitter feed and give up and just sign in early to demos.
People are joining demos. I see Jill at the top of the my list which reminds me that she's sent me questions during demos a few times which have distracted me from seeing demos. So curious if she's going to shoot me a question, send me a message telling me she's not going to bother me, or just not send a message. (update: she didn't send me a message!)
And we're starting BT demos.
Nitro let's me know that my merged story just hit QA. Looking forward to seeing it work in production by end of demos.
Hearing about Tempo's initial launch is pretty exciting.
I get a couple of alerts from Connect. One is an Alex's Lemonade update, but the other is Diego. He's saying appointments aren't getting issued in training QA, but I'm not sure they automatically get issued, so pointing him to Kraig.It should be, so I suggest running it manuallly in QA to see what's going on, but offer to do it myself if he doesn't have access.
And I get a message from Kraig. Taking care of Diego first.
Demos ended. Signing out of Zoom.
Syncing up with Diego on understanding appointment issuing in training. Hop over to see what Kraig says. He's getting some data together he mentioned earlier that I forgot about already. Some minor data discrepancies in production which need updating (first_lead_id) and that Incredibles may have broken something and need help. Something with a job that generates a "big file" but they may only need my help to merge.
And now here's Carlos. He came across the Weekly Commissions report, doesn't see any reference to it in Splunk as being used and so is considering removing it. I say we should get approval first to make sure.
And Kraig is back. He's got a small spreadsheet of data to fix. Downloading that. He says Incredibles merged a fix for their issue which would have "melted Resque at 5am tomorrow". I remind him that at this time on a Friday that a merge will not deploy to production because deploys are now locked until Monday morning.
I remember to check the Gallery Metrics page which is now in production. I'm excited enough for that, that I check it out. It's showing data now, so I shoot a link to Ben Klang who gives me a thumbs up not long after.
Now back to fixing that data for Kraig. Four est. appt. sources without a first appt id. This is a common, routine thing that still happens. Takes five minutes or so to fix. Report back to Kraig who says that Incredibles emergency deploy is going to include a Playbook update that would normally not hit production until Monday and wonders why they couldn't have waited until Monday to merge the Playbook update.
Since I've got Kraig's attention I mention the Weekly Commission report that Carlos brought up to see what his thoughts are. He is unsure. We'll have to check it out next week. I suggest syncing up with Dirk/Infinity on it.
Scanning Connect one last time before shutting down for the day. The Alex's Lemonade Stand message was a reminder that there's only a few more days left in the month. I see in the Playbook room a message from Jon Z. at 4:22pm ET saying "Playbook 10.10.0 was just merged into Nitro".
I let Kraig know I'm about to sign off and be away from my computer. This is because invariably, at this time of day, on any given day, when I step away from my computer is the exact time I get a message from Kraig or Ben Klang asking "Are you near your computer?"
Noticing a build up of screenshots and downloaded files on my desktop I clean that up when I see Kraig reply. Nothing's going on he's aware of other than the emergency deploy with the playbook release, but if I'm not around he can "yell at Ben Klang". I am fine with that.
And Klang's ears must be burning because I get something from him. "Very nice, Thank you."
That's about as good an end to the work day as I'm about to get, so I'm out.