I’ve been doing a little bit of Ruby programming, working in an old codebase. As I threw WTFs at past-me, I remembered this talk from Katrina Owen on the satisfaction of accumulating tiny changes to make things better.

In May, Lee Robinson wrote The Story of Heroku documenting the history of the service from browser-based editor for Rails apps to the low-friction infrastructure support and dead simple git push deployment process. Lee Robinson notes that in 2022 a lot of folks who loved the idea of not having to care about infrastructure are now re-evaluating their options in the face of better tooling and more and more focus on microservices. Last week Heroku announced it is discontinuing free plans.

Did you know that resonant frequencies in Janet Jackson’s song Rhythm Nation could cause laptops with spinning metal hard drives to crash? That reminded me of Brendan Gregg’s advice against yelling at servers: “You’ll discourage them.”

Modern software engineering is changed from when I were a lad. The compile it, test it, and throw it over the wall to ops lifecycle is a thing of the past. Dev teams own their code from soup to nuts, have many more tools at their disposal, and much more autonomy in how they build and manage things. Yay!

With all this power comes complexity, inconsistency, and a heavier “cognitive load” on development teams. Boo!

This talk by Mike McGarr addresses those challenges in more depth and discusses how Netflix organise to meet the challenges. Like many talks about how Netflix works, the principles and patterns here are useful food for thought for any company that has a lot of software.

Camille Fournier: Structural Lessons in Engineering Management

When you see the world in systems, interactions, and efficiencies, you can trick yourself into believing that the humans inside of these systems will be happier if the system is well-organized. While I agree that there is value to organizing your teams effectively, the marginal value that the teams might experience is likely to be offset by the unintentional friction that happens for the people who don’t fit perfectly into your structure. If you don’t work to stay ahead of what the people in your team want and need, if you don’t create growth opportunities or appropriate technical challenges even though it’s not the most optimal thing for the system as it currently exists, you will find yourself losing good engineers and managers.

Lorin Hochstein: Incident response is a team sport

When a group of people respond to an incident, it’s never the responsibility of a single individual to remediate. It can’t be, because we each know our own corners of the system better than our teammates. Instead, it is the responsibility of the group of incident responders as a whole to resolve the incident.

In 2012 Double Fine ran a Kickstarter campaign around the idea of documenting the process of creating a new point and click adventure. They aimed for $300K to make the game and $100K for the documentary. By the end of the campaign, backers had pledged $3.3 million.

Thus began the work: everything from coming up with the story idea, designing the asthetic, building the team, writing the code, writing the story, animating everyhing, fixing bugs, and copious amounts of polish. The huge success of the campaign was both a blessing and a curse as expectations around the scope and quality of the game ratched up way beyond the original intentions, with the project eventually having a significant impact on the financial prospects of the company.

All of this was captured on camera and the resulting documentary series is a fascinating glimpse into the realities of creating something brand new under a weight of expectations. I think I’ve watched the entirety three of four times now and I always find some new aspect that I learn from.