The Software Ark: Issue 5
Quote of the week
You cannot escape the responsibility of tomorrow by evading it today.
- Abe Lincoln
Articles
We need young programmers; we need old programmers
This is quite near and dear to me. I still remember when I was right out of university and chatting with a mentor of mine in a 1:1. They said, with no irony, that they'd need to switch into management at some point because "old engineers just weren't in demand". And this was one of the smartest people I know, who spent a lot of time updating their knowledge of the software engineering world. It's crazy to me they felt that way. But after being around for a while, it certainly feels true - no-one wants old engineers.
Even companies that try to protect against ageism fall prey to interviewer biases. A common one I've had to reject was: "This person has too much experience to be an intermediate dev".
It makes me wonder how many years of software development I've got left in me before the industry forces me to shift. It's a good thing I don't hate management, I guess.
Strangle, strangle, strangler fig is the way to go. It's the one approach that I've seen consistently work. The alternative approach is rebuilding from scratch, but that's significantly riskier. The one benefit is during team-handovers, since it forces the new owning team to work backwards from the end-customer thereby gaining context on the service.
That said - start with monoliths, paying close attention to inter-package dependencies and defined interfaces. Resist the temptation to "call into" a private part of a package just because it's easier than exposing functionality in an interface. Over time these small efforts add up to make your one-day migration that much easier. In the meantime, you don't have to coordinate deployments, worry about supporting a variety of tech stacks, or even worry about the security aspects of cross-service comms.
I have no clue how some of this works in Hack. Compared to Java/Kotlin, packages don't seem to really exist, but method call protection does. I look forward to learning more.
Honestly - I think this applies to any level of leadership, and there's a lot you can take away even as an IC.
It also reminded me of these two stories from the codeless code (which I highly recommend reading - it's very entertaining):
- Did you not understand the terms of your own promotion? We are the feathers.
- For the first rule of herding is that you must produce a herd, and a herd is a nebulous thing, midway between order and chaos.
There's no worse thing than learned helplessness - when people stop trying to help themselves you've fallen into the pit of bureaucracy. But lacking a vision, a common uniting theme that drives a team/org is almost as bad. Those are the two things that I've seen exemplified in great leaders - a clear vision of where to go, and an uncanny ability to unblock both themselves and others.
I've never been sold on TDD but I have been sold on parts of it. I tried it for a month, but writing tests first led to a myopic view of my code which I wasn't a fan of. Having tests was great - but across all the teams I've worked at we've had 85+% lcov, 90+% bcov, and a slow of end-to-end tests as well. So TDD wasn't really driving that - wanting to deliver the best quality software drove that for us.
Personally, I've always believed in documentation driven design: you write the class documentation before you write your code, and then you write your tests. Writing works well for me in that it forces me to crystallize my thoughts and decide what I want said thing to do. It also facilitates a top-down approach to writing code, which I've found leads to better interfaces. Alongside XP and a willingness to refactor and change your mind, this is what's led to my best code output. And of course, don't forget your tests.
When it comes to refactoring, remember Chesterton's fence:
There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.
- GK Chesterton
TTL for software, open source taking over the world
I'd never thought about what the rise of open-source software (and to a lesser extent hardware), meant for commercial software. But Andre makes a good point on discerning what's being commercialized. 90% of the time, it's not the software, but the underlying asset, conveniently packaged. For Uber - it's the car that'll get me from A to B. For Google it's their search algorithm. For Facebook it's the promise of connectivity, and better ad targeting. Similarly for the recent trend towards image-generation AIs: in theory you could build it yourself, but it'd take too long. What you're paying for is the training. Not the software.
I don't know if I agree that software will keep getting cheaper. I think it's still true that commercial apps do a lot better than open-source ones, in part due to the support they provide, and in part due to the quality they can afford. That said, Android vs iOS shows that you can close the gap - but I wonder how much of that is due to Google supporting Android as opposed to it being a true open-source project.
Pitfalls of event-driven architectures
100% true, but with caveats:
- Write to db and then fire event without atomicity: use change-data-capture compatible DB with retries. Some will fail, but you'll get 99.99% durability, and do you really need more than 4 9s? Yes - this is where reconciliation comes in but wait till you get to 4 9s to aim for your 5th.
- Using Event sourcing everywhere: ahh CQRS taking over the world - just don't.
- No context propagation: tag with a trace GUID, and have good search solutions to query and figure out where things are going.
- Large payloads: reference object store - compression doesn't do enough, and chunking complicates the protocol. Instead just point to the payload in S3 (or your blob store of choice).
- Duplicate event management: leverage idempotency - remember at least once is all you're going to get without transactions and rollbacks.
Don't use events everywhere, but they're a great tool for loose coupling.
Building quality software, fast
My read of the situation is that this is what Meta's aiming for: building stuff quickly so we can go to market and make $$$. But not so quick that we can't iterate and add more features to make even more $$$. This is where the culture of total ownership comes in. The simple argument on the tech side is that quality speeds up subsequent delivery, which is true. But it assumes developers focus their efforts on areas of subsequent investment. This has rarely been my experience - I've seen devs polish something that was known to be deprecated / on the path to deprecation in ~1week.
It also goes beyond software development. We need to focus our efforts, and extract commonalities - so we build the truly minimal viable product, and thus the minimum viable architecture.
This is turn informs what I like to focus my time on. Making sure that we build the right things, at the right level of polish, to solve an acute pain point. Software quality is a lever that helps me do so, not an end in and of itself.
Links
- Using canary resources in AWS as decoys to honeytrap suspicious activity
- A bunch of python data-structures & algorithms questions, for if you're interviewing
- Publishing on substack, I'm working on a book so this seemed particularly fortuitous
- The marriage problem, it's always surprising to me where e and pi show up
- Conversational doorknobs as lubrication
- Twitter sucks at security, maybe?
- Writing well
- Estimating projects well and planning for deadlines
- EventBridge supports 3P events via webhooks
- Building your own OTPs
- Good questions for your first 1:1 - for all the managers out there
- At 15kB your site is going to be 500ms slower, blame TCP slow start
- 10x-dev is both too extreme and not extreme enough
- Github Copilot might hobble CS education, who knew. Remember kids, say no to Copilot. I guess CodeWhisperer falls in this bucket as well.
