The Software Ark: Issue 3
Quote of the week
I have self-doubt. I have insecurity. I have fear of failure. I have nights when I show up at the arena and I'm like, 'My back hurts, my feet hurt, my knees hurt. I don't have it. I just want to chill.' We all have self-doubt. You don't deny it, but you also don't capitulate to it. You embrace it.
- Kobe
Articles
Etherium's difficulty bomb as a way to force compliance and innovation
My biggest problem with distributed enterprise is the lack of single threaded ownership. This leads to forks, variants, and a whole bunch of randomness. ETH has always recognized its waste of resources and energy, but I recently found out about a "difficulty bomb" built into the chain. This bomb makes it exponentially harder to mine eth and drives down adoption of the network - since transaction rates fall as well. This in turn forces a move to POS. I wonder if we can use this idea in other places: set up a time-bomb to detonate unless you fix tech-debt by a certain date. I doubt most companies would go for it, but it's a fun thought experiment.
This article walks through how to run a proper DSU.
As an industry (though thankfully not on any team I've directly been on) we've started to walk the wrong road. We've gone from getting things done to tracking work. With Scrum / Kanban / variants of Agile, the purpose of a stand-up was to track progress towards a goal, and course correct as needed. Not "work harder" since we're behind. Remember that when you're at risk of missing a date (on a high functioning team) there are only three levers you have: change the date, change the scope, or add more people. Working harder is not an option.
Related: SPACE - measuring dev productivity. I read this a while back and found it to be a good framework to think of productivity in a team.
I'm a sucker for any article database related. Unfortunately, they tend to be poorly explained. Not so with this article that walks through how Redis works. The diagrams really help convey the message, and they make me wish I was better at illustrations. That would be a course I'd pick as an elective 100% of the time.
Yet another AI for beginners course, courtesy of MSFT
I did not know that MSFT produced curricula, so I was psyched to learn about this self-study course that walked through the basics of AI in industry. Complete with PyTorch (yay, Meta!) / TensorFlow examples. I've got a lot of courses to mess around with - might finally be the excuse I was looking for to pick up that 3080.
They also have courses on Web Dev, ML, and Data Science - I've got to look at those as well.
The tagline explains itself very well: "About the many types of hash functions, their use-cases, dos and don'ts, with suggestions for currently accepted algorithms."
Review: Modern Data Infrastructure Architectures
I used to work on big-data tech before and helped support a team that was working on building out a data-mesh. This article walks through common data-infrastructure set-ups in industry. I wonder how long it will take before this entire thing gets packaged into an AWS offering - we're halfway there with AWS Lake Formation + Glue, so just a little bit more to do.
What I will say is that I had no idea that half of these open-source projects even existed. I've got a lot of sleuthing to do. My pending reads list just keeps growing.
We will watch over your career with great interest.
To be honest though - I'd rather opt-in to type-checking, most of the time it's an unnecessary hassle. It's why I use python for my local projects. At the same time, it's also why I don't use python for enterprise things (that + the GIL restriction), but if this fits that niche then we might be in business. Imagine an opt-in type-checked python variant with a JIT - then we might have a green user-friendly language at long-last. Look at how bad Python does today. Abstractions come at a deep cost.
Retaining attention when presenting
Be succinct; be clear; and focus on the why not the how. Presenting at Amazon came with its set of difficulties. We had a slew of "best-practices" that helped a bunch. The biggest lesson I took away was on building a convincing narrative; one where readers can fall off as / when they need to. Start off with the why - the problem statement; then a summary of what a good solution looks like + tradeoffs; then high-level overview of the how; then details on the how; and then considered alternatives. This way if someone just wants to make sure you're on the right track, they can stop as early as "what a good solution looks like", whereas fellow engineers can read on till the gory details. Narrative structure is key.
Related: communicating as a tech-lead. Lead by federated authority, not by fiat. That requires building consensus, a shared vision, and trust.
The one thing that's worked for me has been to ask the question "where can I best help the team". If you do that, and make sure the work is visible, then you're good. Sometimes that might mean doing the boring work of say "dev efficiency" (for me that's not as much fun as building a product), sometimes it's leading by example and getting things out the door ASAP, and yet other times it's taking that step back and showing how you avoid burnout. If it's what the team & org need, it'll pay dividends in your career.
Ultimately, you're looking for something that can sustain that flow state; that is an Ikigai.
I love the JVM - Java, Kotlin, Scala, Groovy - I've used them all at some point or the other. I'm not a massive fan of Java, but it's tooling and std-library make it a go to option for getting things done. Second only to C++ in a sense. That said, I do have to give it kudos - a 6-month release cycle, new features like "vars" and type inference, not to mention better async support. Java is IMO on the 75th percentile of adoption - which is the right place for something enterprise-y. New features come to java constantly, and by the time they get there, the kinks have been worked out. It's great!
How AWS benefits from re:Invent and deadlines
While this article talks about re:Invent in particular, I've always held the position that deadlines help pressure teams into making careful tradeoffs. This in turn makes for better engineers (since you can only choose the wrong tradeoffs so many times before you're in tech-debt hell), and for better user-experiences since stuff is getting out on a regular cadence. While you could in theory do this without any kind of deadline - in practice it doesn't work that way. What you've got to be careful about is hitting deadlines while not working more than 40 hours a week (or really 36 - you got to take some time for lunch).
Python concurrency APIs: choosing the right one
Python has always been batteries included, and with the inclusion of aio, in addition to threads & processes, we have way too many options and it's not always obvious which one fits where. This article aims to remedy that. It is overly reductive though - the true answer is to profile, profile, profile. But it's a good place to start.
Links
- Terminal dashboarding
- Clojure - my favourite lisp, no longer last to be picked
- Apple asked Meta for a cut of ad-sales... wtf
- A free version of Dall-E-2 to launch soon... get ready for the influx of fake news posts
- Language models can teach themselves to program better
- Amazon expanding its use of palm-prints, do you trust them to keep it safe?
- Why inheritance doesn't make any sense
- How to pick a starter project to make someone quit
- Building maintainable & usable python libraries
- Python 3.10.6, hello union types, hello structural pattern matching
- Shaming Apple into supporting RCS
