Stephan
Schielke
Agentic Workflow & Context Engineer
Jack of all trades and master of some
For over two decades now, I have devoted (nearly) every waking day of my life to the absurd mission of making myself obsolete through software automation. Yet I failed to automate myself out of a job thanks to the continuous efforts spent that further improve software systems while finding ever more new ways to increase their efficiency & reliability. Though now, for the first time in my and anybody else's lifetime, there is a realistic way forward to achieve that dream of full automation.
Engineering Philosophy
Even though there is an optimal point for deciding to automate a task rather than manually repeating it over and over again, we mustn't neglect, let alone forget, the additional overhead any extra step in a process or pipeline creates. In my experience, we often underestimate how long tiny pain points persist and accumulate into a thousand cuts. For me, it is less about the value of automating a problem away (it's always worth the effort measured against a decade of pain as the alternative) and more about whether one can afford the spike in effort right now, given potentially more urgent tasks. But if you can find the time, spend it on automation.
Without it, you are flying blind, will keep endlessly searching and sifting through code and unstructured logs, increasing your debugging efforts tenfold, and overall will have a hard time growing your system beyond simple "Hello World" territory. In my experience, creating and utilizing a solid observability framework can easily be achieved from the start and get-go in any project without the falsely feared additional overhead. I think of it this way: Every time I'd decide to write a console.log(e), print(e), or echo x rather than investing in setting up a base logger library, request tracing framework, or metrics gathering system, I with 100% certainty will lose time by having to revert the debug code or running the program multiple times to find the part, and will end up being less efficient (and happy) than I could be right now. Observability is a solved problem. Even if you can't afford to buy into the dozens (if not hundreds and thousands) of professional frameworks, setting up and growing your own shared base library is trivial and should be an enforced part of every project template in your organization.
It's straightforward, really: Do it always and continuously or go home. Stop finding excuses for not doing it. Make your and your team's life 100 times simpler by always linting and formatting on every save; compiling and building before every commit; unit and integration testing on every push; deploying on every pull request (and update); and releasing every successful pipeline artifact straight to prod. Your goal should be to create a well-oiled loop with the shortest possible feedback time. Only after this basic CI/CD system is in place, you can (and should) go wild with additional CI extras (build caching, file watchers, custom hooks, parallel test executors, notification loops, artefact signing, secret and vulnerability scanners, SAST/DAST, …) and CD extras (A/B, blue/green, canary deployments, automated rollbacks, feature flags, env promotion, Service-Level-Objectives, error budgets, deploy-freezes, approval gates, mutli-region, …), but not before.
Every layer in your stack, from bare metal, up and down through the ISO/OSI layers, and your cloud architecture, into your service, has levers to crank up the volume. Don't get lost in optimizing parts of your system, spending hours and days on optimizing that one slow-running SQL statement, managing to speed it up by 1000%, feeling massively proud of your achievement, if in reality that SQL only ever gets executed once during the yearly ETL report pipeline… Check if it's worth it, shaving off a millisecond on that one thing that gets executed a billion times a day, versus spending your entire week shaving off an hour of a process no one cares about. My point is that very often you can find tiny improvements, "quick-wins" or unsuspected bottlenecks in all sorts of places that will free up capacity on the other end, overcoming your scaling challenges.
I love writing documentation. There, I said it. No, but really, it's one of the best parts of the job, and I believe can only be done well on a finalized, finished, and working piece of new software you fully understand yourself. That's the moment when you can highlight your genius (or madness) to the outside world. But if you can't put it in words or fail to deliver a clear piece of documentation of your work, you might as well delete it and start from scratch. This point becomes even more important in the age of agents trying to figure out what the hell this piece of software is trying to do. Give an agent bad, outdated, misleading, inaccurate, too dense, or too thin of a text as "documentation" to use as context for maintaining and extending your module, and it will most definitely not do what you want it to do. Don't be a scumbag and leave the next engineer/agent alone with no guidance on what the hell you were thinking.