Sometimes It Is Too Late
I saw a tweet a couple of weeks ago. I can’t find it anymore and I wish I could, so I could get the wording right.
Essentially, it said “A good engineer is never afraid to scrap everything when they find a better solution.” But, you know, phrased more memorably. The general idea was that it is never too late to start over. It’s a great sentiment that I happen to agree with.
It’s not always possible. If you are (or seem to be) making good forward progress on an important project for your organization, there is often huge pressure to keep up the momentum.
It can be very tempting to say “we’re being agile, we can just refactor it in the next iteration.” No problem, everyone thinks, we’ll get it sorted out. We won’t be taking on this technical debt for long.
Oh, how wrong hypothetical-everyone is.
Much has been written on the subject of technical debt. By people much smarter than myself. But this is more than technical debt.
Everyone is so jazzed about making progress that they are able to rationalize it away. They can write it down in the tech debt column and worry about it later, but really “re-design the entire system from scratch” isn’t traditional tech debt. It’s how you get into a cycle of “complete re-write every 2 years” or - perhaps worse - “crusty, old, unmaintainable-and-frankly-baffling system nobody understands still hanging around 8 years later.”
You want the project to get done, but you want it to get done right. How do you find the happy medium?
It sounds pretty simple. It’s not really very simple.
It’s tempting to jump right at the first solution that occurs to you and get down to work. Don’t do that.
A big part of getting to a good solution is understanding the domain. I mean really understanding.
How many blog posts have you read from people or organizations that are re-doing a system. Heck, everyone has first-hand experience with this. How many times have you looked back after 6 months with a “what was I thinking?”
Specifically, I’m thinking of a number of blog posts I’ve read from organizations switching from
[Configuration Management Product A] to
[Configuration Management Product B]. We’ll say Puppet to Ansible because that’s the pattern I’ve noticed.
These posts invariably detail their grand masterplan to implement Puppet. And things go great at first. And Ops speeds up. Then their Puppet tree gets crazy and unmaintainable. Changing and applying configuration becomes a real risk. Perhaps even riskier than changing things manually. And Ops slows down.
Someone then has the brilliant idea “let’s switch to Ansible.”
They throw away their Puppet and re-write it in Ansible. Miraculously, their Ansible tree is cleaner, more maintainable, faster, better. Maybe some of this can be attributed to Puppet vs Ansible. But really, most of this is down to actually understanding how to use Configuration Management Tools in general. Understanding the problem they are trying to solve. I suspect that simply reimplementing Puppet from scratch would have been just a good.
I’m not suggesting that you should take forever to fully solve the problem in several different ways. But do real research with an open mind.
Figure out other people have solved your problem. Because other people have probably solved your problem.
I’ve noticed that people (myself included) have a tendency, when first learning something, to say “this is stupid.” Or sentences starting with “why can’t I just…”
Because, generally, there is a good reason why you cannot just.
Very likely, your use-case is not so specialized that you need to contort existing technologies into unrecognizable shapes.
Understand the intended use case before you go claiming that you have an extraordinary one.
As I mentioned above, there can be tremendous pressure to keep forging ahead. To get something that works well enough and iterate on it later. In all the excitement you may even think that it’s a great idea.
Just make sure you have something worth iterating on.
Before you get in too deep, ensure that you have a solution you are ready to be married to. Or ensure that you can escape the situation and stick someone else with it.