Everybody who has managed software projects has come across the term: technical debt. No software is free of it. What is it? How is it created? Is it really that bad? Can we get rid of it? This deep dive aims to clear these questions up, and answer wherever possible.
The name originates from Ward Cunningham, who was looking for the right terminology to explain to his boss at WyCash why code maintenance is important. The issue at hand was the creation of good, reliable software, capable of delivering consistent results. This means asking the same question twice in a row should result in the same answer. In refactoring, updating the code, the result of this work should maintain the exact same capabilities as before. In short, no immediate user value is created, so at first glance we could say we have simply burnt money.
But Ward realized the logic of what was happening behind the scenes, and he chose a term to discuss the question in a language familiar to his stakeholders. He understood that to lead a successful software project, it is not enough to create the requested capabilities for the product; you also have to meet deadlines and keep within budget. And in order to reach these goals, sometimes you have to cut corners.
These shortcuts do not necessarily mean poor quality code; sometimes they just result in less flexible solutions. In other words, saving time and effort on the front end may result in paying the penalty of difficult future changes later. In Ward’s terminology, the fixed capacity of the team is comparable to a fixed budget, and the increasing effort needed for future improvement is the same as paying back interest on the debt you took today. (Refactoring in this sense could be seen as payback - so that the debt does not keep on accumulating.)
At times, taking on technical debt may be the right thing to do. However, it will undoubtedly limit your capabilities in the future, so technical debt should be addressed as soon as you can afford to do so. It is inevitable that you will find yourself in the same situation again, facing the difficult decision to compromise by trading rapid changes for the accumulation of even more debt, further persisting the problem.
When you zoom out from software development to software ownership, the perspective changes. The greatest value of software is that it is soft: you can always tailor it to your actual needs. But this is also the source of its greatest risk: it is always changing, and one desired change can always force many more changes on you.
Imagine a software system with no loose ends, no corners cut, everything is perfect, everything is state-of-the-art. You install it on computers, lock them away from the world, set them in stone. Can you build your business on it and avoid change forever? Of course not. Your requirements are going to change, your business is going to change. The world will inevitably change, and you must adapt alongside it. The same goes for your software.
As time goes by, new vulnerabilities are going to be discovered, and new methods of attack are going to be invented. You can not avoid all threats by simply locking away your computers from the world. Even if you bet on this strategy, new government regulations can force you to apply precautions anyway. The best you can do is update continuously. But software products do reach their end-of-life and you might face an inevitable upgrade to a new version which potentially brings breaking changes to the table.
When you decide to use software to solve a particular problem, you expect significant speedup and cost reduction. If you can bring a larger part of your business into these systems, you will be able to make more money - or at least be able to save more by minimizing effort. These motivations push you towards a variety of computerized solutions, but if your time is spent on jumping from one system to another, then your pains can outweigh your gains. The answer is to build an integrated system, and let the computer take the burden from your shoulders.
The better your system represents your business, the more value it creates for you. Also, as your business grows, your system will become more and more complex. If you support your work with a homegrown software system, built from independent components found on the market, you also take on the task of continuously shaping your solution - or take the risk of suddenly facing an obstacle you cannot get around fast enough.
There is no surprise: integrating independent solutions is practically the same thing as how software is built from small building blocks. Therefore, it follows that this approach will also naturally result in technical debt. All of your excellent decisions today can turn into expensive shortcuts tomorrow, despite how great of an idea it seemed at the time.
One of the decisions you have to make is to choose between free software and commercial. In reality, there is no such thing as a free meal. Every free or open-source software results in the assumption of risk: you bet your business on the community that produced that piece of software. Even the most active communities can change direction and re-prioritize ignoring the specific needs of the largest companies if they don’t see the value themselves. What happens then? You can take on the development or specialization of the code yourself, turn to paid support, or you can spend money on software replacement. As a last resort, you might even be forced to tailor your requirements to the capabilities of the code. No matter what you decide, the outcome is the same: lost time, money, and resources.
At Chemaxon, we are quite familiar with this. We spend tremendous effort to ensure backward compatibility. The same code examples we have created as an integration offer more than 25 years ago are still working today on bleeding-edge technologies. We are fighting the wheels of time day-by-day, so you can upgrade our components in your system without worrying about the sustainability of your in-house solutions. If you have started to use Marvin in 1998, you can still use it today on the most modern operating systems and Java versions.
We don’t just keep old software alive, we also create new solutions to better suit the needs of today. As Java was banned from browsers, we were waiting for you, ready with MarvinJS. As more and more of the industry attempts to move their in-house systems to the cloud, we continue to support them throughout this process. Nowadays, everyone is moving toward serverless solutions, and we are prepared with our ready-made containerized solutions. And what is more, we have already started to work on the SaaS of the future.