Entropy in Software and the Broken Window Theory
Yes, you heard it right: Entropy in Software and the Broken Window Theory. It might seem a little strange at first, perhaps you're even asking yourself "but what do these concepts have to do with the world of software?". These two concepts are not terms commonly associated with each other, but upon further exploration, we discover that these two theories can offer a unique perspective on the effective management of software development and maintenance. In this article, we explore how these two concepts interrelate and how they can be used to improve software development practices.
Entropy
Yes, you heard it right: Entropy in Software and the Broken Window Theory. It might seem a little strange at first, perhaps you're even asking yourself "but what do these concepts have to do with the world of software?". These two concepts are not terms commonly associated with each other, but upon further exploration, we discover that these two theories can offer a unique perspective on the effective management of software development and maintenance. In this article, we explore how these two concepts interrelate and how they can be used to improve software development practices.
Entropy Sounds like a complicated scientific term, doesn't it? Don't worry, together we will demystify this word and understand what it really means, especially when applied to the world of software. Grab a coffee and let's go! ☕
Let's start in the 19th century, with a scientist named Rudolf Clausius. He was the one who introduced us to entropy in the world of thermodynamics. But how can this be compared to software development? I promise we'll get there, but first let's understand a little more about what Clausius meant by entropy.
Entropy, in the field of physics, is a way of measuring unpredictability or, as some prefer to simplify, the "disorder" within a system. Now, when I talk about "disorder", I don't mean mess in the common sense of the word. In fact, I'm talking about how energy is distributed in a system.
Imagine we have a box full of particles. If all these particles are concentrated in one corner, we can say that entropy is low - there is a certain "order". But, if the particles are randomly scattered throughout the box, then we have high entropy - the energy distribution is more "unpredictable" or "disordered". Nature tends to the state of higher entropy, that is, the particles will disperse, unless something external intervenes. It's as if nature likes a bit of unpredictability.
Now, it's time to connect this idea with the world of software 🌉. When we talk about entropy in software development, we are referring to this natural tendency of software systems to become more complex and "disordered" over time. And when I say "disorder" here, I'm talking about things like poorly structured code, insufficient documentation, uncorrected bugs, code smells, and so on.
What happens is that as software grows and evolves, constant effort is required to prevent this entropy from increasing. We need to work to keep the system organized and efficient - similar to how the particles in the box require an external force to not randomly scatter.
Now that we quickly understand what entropy is, let's understand some points that can contribute to its increase within software. I will take this part to comment on some that, unfortunately, I have also witnessed in the software industry.
The lack of clear requirements!
Let me share with you a story that, although fictional, could easily pass in the reality of many software development teams. I promise it's a story with a valuable lesson for all of us: programmers, project managers, product owners and anyone interested in building successful software products.
Think of an innovative startup that is developing a food delivery app. Among the many features of this app, a special requirement arises: the option for users to share an order with friends, allowing everyone to add their favorite items to the same order. This seems like an incredible competitive advantage, doesn't it?
However, the description of this requirement is a bit vague and does not go into details about how this functionality should be implemented. Should there be a limit of users per shared order? How to handle the division of delivery costs?
Our development team, always seeking efficiency, interprets the requirement in the best possible way with the available information. They decide that there will be no limits to the number of users in a shared order, and that delivery costs will be equally divided among all participants.
When the feature is launched, the reaction from users is mixed. Some love the idea, but others are frustrated. They don't think it's fair that a user who added just one small item pays the same delivery fee as someone who ordered several larger items. The lack of limits also causes problems, with gigantic orders causing delays in deliveries.
The lesson here? Poorly defined requirements can lead to misunderstandings, imprecise implementations and even negative user feedback.
Therefore, every time a new feature is proposed, it is crucial not to accept requirements that seem insufficiently detailed or that are not coherent with reality. This is a shared duty: the domain expert team must strive to clarify the functionalities, while technology managers need to be ready to question and clarify any requirement that seems vague.
And how does this affect and contribute to the high disorder of the software?
When a feature is implemented based on poorly defined requirements, it can lead to poorly structured code or a software architecture that is not aligned with the true business need. This can cause problems, such as maintenance difficulties, performance problems and even the introduction of bugs. Additionally, if developers need to revisit the feature continuously to correct misunderstandings or implement changes, this can cause even more complexity and "disorder".
This generates stress for the team. After all, probably features that were not planned correctly need new definitions and data modeling. Perhaps new tables need to be created, new relationships and entities. This consequently can generate unrealistic delivery deadlines to catch up. We know that many companies leave quality aside in critical moments, many new features or even existing ones may need a complete review, but this is not done with quality. Do you see the cascade of problems generated? This is the disorder!
Therefore, poorly defined requirements increase entropy in both the software and the business. They contribute to more chaotic and complex code and can lead to unwanted business outcomes. At the same time, dealing with these requirements and their consequences demands time and resources, which could be used to create new features or improve other aspects of the system. That's why it's so important to strive to clearly define requirements from the beginning.
Poor Communication Among Teams!
Let's explore another common scenario in the world of software development. This one is especially relevant if you have ever been a part of a development team.
Let's picture a startup, working on a social networking application. We have two teams; there's a developer in charge of building an API, which is the heart of the application's notification system, responsible for alerting users about new likes, comments, and messages.
On the other hand, we have another programmer, in a different team, whose task is to develop the user interface (frontend) that will display these notifications. This developer relies on the API to get this data and display it in an intuitive and appealing way to the users.
Now, imagine that the developer responsible for the API, in pursuit of excellence, makes some significant improvements, optimizing how data is delivered. However, he forgets to inform his colleague who depends on this API about the changes made or about the new version of the service.
The frontend developer, completely oblivious to the update, continues to work with the old version of the API. When the application is launched, users start complaining that notifications aren't working as expected, or even appearing wrong.
The above situation illustrates how a simple slip in communication can lead to an increase in software entropy. This dysfunction not only introduces bugs but also can cause frustration and confusion in the team, and worse yet, dissatisfaction in the end-user.
Therefore, the lesson here is clear: effective communication is crucial for controlling software entropy. Keeping everyone on the team informed about changes and updates can prevent misunderstandings and ensure everyone is aligned, contributing to more efficient software and a more harmonious work environment.
And how does the above situation contribute to high disorder in the software?
The situation described above significantly contributes to high software entropy, or disorder, in several ways. Let's break it down to understand better.
Outdated or Incompatible Code: The developer working on the user interface is using an outdated version of the API. This can lead to inconsistencies in the code, compatibility errors, and even system failures. Additionally, it can impact the system's performance.
Unnecessary Effort: The team will now have to spend extra time fixing the problems caused by the lack of communication. This means lost development time that could have been used to add new features or improve other aspects of the software.
Introduction of Bugs: The user interface that depends on the outdated API might not function as expected, leading to bugs. Depending on the severity of these bugs, they can affect the user experience and even lead to data losses.
User Dissatisfaction: The user experience might be damaged due to poorly implemented functionalities or bugs. This can lead to negative feedback, loss of users, and damage to the product's reputation.
These combined factors increase the "disorder" or entropy within the software project. They create a situation where complexity and confusion prevail, making the implementation of improvements and fixes difficult. This is why good communication is so crucial!
Frequent Changes in Requirements!
I imagine you've already encountered a situation where the requirements for a software are constantly changing, right? This can be a major challenge. Such changes may include new functionalities, requirements, and processes that were not initially considered. And what does this cause? Increase in software complexity, introduction of bugs, and difficulty in maintaining and evolving the system.
To illustrate, let's imagine a programmer developing an inventory management software. During development, the requirements are constantly changing: new types of products are introduced, discount rules are altered, and product information varies. This constant adaptation to changes can result in an uncertain and constantly flowing software architecture.
This lack of stability and clarity increases the entropy within the software. The planning phase seems to never reach a definitive conclusion, while the requirements list grows and transforms. The addition of unnecessary or redundant code paths, in response to these frequent changes, can lead to even greater complexity, making it harder to understand the code and maintain it.
Furthermore, with each new change, there's a higher risk of introducing bugs and failures into the software. After all, each change requires careful review and thorough testing to ensure that it's not introducing new issues. Unfortunately, with tight deadlines and ill-defined requirements, this crucial step can be neglected, which only increases the system's entropy.
Finally, what does all this have to do with the code we write? Well, everything! The more complex the system becomes, the harder it is for us, developers, to understand and maintain the code. Let's talk more about this subject.
Unwanted Code
The lack of dialogue between teams, the lack of clear specifications, and frequent changes in requirements can negatively impact the way we write code in several ways:
Let's start with Inconsistent code. Imagine a team of developers working on a project, but with poor communication among themselves. This can result in different interpretations of the requirements and therefore inconsistencies in the code.
Now, think about Poorly maintained code. Nebulous specifications and constant changes can lead to a tangle of "dirty" code, similar to a poorly maintained building, which becomes increasingly difficult to maintain.
Next, we have the so-called "code smells". Imagine walking down a street and smelling a foul odor. It's a sign that something is not clean. In our code, the infamous code smells, are aspects that suggest potential quality or design problems, usually caused by frequent changes in requirements or even lack of experience of the programmers, lack of adoption of good practices among other factors linked to code readability.
Redundant code is also a problem. Without clear specifications, we may find parts of the code repeated, increasing complexity and maintenance time, mainly due to a lack of clarity and the adoption of the DRY principle.
And what about the increase in bugs? This can occur when there are communication failures and the specifications are not understood correctly. But they mainly occur due to the absence of automated tests to validate the behaviors that are expected by the software.
How do many deal with these cited problems when they become unsustainable?
Developers may consider significant refactoring or, in extreme cases, rewriting the software from scratch. But beware, these are short-term measures. If the team does not adopt a consistent plan to control future entropy, code deterioration will return. But what does all of this have to do with the broken window theory?
The Broken Window Theory In the book
The Pragmatic Programmer, the authors argue that software entropy is contagious and, if not controlled, becomes an epidemic. Furthermore, they make an analogy with the broken window theory. Let's delve into this further.
Proposed by criminologists James Wilson and George Kelling and having a massive impact on police policy during the 1990s, the broken windows theory is an idea that suggests physical disorder in an urban environment, such as graffiti and broken windows, can lead to an increase in crime in that area or degradation by vandals at the site.
Well, you might be wondering, "But what does this have to do with entropy in software?" The answer is: much more than you might imagine!
Consider code with inconsistencies, redundancies, bugs, or even code smells. Each of these elements can be seen as a "broken window" in software engineering. Leaving them unsolved, or worse, allowing them to accumulate, generates increasing disorder - that is, software entropy. This high entropy makes the code increasingly difficult to maintain, understand, and evolve.
Similar to the Broken Window Theory, poorly maintained and disorganized code can encourage bad programming practices. Imagine a new team member encountering code riddled with inconsistencies and bugs. If there isn't a conscious effort to maintain code quality, this new member might be led to think: "If the code is already in this state, why should I strive to write quality code?".
The consequence is a vicious cycle, where the software quality deteriorates, productivity decreases, bugs increase, and entropy becomes uncontrollable. Just like in a neighborhood house where the broken windows are not fixed, disorder ends up taking over.
We can add even more, confusing codes without documentation and inadequate tests can indicate to developers and project managers that quality is not a priority, leading to behaviors and decisions that are inappropriate for the context. This can include adding new features without considering maintenance, omitting necessary refactoring, and not monitoring code quality. These inappropriate behaviors can lead to a domino effect of problems, where code entropy gradually increases.
The relationship between the Broken Window Theory and entropy is this: both are ways of understanding the importance of continuous maintenance and attention to detail. Both in a city and in a software system, disorder sets in when we allow small problems to accumulate. And the way to combat this is through constant care, regular maintenance, and attention to detail.
What can we do about this serious problem?
The first thing to keep in mind is that quality and maintenance are not aspects to be considered only at the end of a project - they should be present from the beginning. Stop thinking that testing and refactoring are a waste of time!
Secondly, maintain communication. It sounds simple, but it is crucial. It's necessary for all team members, whether developers, managers, product owners, or other stakeholders, to be aligned in terms of requirements, deadlines, and objectives. In this way, we can reduce misunderstandings and avoid inconsistencies that contribute to entropy.
Thirdly, have a clean code policy and follow best coding practices. Here we include consistent coding standards, code reviews, automated tests, and a proactive attitude towards refactoring. It's not just about writing code that works, but about writing code that others can understand and maintain.
Fourthly, we should pay attention to the architecture of our software. A well-planned and scalable architecture reduces the likelihood of us having to make drastic changes in the future, which can introduce new bugs and increase complexity and entropy. An architecture planned in a hasty manner leads to unnecessary complexity and conflicts of ideas among developers within the project.
Lastly, effective change management. Changes are inevitable in any project, but the way we handle them can make a big difference. An agile approach, which encourages continuous feedback and adaptation, can help manage changes in a way that reduces entropy instead of increasing it.
Each broken window we fix is a step in the right direction. Every bug fixed, every inconsistency eliminated, every "code smell" resolved contributes to the control of entropy. And, by keeping entropy under control, we ensure that our software is sustainable, scalable, testable, understood and, above all, easy to modify and work with.
How do tests help keep entropy under control?
Tests continue to be an underestimated tool by many developers in the fight against high entropy in software. Why? Have you ever tried to assemble a puzzle without the reference image? Well, if you have, you know how tricky it is to put together the picture. Tests, my dear readers, are like this reference image in the construction of our big software puzzle. Without a clear picture of the system, we can't know exactly what software does, that is, its purpose. Its main features and objectives. Tests reflect the behaviors of the software, they are a mirror of the software, which we can run and understand what each piece of code is responsible for!
Software tests, unitary, integration, functional, stress, among others, have a crucial role in the prevention and control of software entropy. How exactly?
Detecting problems earlier: Well-written tests help identify bugs, inconsistencies, and failures earlier when they are easier (and cheaper) to fix. Regularly run automated tests ensure that any "broken window" is promptly detected, preventing the spread of entropy.
Preventing regression: As the software evolves, it's easy to break existing features without realizing it. With a comprehensive test suite, you have a safety net that helps prevent these problems, ensuring that the software continues to function as expected after each change.
Living documentation: Tests are a kind of living documentation of the system, which describes how it should work. By consulting the tests, new team members or even veteran developers can better understand the code, keeping the codebase consistent and minimizing entropy.
Improving code quality: Tests encourage developers to write more modular and independent (easy to test) code, which in itself already contributes to reducing entropy.
Provides confidence to refactor: Improving your design without changing its behavior is a necessary activity to keep entropy under control. Tests provide the security that, if something breaks during refactoring, it will be promptly detected.
In the book I'm writing that I plan to announce the name of soon, I talk about these points from a more practical perspective for the software industry. Really when we take tests very seriously, we have significant productivity gains in everyday life when programming new features and maintaining those that already exist. Obviously, it is not a simple task, there are challenges and above all commitment to quality.
In conclusion, the Broken Window Theory teaches us that tolerance for small faults or "broken windows" in our code can lead to a cascade of problems. Developers may start to think it's acceptable to produce low-quality code, or ignore tests, thus increasing entropy. By embracing good practices and keeping a keen eye for "broken windows," we can keep entropy under control and ensure our software projects remain healthy and sustainable.
Again, it's easier to write than to put into practice, so I'm centering this theme in a book to approach with more freedom and depth this complex and interesting subject that is part of our daily lives.
I'll stop here. I hope you enjoyed it! Thank you so much for reading until the end, I hope the tips can help in your daily life! Until the next article! 😄🖐️