Keeping Tests Valuable: Think about the Inputs!
For unit tests to be really effective, it is essential to think about inputs!
In this article, we will discuss why it is so important to avoid incorrect inputs, to think about broader inputs in unit tests, and how this practice can directly affect the test results. We will understand this in practice with an example. I hope you find the article useful!
📍 We need to think like the user
The example we are going to comment on demonstrates the importance of thinking about other scenarios and inputs for unit tests. This way we can ensure a more robust set of tests that care about the behavior of the functionality in a more broad way.
Suppose we have an IsPalindrome() method that checks if a string is a palindrome (if it can be read both from left to right and right to left and still be the same string). Nice, we know that then the word level is a Palindrome. So let's write the code and then the test:
Look closely at this code and see if you figure out your problem. I will paste
here the unit test for this method:
Apparently all right. If you execute this test, everything will run successfully. But what if the developer inserts that input below?
var result = IsPalindrome("LeveL");The developer runs the test and everything passes successfully! Cool, then we can send the code to production and get rid of this task and pull another one because that is what your manager wants, right?
Note that here we only tested two different inputs, even if these inputs are valid and make sense for the context and the test, don't you think there are too few scenarios explored? Well, let's see how this story ends.
Well after a few hours, a user complains that the word leveL is not recognized as a palindrome. The developer who wrote the feature is startled and thinks; "How is this possible? I even ran the unit tests!".
However hypothetical this situation may be, it happens quite often. The problem here is that to the human eye, these variations of the same word below are the "same thing", and have the same meaning:
"Level", "LeveL", "leveL", "level" // differents inputsBut to a machine, there are differences. Is the word written leveL the same as the string level for a human?
To a non-programmer, it could be, but those who have programming knowledge know that programming languages can make distinctions between these two strings. So what happens if the input leveL is entered into the test? Take a look at the picture:
The test fails because it doesn't transform the strings into input.ToLower(). This indicates that there is a flaw in the conversion process of our implementation code, we should have changed the input to lowercase before comparing the strings. We should never do this in tests. Who must carry out this logic is the unit of the code under test. The image below makes it clear what we must never do!
Unit testing proved that the Act section needs to be revised, the code being tested does not correctly convert all entered inputs! So that's why we should go further in our tests and think clearly about what the Act execution block will do, we should always think:
Does passing this input make sense?
Would an input with an empty list make the unit of code under test return what I expected?
The incentive here is to think beyond the happy paths and not be satisfied with the few results generated.
The business rule, in this case, does not care if the letters are capitalized, for the user the word leveL and level have the same meaning, regardless of whether they are capitalized or not, the same goes for other words.
So this is why we ignore uppercase characters and when receiving the information the input is converted to a lowercase string. I'm going to leave the code refactored, and if you want to test it, you'll see that with the string leveL, now the test will pass!
I hope this example has helped you to visualize the importance of checking and thinking carefully about passing inputs to the Action block. In addition, it is always important to verify that the inputs of the code under test is correct. Remember never to put domain logic in tests. Unit tests need to know as little as possible of the code under test!
Another important point to note, the developer did not think about whether the logic could be incorrect because he did not plan other test cases with different inputs.
Remember:
Having tests with hasty inputs can lead developers to use values that don't match the actual use cases, which can lead to misleading results or test failures.
Now to finish this topic, I will leave a picture with a test where we consider various scenarios with different strings. Obviously, after the correction in the code, all tests should pass:
Again I reinforce questioning the code and the behavior as if you were a user. Think about how a user could enter a value and send it to your software.
What effect would that have?
Do I have validations for this input?
These questions can help in the process of planning and writing the tests!
📍 Harmful effects of testing with incorrect inputs
Incorrect inputs in unit tests can negatively affect the quality of tests and software in many ways, we can list:
False Negatives: Tests with inappropriate inputs can pass, even when there are problems with the code. This can result in undetected bugs that negatively affect the quality of the software and can cause issues when the software is released into production.
False Positives: Tests with incorrect inputs or exposed code implementation can fail, even when the code is working properly. This can lead to wasted time and effort trying to solve problems that don't exist, delaying software development and delivery.
Uncovered Test Cases: Unit tests with or inappropriate inputs may not cover all paths in the code, leaving parts of the code untested. This can result in undetected bugs that will only be discovered in later stages of development or in production.
Difficulty in code maintenance: Unit tests with incorrect inputs can make the code more difficult to maintain. When code changes, tests can start to fail because of their incorrect inputs, making it harder to identify whether the failures are caused by real issues in the code or just the bad tests.
Poor reliability: A unit test suite with incorrect inputs can undermine confidence in the test suite as a whole. This can lead to less confidence in code quality and increase the risk of undetected issues in production.
Let's quickly talk about the benefits!
📍 Benefits of testing with stable and reliable inputs
It becomes evident that when we elaborate tests with greater care and avoiding simple inserting inconsistent information, we obtain countless benefits for the development and maintenance of the software, we can list:
Safe refactoring and maintenance: With high-quality unit tests, developers can modify code and ensure that the changes do not break existing functionality. This simplifies code refactoring and maintenance over time.
Early identification of problems: Accurate and well-organized tests can detect bugs and problems in the code before they are taken to production, saving time and effort in solving problems later.
Makes incorporation of new functionality easier: A correct and consistent test suite ensures that new features integrate properly with existing code, minimizing compatibility difficulties and ensuring that features operate as intended.
Increased software reliability: Reliable and consistent unit testing increases software reliability, ensuring that requirements are met and that the software operates properly in different scenarios.
Reduces risk: Testing with valid and comprehensive inputs helps reduce the risk of software failures and security issues, protecting users and the company's reputation.
Avoids regressions: By thinking through the inputs for each test scenario, you create a comprehensive set of tests that can be used to verify that future changes to the code do not introduce new errors or break existing functionality.
Eases maintenance: A well-designed set of unit tests with different scenarios and inputs makes it easier to maintain the code. When developers need to make changes or fixes, tests provide a quick and reliable way to verify that the changes have not negatively affected other parts of the software.
📍After writing the tests ask yourself!
In the previous topic, we saw the benefits. I am leaving some tips for questions to be asked so that each of us thinks after writing a test, so we will be able to detect incorrect inputs in unit tests:
Does the test examine specific functionality or behavior? Unit tests should be focused and evaluate only one feature at a time.
Are the inputs used in the tests realistic and reflect the software usage scenarios? Keep input information that represents actual use cases, boundaries and possible variations.
Is the test logic understandable and easy to follow? Ensure that the test logic is simple, objective and properly documented, making it easy to understand and maintain in the future.
Do the tests verify the appropriate conditions? Confirm that the test is actually measuring expected behavior and not an irrelevant or minor condition.
Does the test account for failure scenarios or exceptions? Ensure that tests address cases where errors or exceptions may arise, ensuring that the software properly handles unforeseen situations.
Is the code coverage adequate? Use code coverage analysis tools to make sure your tests cover all relevant code paths.
Are the tests independent of each other? Each unit test should stand alone and not depend on the results of other tests, ensuring that they can be performed in any sequence.
Is there redundancy or repetition between tests? Make sure your tests aren't examining the same behavior multiple times, which can make your test suite less effective and more complex to maintain.
📍Conclusion
In conclusion, avoiding incorrect inputs is critical to ensuring software quality, reliability, and maintainability. By dedicating time and attention to test inputs and the creation, review unit tests, you are investing in the long-term success of your project. Remember to question and critically analyze all aspects of your tests!
I leave the incentive to share knowledge, experiences, and lessons learned so that we can lead to a more productive and efficient work environment, where everyone is committed to producing high-quality software.
I hope this post helped you, if you liked it, please share it with others! See you next post! 😉😄
Books that I consider essential for everyone ✅:
Effective Software Testing: A Developer's Guide - by Mauricio Aniche
Unit Testing Principles, Practices, and Patterns - by Vladimir Khorikov