Automated testing of a vertical solution on a Dynamics NAV basis

Intention

Our software is constantly subject to enhancement and further development. Currently, the manual generation of test data as well as the manual testing of implementation is a challenge for tegos group developers. Manual tests are time-consuming and costly. For every execution of the process the test scenario has to be regenerated. This complex process is prone to errors. Automating this process can save money and time as well as avoid errors. In order to adjust existing and new components during the development stage, we need to develop tests that are simple and executable by everyone. This ensures the software’s high quality. The goal is for developers to be able to verify the correct functionality of implementations with these automated tests. This process needs to be fully automated. The automation includes updating the source code, the generation of the executable program as well as its automated tests. This is called continuous integration. This process can be executed in a predefined cycle (e.g. daily) in order to test all changes of the implementations. The intentions are shorter and higher quality delivery periods to supply our customers with new versions of our software faster and easier. When a test fails, a developer has to deal with the problem immediately to guarantee a continuous delivering capability. Nevertheless, this incident should be an exception because the tests must be executed and adjusted when the software is changed.

The tests

There are different types of software tests. The unit test is executed on a very deep level of the source code. Here, an early test is essential, because unit tests are developed right after the creation of implementations. In contrast to that, “Test Driven Development” is a process where the tests are developed before the implementation. Unit tests should only test the smallest verifiable units in certain behaviours. When it comes to agile software development, the objective is the regular execution of unit tests, which provides a high standard of quality. Unit tests substantially contribute to code refactoring and securing permanent reusability, expandability, and maintainability of the software. Unit tests also help document the system by explicitly showing the test object’s intended uses and reactions.

All unit tests are defined prior to their development. Here, we determine which features have to be tested and which results are to be expected. Methods often reference additional components, which have to be defined as well. For example: A new contract module has to be developed. For this, use cases are defined. The contract deal between two traders is the use case. Based on that, a test scenario is established. This test examines wheather a contract with certain values can be generated.

We define a second use case: The amount calculation. The amount calculation is executed on basis of the given quantity and price in the contract and should return a specific result. This means, that the amount calculation is based on the contract. For the implementation, we take the methods from the first test of contract generation. This is how the test framework is being further developed bottom-up. Eventually, the test asserts if the actual amount matches the expected one. If this not the case, the test is marked as failed and the developer must begin troubleshooting. The error can be found in the implementation as well as in the unit test itself.

According to its universal definition, integration tests combine several unit tests in order to test the interaction between different components. We expand the example of the contract and the amount calculation to explain this definition. We develop a new unit test for the use case “weighing”. It is tested if a weight in our weighbridge module can be declared and be returned correctly. The quantity in the contract, which is necessary for calculating the amount, should be provided by the weighbridge module. The three unit tests contract generation, amount calculation and weighing are treated as black-boxes. The three unit tests are merged and their integrity is verfied.

The line between unit tests and integration tests is blurred. At tegos, we  do not care about a strict separation of these two terms. Unfortunately, automatic unit tests and integration tests cannot entirely replace manual testing.

Microsoft already provides their “Test Tool Kit” for NAV, which is used for testing the NAV standard product. In result, we build and execute our units tests for our vertical solutions based on Microsoft’s “Test Tool Kit”.

Conclusion

Providing continuous integration, documentation, use cases and tests results in high-quality software standards for our customers.

Continuous integration and testing help guarantee the software quality during continuous deployment. The specific types of tests are not decisive for the quality, yet essential. Automatic tests do not void user-acceptance-tests but simply signify fewer required adjustments because the features were already tested in advance. A good test coverage is required to achieve a good software quality. A complete coverage of the source code does not make sense, because it is nearly impossible to execute for extensive methods. Rather, we should focus on the most important components like the amount calculation in our second use case.

The tests should be defined based on use cases. What has to be tested? Which components are necessary? And which results are expected? We can choose between the unit test and the integration test as we see fit. When the software is changed, the tests should be created or adjusted. In the long term, automated testing saves money and is more reliable than manual testing since it is developed only once. These tests can be executed arbitrarily without additional costs.