Reasons why you as customer should test your new software intensively and at an early stage

The core of agile software development according to Scrum is regularly releasing product increments to the customer. The software’s functions are gradually extended, but the customer receives a version of their software right after the first sprint. That’s why at the end of each sprint, a review meeting is held where the development team presents the new functions to the customer. In this article, we will explain why it makes sense for the customer to participate in the reviews and why they should test the software created for them intensively and at an early stage.

Tests: How extensive are the tests?

Initially, the development team is responsible for testing, which is an integral part of agile software development. After all, the team wants to present software with as few errors as possible in the reviews, from the first sprint to the last. The scope of this internal team testing is agreed upon with the customer at the start. Ultimately, the decision which browser and systems a software product should be optimized for can have a large impact on the budget. The team members conduct a broad range of manual tests by running the software on different operating systems and browsers to test the newly developed functions. In addition to functional testing (“Was it completed according to the relevant acceptance criteria?”), it is important the user interface does not have errors.

However, these interface tests cannot always cover all possible browser combinations and versions or operating systems. This would lead to great additional costs and we would have to receive the customer’s consent. The team must perform the tests as specified by the customer when the user stories were processed. After all, the customer remains the expert for his specific domains and users.

If the customer does not want to invest in comprehensive testing, we look for a compromise that covers many but not all cases and configurations. The challenge is that the team may overlook problems and errors. What can also happen is the software fulfills the acceptance criteria but is still sub-optimal. And the later you fix errors, the more complex and costly the process is.

Errors detected late are expensive

The 1-10-100 principle applies in software development. According to this principle, the effort for fixing problems increases depending on when an error is discovered. Errors detected by the developers while programming can be fixed quickly and with less effort (1). If a tester detects an error in a test system, the effort is significantly greater. They must document the error, forward it to the developer and update the test system after the error is fixed and then repeat the test (factor 10). The worst possible case is when errors are detected in live systems. Fixing the bugs takes as much effort as in the test system. Furthermore, the live instances needs to be updated. The consequences are critical: bugs annoy users, functions meant to generate sales may need to be deactivated, news regarding the defective software spreads online faster than ever before and the customer’s reputation may be tarnished (factor 100).

That is the reason why customers should systematically troubleshoot as early as possible and conduct comprehensive tests. The customer needs to always be up-to-date on the development of their software and be active in the process.

Participating in the review is important

We have learnt it is a good idea for customers to participate in the reviews and product presentations with the team on-site. This personal interaction is by far the best way for the customer and development team to exchange information. One of the greatest advantages of developing software using the Scrum model is integrating the customer into the process.

We want to make this a reality in our projects by designing and developing products with our customers. For customers, development does not occur in a “black box” where they only see the result at the very end. The process is transparent and adaptable. The end result is customers get the software solution they expected and save money by preventing additional rework and modifications to the product.

And depending on the project, there may be other stakeholders involved whose opinions on the development status are relevant. Involving these stakeholders in the reviews can help prevent changes later on.

The customer must seriously test and accept functions

Participating in the review is not the only important thing because not every detail can be discussed and tested in these meetings.

The customer should also seriously test and approve the product after every sprint in order to save costs (unless comprehensive testing was explicitly agreed upon with the team). Additional and short-term interface tests of product increments help quickly identify any need for changes and avoid (expensive) changes in the future.

Conclusion: additional expenditures for the customer but worthwhile

Admittedly, regular visits to the development team initially cause additional expenditures for the customer. Testing does consume time and resources. Investing more time more often makes sense though. Errors can be detected and fixed more quickly. The customer is actively involved in the development and QA process and receives a product that truly meets his expectations. The additional expenditure is worthwhile. At //SEIBERT/MEDIA, our doors are open for all customers!

Your partner for individual software development with Scrum

Are you planning a software project? Do you want to expand an existing system or migrate a software platform? Do you require interfaces between applications within your company? And you insist on top-notch quality at all times?

Then //SEIBERT/MEDIA is the right partner for you. For us, expandability, performance, scalability, platform independence and testability are what really matter. We offer solutions that can be expanded and modified flexibly in the future. Feel free to contact us with no obligation! You can find more information in our public wiki: Introduction to software development with //SEIBERT/MEDIA.

Read this post in German.