The success factors described are the result of personal experience. This article does not claim to be complete. It is intended to provide suggestions and inspiration for daily work.
Some time ago, a customer asked me what I thought about Built-in Quality.
There is a lot of talk about it in many organisations, but there is a lack of concrete recommendations for action on how quality can be built into a software product and ensured in the long term. More buzzword bingo than focus on actual implementation - that's annoying!
So what does Built-in Quality mean? What are the critical success factors for Built-in Quality?
Before we can devote ourselves to answering these questions, we need to clarify what quality means in the first place. Quality means something different depending on whether software, hardware or something in between is being developed. Quality requirements often depend on the industry. It makes a difference whether the product is an online game, a medical laboratory device or a securities trading platform.
This article focuses on software, although some statements are also applicable to other products.
Quality
When defining quality, I like to refer to the ISO/IEC 25010:2011 standard. Actually, I am not a friend of strict standards, but this one has served me well on several occasions. ISO/IEC 25010:2011 supports us with 8 concrete characteristics in the definition of quality attributes and the associated requirements for a software product:
Functionality
Efficiency / Performance
Security
Compatibility
Reliability
Usability
Maintainability
Portability
Unfortunately, it is often forgotten during requirements elicitation how requirements fulfilment, especially of attributes 2 to 8, is to be measured - in other words: non-functional testing. Although these tests are crucial for a high quality product.
It is equally important that every team member knows the quality requirements.
Suppose we have defined our quality requirements. Why should we now bother to build in quality? Isn't it enough to test at the end, find the problems and fix them?
Built-in Quality
Do you know your "why"? Why is quality and Built-in Quality important to you?
I would like to encourage you to ask "why". Ask yourself, ask your team, ask your organisation. Because in the end it is always about the "why". No one buys "what" you do, but "why" you do something.
Make sure you know your "why", especially in terms of quality.
As an example, I would like to share my personal "why", which is as follows:
I want to make life easier - for our users, customers, my team, for everyone who comes into contact with our product.
I believe that this can be achieved by providing sustainable and high quality software products. To develop sustainable products, Built-in Quality is non-negotiable.
But what exactly is meant by Built-in Quality?
SAFe says the following about it:
"Built-In Quality practices ensure that each solution element, at every sprint, meets appropriate quality standards throughout the development.
All teams — software, operations, product marketing, legal, security, compliance, etc. — share the goals and principles of built-in quality.
Practices vary by discipline but all support Agile values."(1)
So far, so good. This all sounds great and I share these views. However, the above statements are too abstract, too vague for me to work with in daily business.
That's why I have compiled the success factors from my project experience, condensed them and come up with the following picture:
The cornerstones are clearly ownership, shift-left, a good test pyramid and a continuous quality mindset.
Various automation tools and agile frameworks can have a supporting effect.
However, the foundation for all of this is always the people and the organisation. It needs a supportive organisation, culture and motivated people. Without a stable foundation, you can have the coolest fully automated shift-left set-up, but it will not solve the basic problem.
Ownership & Responsibility
Ownership means that the whole team is responsible for quality, quality assurance and testing.
A good starting point for establishing ownership is the creation of common principles, so-called "quality principles". The team agrees on the most important principles in the area of quality.
The Testing Manifesto (2), for example, provides inspiration for what such principles might look like.
No matter whether you define your own principles for your team or adopt the Testing Manifesto, it is important that all team members agree and commit to the principles.
Once you have agreed as a team, make your principles visible. Make sure they are not forgotten. Print them out, put up a big poster in the office, publish them on your Confluence homepage etc. You don't want to hear excuses why someone doesn't know your quality principles.
Responsibility
In the past, it was often the case that one person was responsible for the quality of the entire product. That was never fair and we really don't want that anymore today.
Instead, we want the whole team, all disciplines and roles, to take responsibility for quality and decide together whether they are ready to ship and what risks the team is consciously prepared to take.
To be able to fulfil this responsibility, the team should have professional QA/testing skills.
This can be achieved through testing/QA specialists in the team or through dedicated coaches. Which way is chosen is not relevant. What is important is that the team has all the necessary skills, including testing & QA, to develop the best product and avoid bugs. Every team member should wear the tester hat and ask critical questions.
Shift-Left
Shift-Left in the process. Shift thinking about quality, quality assurance and testing further to the left. Quality must be the focus from the beginning and throughout the lifecycle.
What must be avoided at all costs is to see testing as a phase within a sprint. This results in a mini waterfall and a single person is under even more stress than in a traditional waterfall setup. The likelihood of people burning out and bottlenecks in the process is very high.
Testing is an activity and always part of development. So it cannot be that something is considered "done" without having been tested. That is simply not correct.
In order to make testing an activity and to place it further to the left in the process, Test Driven Development (TDD) or Behaviour Driven Development (BDD) can be used.
Both approaches have the advantage that tests are defined before code is written. That means you know what you want to test, then the code is written and if something doesn't work as expected, you immediately have a failed test case. So you've found the bug super early and should usually be able to fix it cheaply at this point.
If you choose BDD, you have another advantage: BDD gives you a common language, Gherkin. Gherkin helps you to build a shared understanding. The simple syntax (Given-When-Then) is extremely helpful in involving everyone in the team. It saves time, motivates and fosters team spirit.
Before writing tests, we usually think about requirements. A good way to approach requirements is to use the "3 Amigos".
The "3 Amigos" refers to the three perspectives of business, development and QA/testing. It is about the different perspectives and not about three people per se. It is important to know the different perspectives and to take them into account from the beginning. The collaborative approach of the "3 Amigos" can significantly increase team performance, helps to ensure that no perspective is forgotten, promotes mutual understanding and reduces the room for misunderstandings and bugs to a minimum.
...and it's also really fun - so give it a try!
Test pyramid
A test pyramid can seem like a maze. I have often experienced that it was not clear what was meant by a certain test level and what is or is not tested on this or that level and to what extent.
Reason enough for me to design my own version of the test pyramid. One can discuss the shape, the levels and the degree of automation of the test pyramid and there is certainly more than one valid version. The important thing is that everyone in the team has the same understanding of the levels and content of the test pyramid. Everyone must have the same perception of, for example, integration testing or system testing.
Transparency is equally important. You want to know at all times what is being tested at what level and with what coverage. This knowledge drives efficiency and speed.
To illustrate, here is an example from one of my projects:
It was the old classic problem with big quarterly releases and a cumbersome process organised in silos. Each release required many manual tests and it felt like they took ages. This was tedious, stressful and frustrating. The feedback cycle was unacceptably slow.
At this point I was an embedded tester in an agile team and really tired of working like this.
I approached my product owner and said, "I want to reduce this large (manual) testing effort before each release as much as possible. To do that, I need to know our most important use cases from the user's perspective."
So we sat down and defined the most important use cases high-level together.
Far too often, the most important use cases, the actual purpose of a software, are neglected and the focus is only on user stories, new features or epics. The result is many ("one-time") test cases that are far from well-organised regression sets.
With the most important use cases in hand, I sat down with my fellow developers and went through each case step by step. We dived into the code, checked what was already covered by automated tests during development and documented our findings accordingly. We continuously added what was not yet covered. Not only did we create transparency, but at the same time we increased and improved our testing and test coverage.
Yes, it took time, but it was worth it.
Soon I had full transparency of our test coverage starting from unittest level. I reviewed my existing manual tests and was ultimately able to eliminate 60 per cent of them. Pretty amazing, isn't it?
This clean-up freed up a lot of resources and I finally had time to deal with more exciting things than clicking through manual test cases, e.g. real exploratory testing.
My team was excited and proud of our joint effort to increase our test coverage, our efficiency and the quality of our product.
You see, transparency frees up resources and if you pay attention to transparency from the beginning, existing resources can be used more wisely.
Quality Mindset
Every team member, independent of role or discipline, should be concerned about testability and quality. During the design or architecture phases, quality attributes and their testability must be kept in mind.
During development, clean code is the top priority. Make sure your colleagues know and follow the Clean Code Standards.
There are a few general rules that are very easy to remember. These are supplemented by specific rules for particular areas such as understandability, naming of functions, structuring of source code, etc.
Here is a good summary that you can share with your team so that everyone, not just developers, can look up the basics: https://gist.github.com/wojteklu/73c6914cc446146b8b533c0988cf8d29.
I also recommend the standard work on Clean Code by Robert C. Martin (Clean Code, A Handbook of Agile Software Craftsmanship). In my opinion, it's a must-read for everyone who writes code.
Last but not least, get specialists on board. Get expertise, everything you need to develop the best product.
Whether you already have the expertise in your project team or not, involve the people you need. The earlier the better. You don't want to find out about mandatory security requirements just before release.
Early and continuous involvement of all relevant disciplines is the key to continuous value delivery.
Conclusion
Built-in Quality has many aspects. My recommendation to you is:
Know your "why". Know why Built-in Quality is important for you and get others excited about it.
Establish a quality mindset across all roles and disciplines along the product lifecycle. This is a key factor for continuous value delivery.
Have a transparent test pyramid so that you always know what is being tested at which test level. In this way, resources can be used in an optimal way.
Shift-Left. Shift the awareness for quality, quality assurance and testing further to the left, to the beginning of the process. This enables fast feedback in short cycles.
And last but not least: Built-in Quality is a team effort. It needs the whole team. It needs each individual with their own personal skills and experience. Diversity in the team allows us to build quality in from the beginning, develop the best product and make life a little bit easier.
Sources:
2 Growing Agile, https://www.growingagile.co.za/
3 HBO Pictures, https://www.amazon.com/Three-Amigos-Steve-Martin/dp/B00287Z18Y
4 Zühlke Group, https://www.zuehlke.com/