Today's guest in my Built-in Quality interview series is Christoph Zuber, a business analyst, product owner and requirements engineer. Depending on the project, you can also find him in various testing roles.
(The following conversation has automatically been translated from German).
Pia:
Welcome, Christoph! I'm delighted that you've taken the time to chat to me today about software quality and built-in quality.
Please introduce yourself briefly.
Christoph:
I am Christoph Zuber and have been working as a Business Analyst at Zühlke for over 7 years. I have roles in which I bring together various stakeholders. These are often specialist representatives. I translate the specialist representatives' requirements so that software developers can build something suitable.I am often involved in activities that involve checking whether what has been built is what was requested. I had my first touchpoints with testing during such activities.
I think testing fits well with business analysis, because quality always depends on who uses the software in the end. What are the users' needs? What is the business behind it?As a business analyst, it's my job to understand these needs and formulate them accordingly. Afterwards, I can then check (test) whether what has been built matches what is required. Such a test is a type of requirements analysis. Simply at a different time and from a different perspective.
Before my time as a business analyst, I spent some time developing software on the mainframe. Even though this is a far cry from today's modern technologies, this experience helps me to understand what software developers need and how I can best formulate my requirements so that they are quickly understood.
Pia:
Thank you very much, Christoph.
We both got to know each other at the beginning of 2019 at Zühlke in the so-called Topic Team (= Community of Practice) Software Quality. We have also had the pleasure of working together on two specific projects in recent years.
Christoph:
These were two really exciting projects where quality was a major issue.
The first was a "fire drill" where we realised shortly before the release that we had no idea about the quality and urgently needed help. In the second project, we took care of software that had grown over a very long period of time. Here too, the quality throughout the entire process up to the delivered product was not convincing.
Pia:
Yes, exactly. Which brings me to the question: What does software quality mean to you? What is your definition of built-in quality?
Christoph:
For me, the most important thing with Built-in Quality is to avoid surprises. Software development is a complex process per se. Very creative, with many unknown variables involved.
Built-in quality is about making sure that what you've built is good enough so that you don't have to come back to something later when you think you've already done it.
Building a function is one thing, but the time you need to think yourself back into context increases the further you get away from the topic. That's why it's more efficient to finalise a function really cleanly than to throw something in and then clean it up later and fix mistakes.
For me, built-in quality means that when we build something, we don't just make sure that something is there, but that we are sure that it meets the requirements and has the necessary quality for our users and for our business. To do this, of course, we need to know what our business is, who the users are, what their requirements are and what quality means in this context.
On the one hand, quality certainly means whether the functional requirements are fulfilled. On the other hand, there are also non-functional requirements such as performance. These are very context-dependent and unfortunately often go unnoticed. Suddenly you are productive, with real users on the application and realise that the response time is unusable.
These are topics in which I am involved as a requirements engineer. How do I clarify such requirements in advance? And once we have defined the requirements, how do we test whether they have been fulfilled? All of this plays into Built-in Quality. Relevant topics must be considered from the outset. Testing quality later is not an option!
Pia:
Have you always seen it that way?
Christoph:
Certainly not. The first experiences at a bank were still very much a waterfall. There was a development phase. That's when development took place. Maybe there was talk of developers doing unit tests, but we didn't really understand it at the time. Then came the testing phase, in which someone else tested the solution. That was always difficult because our testers didn't/can't understand what we had built. I spent a lot of time explaining what we had built, which unfortunately also influenced the testers, causing them to lose their critical distance.
Today, we work in a much more agile way. Features and stories are finalised in every sprint. There is actually no code freeze and no subsequent test phase. I want quick feedback here. Testing takes place in a team. The tester provides a service to the team. As soon as someone has programmed something to the point where it can be tested, a tester can be called in for a four-eye check. This person provides critical feedback and brings a different perspective.
Pia:
In other words, do you also see "tester" as an important role in an agile team? Is it necessary to have a dedicated person in the team for this?
Christoph:
Yes, the difficult thing about testing is maintaining a critical distance.Developing a vision for new features is a different way of thinking than critically scrutinising how something works and developing (test) scenarios.It is difficult to think ahead with vision, sell solutions well, be critical and scrutinise yourself at the same time.
That's why, in my view, it's worth having someone explicitly responsible for critical thinking. This person doesn't have to test every feature or every story. But when it comes to the important points, someone like this brings in another perspective and scrutinises.
Pia:
Yes, absolutely. That often gets lost. We want to bring something cool onto the market as quickly as possible. We don't like the critical view of our cool "baby" and therefore often block it out as "negative".
Christoph:
There is also something negative about criticising.
That's the difficulty of the quality role. You quickly find yourself in a situation where you criticise your team colleagues. However, the goal is not to critically scrutinise the person, but the result and to recognise problems. Anyone who thinks professionally is happy to receive constructive feedback. Nobody is perfect! Of course, criticism should always be formulated in an appropriate (appreciative) way.
Scrum favours the mindset of closing story after story.In my current project, we unfortunately have the bad habitwanting to close all stories at the end of the sprintIt's better to close a story that hasn't been properly tested yet or, even worse, a story that doesn't have all the acceptance criteria built in so that the sprint can be completed. It's all about "we've planned it in this sprint, we have to complete it in this sprint". That's exactly not my understanding of built-in quality.For me, built-in qualitymeans that we only close the story when we know it works reliably and we have tested it. We are still working as a team to have the same understanding. We have to make corrections anyway if something doesn't work. But then we do that separately. When we finalise unfinished stories, we give ourselves the illusion of making rapid progress, but in fact we build up a backlog of bugs.
Pia:
As a product owner, how do you deal with your team not being able to deliver everything they have committed to? (Delivering everything would mean compromising on quality).
Christoph:
This is always a question of prioritisation. There is a backlog with a sequence. Sometimes, of course, there are external dependencies and hard deadlines.
In the current project, it would actually be easy. Nobody is waiting for us, but there is a deadline in 2 years. Until then, we are relatively free. From my point of view, we can afford to clean up the quality. However, we can't gold-plate everything. We have a lot of functionalities to implement. You have to weigh up whether something is good enough to get feedback. We don't want to annoy our users.
Pia:
What is your specific role in the current project?
Christoph:
On the one hand, I am a business analyst. I support the PO in planning the backlog and developing features and stories. I also support the PO in strategic issues.
On the other hand, I have a quality role. I coordinate the testing for our team. At the moment, I also do a lot of testing myself. However, we are in the process of setting up an automated system to simplify our testing. We want to be able to test an important aspect of the product, namely good data quality, as automatically as possible. We are currently building a test set for this purpose.
On the one hand, building a test set means that you cover all critical cases, so it has to be relatively broad. On the other hand, it should be maintainable and fast. So it must not be too big.
Pia:
Exciting and certainly not easy. You are in a dual role with these two different focuses.
Christoph:
Yes, it is sometimes really difficult to distribute the mental focus correctly.
Pia:
In my opinion, a good test set should be risk-based. However, in the past, I have mainly heard from product owners or business analysts that they want to have "everything" tested.
How do you see it? How do you find a middle ground, or do you also want to test everything ;-)
Christoph:
Of course, it would be nice if you could test everything. But the combinatorial possibilities go so quickly into infinity that it is absolutely impossible to test everything. I think the most important thing is to understand what realistic scenarios are. It depends on what can happen. What are the risks?
For me, the most important point of reference is whether we are testing public software or software that is intended for experts.
In the current project, we are building software for specialist users. We assume that they won't try to break the system. Here we can omitmany edge cases that we would have to test with public software But that's not a universal answer. The point is to really understand what the environment is that we are operating in and where the risks lie in the business domain.
Pia:
Accessibility is a very topical issue. In the not too distant future, there will be legal requirements for this. Have you already come into contact with these non-functional requirements?
Christoph:
We once tried to navigate an app using only the screen reader. That was an exciting experience. However, we never got so far as to optimise for accessibilityUltimately, I also don't know how much youcan actually test accessas anon-disabledpersonIf my software has a UI and I want to test whether it is usable for a blind person, I canclose my eyesbut I don't think like someone who can't see. The same applies to people with motor impairments. I can make assumptions, but what it actually means in detail...? There is still a lot of work ahead of us.
Pia:
I agree. I think it is particularly important to think about accessibility as early as possible. As early as requirements engineering and especially during development. It is essential to sensitise developers to how an application can be built accessibly. If you have to/want to make an app accessible afterwards, this usually involves a lot of effort and is very error-prone.
Christoph:
This makes it all the more important to keep it simple! The more complicated a UI is, the more difficult accessibility becomes. The most important principle of accessibility is therefore: leave out everything that is not necessary.
Pia:
We have to find the right balance. Our software should be accessible, look good and be intuitive for everyone to use. We are going through an exciting time with important changes.
Christoph, you have been and continue to be active in various industries. Do you see an industry that is more open to the topic of quality?
Christoph:
It depends on what good quality means ;-) For me, it always depends on the context.
In the financial sector, the greatest damage is either loss of reputation or monetary damage. Monetary damage can be repaired with more money. In the financial sector, the question of how much quality is needed is a purely financial one. Normally, no human lives are affected. There is no health risk. Therefore, in my experience, people are more inclined to take a certain amount of risk here.
But when it comes to health and human life, I hope that quality has a different status. I would rigorously ensure that the products are safe right from the start.
At the moment I'm somewhere in between, in an engineering department at a transport company. You realise that engineers are used to working very precisely. But they also know the real conditions on a construction site. A lot of things are solved pragmatically. So we are somewhere in the middle here. It's not like the financial sector, where you can regulate a lot of things financially. But it's also not as rigorous as in a regulated healthcare sector.
Pia:
I would like to talk about agile teams again.
Apart from the fact that it makes sense to have a person on the team to take a critical quality view, how do you think an agile team should be put together? How should the team members work together?
Unfortunately, I have often seen that a waterfall organisation has been squeezed into an agile team, which has only exacerbated the existing bottlenecks and stress many times over.
Christoph:
I think the most important thing is that you can work very closely together and have trust in each other.Both as a business analyst and in a testing role, I enjoy working with development. I like to look at the code and what thedeveloper has thought about.
I prefer to work with short feedback loops and do so very successfully. Ideally, I look at finished parts with the developers in the development environment. That way, we can make adjustments directly. We are also both in the same context at the same time. When something is finished, we have already looked at it with four eyes and don't have to test it waterfall-style.
Pia:
Yes, that's also my favourite way to work. Fortunately, many industries and projects are moving more and more in this direction.
Christoph:
Exactly. In view of this development, it is all the more important that all roles are represented in the team. That someone with a focus on quality is a fully-fledged team member. Requirements engineers are also part of the team. Depending on their skills, the product owner can be involved in requirements engineering. However, this requires that this person really has time to be close to the team. This can be a challenge. Especially if a lot of stakeholder management is required.
Pia:
I agree with you, Christoph. I think that we are very well positioned with cross-functional teams to ensure quality and simply work together in a relaxed manner.
Lately, I keep hearing "We only test in production" or "If something isn't good, the users will get in touch". Have you noticed that too? What do you think?
Christoph:
This again raises the question: What are the risks? What is the worst that can happen? For me, pre-production testing is an important safety net. You use this net to catch the big fish. You make sure that the worst problems don't make it to production. Of course, you can't cover everything with testing. Especially so-called "unknown unknowns". Testing is an important means of checking that such errors only happen once.
The key here is automation. A machine should check the most important functions in advance. Of course, someone must first consider what the most important functions or the worst problems are and how we can ensure that these do not occur. This requires a lot of expertise, which for the time being should be provided by real people.
Pia:
Have you ever been supported by Chat GPT?
Christoph:
To be honest, I haven't. I'm still a bit critical. I think the projects I'm involved in are very specialised. I don't know how much training data the AI already contains for this.
When it comes to producing combinations of test data, I can imagine that AI can be a good support. I think it remains exciting.
Pia:
Absolutely. I see great potential here and at the same time I am not afraid that we will be automated away, not even by artificial intelligence.
Christoph:
AI also brings completely new challenges. How do we ensure that artificial intelligence really fulfils the requirements? Playing with chat GPT is one thing, but basing entire business applications on it is something completely different. And I think testing this is a very exciting challenge.
Pia:
Exactly. We'll definitely stay tuned.
From your many years of experience, Christoph, what do you think are the most important success factors for Built-in Quality and for a high-quality software product?
Christoph:
The most important thing is to know the environment.
There is no such thing as THE product that is good in every environment. It depends on the context. You can only understand the context if you talk to users and gather feedback with prototypes and initial solutions. You then improve these iteratively.
In this sense, there is no such thing as THE quality; quality depends on the context.
That is absolutely crucial for me in order to build good products. And how do you build them?
This is only possible in a team where people trust each other, work closely together and can give each other quick feedback. You don't work against each other, but with each other.
Pia: You said it! Thank you very much, Christoph.
