Pia Wiedermayer & Romano Roth
In many organizations and projects, software development involves numerous employees and machines performing tasks separately. This approach results in problems. Here's how going back to the original way of developing software and building an organic digital factory can help.
The following blogpost was written during my time at Zuehlke and first published on their website in Jan. 2023.
In today's software development oodles of resources are consumed, and manual steps are carried out. The process phases run consecutively, completely separate from each other. After completing a phase (whether successful or not), the work results are tossed over the so-called ‘wall of confusion’. True to the motto of ‘devil-may-care’. This procedure results in the following main problems, which can be encountered in the same ways in a wide variety of organisations:
Many manual steps, which make the overall process slow and susceptible to errors.
Quality assurance and testing as separate phases at the end of the overall process.
Security requirements and other non-functional requirements, such as load and performance or maintainability, are only discussed and checked just before final delivery.
Lack of collaboration due to separate working areas and phases
To make matters worse, there are transformation projects that do not take the organisation as a whole into consideration, but merely transfer new names to existing structures. We often come across departments that are now called Dev-[your department could appear here]-Ops, but operate in exactly the same way behind the new Agile façade and function just like they always have. The only difference is that everything has to be carried out in a faster and more agile way. This puts the employees and the entire organisation under tremendous pressure. There is a serious risk of wasting valuable resources and overwhelming employees. In times in which sustainability and the ‘war for talent’ are becoming increasingly important, organisations can no longer risk losing their best minds and carelessly damaging the environment.
We see great potential in a return to the original type of software development, which naturally includes interdisciplinary teams and close collaboration. Enriched with modern approaches such as DevOps, integrative quality assurance and Agile working methods, classic software development can solve the above-mentioned problems in an efficient and sustainable way. But what exactly do these terms mean?
DevOps
DevOps is a mindset, a culture and a series of technical practices. It provides communication, integration, automation and close collaboration between all persons who are required for the planning, development, testing, provision, approval and maintenance of a product.
Both a cultural change and a change of mindset are of central importance when doing this. It’s no longer a case of ‘them’, but ‘us’. DevOps is based on teamwork. Mutual trust, empowerment, responsibility, continuous improvement, data-driven decision-making and customer empathy are central DevOps values.
The goal of DevOps is to speed up the market launch, make experimentation possible, release software at shorter intervals, reduce the lead time for bug fixing and improve the average restoration time. When doing this, it is not only about collaboration between Dev (development) and Ops (operations); it is about collaboration between all persons who are involved in the production of the product.
As shown in the illustration, these are development, security, business, architecture, compliance, quality assurance and operation. However, we are certain that we have forgotten a few groups of people.
You have probably already heard of the following new terms:
DevSecOps: Dev = development, Sec = security, Ops = operations
BizDevOps: Biz = business, Dev = development, Ops = operations
These new terms are an attempt to complete the term DevOps with other groups of persons. Unfortunately, these new terms are also incomplete. This because a term such as DevSecBizArchCompQAOps or Dev*Ops or DevXOps would be needed to do justice to what we want to achieve with DevOps. For this reason, we will simply stick with DevOps, even though it is not the most ideal term. DevOps is all about bringing all persons, processes and technologies together for the purpose of continuous value creation.
Quality Assurance
Quality assurance is often used as a synonym for testing. Since this is not only incorrect but introduces a considerable amount of confusion into the everyday working environment, we will explain the main characteristics and differences between the two terms in the following.
Quality assurance is clearly process-oriented, whereas testing mainly focusses on the product. Testing is a part of quality assurance. Quality assurance, on the other hand, is a part of a larger quality control system, and is primarily about avoiding errors.
This ensures that the processes for managing and creating the results are functional. Well-tried (Agile) methods, such as reviews or retrospectives, are used to check whether this is the case. With quality assurance, it is also about technical processes that ensure that quality is achieved in an effective and efficient way (built-in quality). There is major synergy potential with DevOps at this point.
Testing is an essential part of quality assurance. The main focus when doing this is on detection, i.e. finding bugs. Testing includes tasks for planning and control, analysis and design, implementation and execution, evaluation of termination criteria and reporting, as well as test completion activities. Testing also includes functional and technical reviews, and code inspections.
Testing in Agile development requires a high degree of automation. However, the following applies here: quality over quantity! The fact that tests can be automated doesn’t say anything about whether they are meaningful. Each test, be it manual or automated, involves effort. The focus must therefore clearly be on having tests that add value.
Unfortunately, the myth that test automation must replace manual testing is persistent. However, our experience has clearly shown that test automation is not a substitute for exploratory (manual) testing or specialists with comprehensive expertise. However, this role is clearly in a state of flux. Test automation tools and other development procedures help the entire team to work faster and more efficiently. This allows the individual team members to apply themselves to tasks for which the human brain is required.
Agile and interdisciplinary collaboration
Agile work models change the nature of our collaboration. It is very dependent upon the organisation, its culture and its people as to whether this is positive or negative. Diversity is required in the team in order to efficiently experience and implement Agile practices. An entire football team consisting of strikers or defenders would only be moderately successful. It is the combination of different specialities that makes the difference and is the key to success.
With regard to product development, this means: away from the silos towards interdisciplinary teams that combine all of the disciplines and roles that are relevant for the product under a common team umbrella. Each discipline is equivalent, each team member is important and each role creates added value.
As simple as it sounds, it can be challenging to implement this in practice. An excellent starting point for interdisciplinary teams is the definition of common quality principles. Here, it is vital for the principles of the entire team to be jointly defined, understood and actively experienced and called for by every member of the team. These shared quality principles must not be equated with the ‘definition of done’ or ‘definition of ready’, which we know about from Scrum. The principles are more about the ‘definitions’, and form the shared values of the team.
The following well-tried approaches can provide a good jump-start and support in everyday interdisciplinary collaboration:
The 3 Amigos
Whole-team testing
The 3 Amigos
Good requirements require different perspectives. Business, development and testing/QA therefore work closely together during the entire product development cycle. All three disciplines must be involved at all times, from the requirements analysis to the cooperative definition of specific requirements (e.g. user stories) to development and final acceptance. This creates a common understanding, and minimises the likelihood of misunderstandings and errors.
Attention must be paid to potential stumbling blocks:
restriction of the 3 Amigo discussions to just three people. If other people are involved who are relevant for a particular work step, they must be included in the discussion.
expansion of the 3 Amigo discussion to the entire team. The goal of this practice is to involve every perspective that is needed in a group that is as small as possible.
3 Amigo discussions become regular meetings, and are dealt with as an additional team ceremony rather than being a practical road map for the perspectives that should be involved in a discussion about a particular work increment.
Whole-team testing
The essence of whole-team testing is shared team responsibility for quality assurance and testing. This means that testing is no longer exclusively the task and the focus of professional testers, but of each individual person within the team. The testing role changes to a role that is oriented towards support and coaching. The team therefore has what’s known as a quality coach, who brings professional testing and QA expertise into the team. Of course, this person can provide active support with the testing, but this is no longer the main focus.
Another important aspect of whole-team testing is the agreement on common principles. These can be derived from the Testing Manifesto, for example, or be individually defined. Shared team understanding and commitment to the principles are decisive for success.
Irrespective of the chosen approach, the tools that are used or the degree of automation: success comes with modifying the mindset of each team member and actively adapting personal ways of working. A mini waterfall in an Agile sprint is worse than any cumbersome project in accordance with the V-model. This generates excessive stress, frustration and – in the worst case – leads to burnout, reduced product quality and higher costs (for bug fixing, etc.). A lose-lose situation is inevitable.
What does the future look like?
In future, companies must supply the right products or features at the right time and in high quality. In other words, they must identify the ideas with the highest economic value for their customers from all of the good ideas, and supply them with the correct quality, as fast and as cost-effectively as possible.
This means that companies must professionalise their digital product development and standardise it to a certain degree. The future therefore lies in the industrialisation of digital product development, and the set-up of an organic digital factory within the company.
Everyone in a value stream works together on a product in the digital factory. This one team takes over the complete end-to-end (E2E) responsibility for the product when doing this, and therefore has full responsibility for the vision, mission, backlog, budget, quality and operation. An organic digital factory such as this is not static. The product’s development platform, i.e. the conveyor belt, is developed and operated by a central engineering team. This team coaches and supports the product teams so that they can take over the responsibility for the conveyor belt themselves, and no dependency arises.
The processes, tools and methods in a digital factory are standardised. All processes are automated wherever possible, and built-in quality is the norm. This standardises digital product development. Quality, time-to-market and customer satisfaction are increased over the long term. However, with all of this standardisation, we shouldn’t forget the human factor. Like any other concept, the digital factory will not be very successful without people and organisations that are open and motivated to develop the best product together as one team.
Irrespective of the degree of automation and standardisation: companies must establish a supporting and appreciative collaboration culture, and actively cultivate it beyond all of the hierarchical levels. Employees must feel personally appreciated and accepted; this is the only way to achieve top performance and sustainable product development.