I'm a big fan of automation and any tools that make my life easier. At the same time, I like working with people. Is that a contradiction?
I love my home office and need to be able to work independently and autonomously. I also need to exchange ideas and work together in a team.
Not because I can't do it alone, but because I know that we can do it much better as a team.
My ideal team is diverse and has all the skills needed to create and support a high-quality product. It goes without saying that different disciplines and different roles are represented in this team.
Recently, however, I have been increasingly confronted with a familiar trend:
Automation (in testing) first!
I hear and read sentences like:
"We need (more) automation to improve our quality."
"We are looking for test automation engineers, automation engineers, engineers in test automation, [you name it]."
"We have no use for you if you don't want to or can't automate tests."
I seriously thought we had overcome this tiresome either-or-topic. Obviously not.
The automation hype is like a boomerang. It keeps coming back. Like that annoying relative who finds you in the last corner at every family party and chats you up until closing time. Like the mosquitoes in summer that find and maltreat me even when I'm bathed in the strongest insect repellent.
I don't know about you, but I've had enough! My ears are burning from the same old statements.
For this reason, I would like to dispel my top 5 myths about automation based on personal experience.
What Automation CANNOT do for you
1) Compensate for missing strategy
An example: You have this fancy one-for-all tool in-house that someone has bought for a lot of money and which is causing you ongoing maintenance and licence costs.
To ensure at least a small return on investment (ROI), you use the tool at the end-to-end (E2E) test level. After all, that's what the manufacturer advertises. Everything is tested there, end-to-end, as the name suggests. So far, so wrong. The top test level, a.k.a. E2E, is often the wrong one.
You need to be clear about which scenarios you want to test at which test level and in which form, automatically, semi-automatically or manually, and above all WHY.
What are your critical paths, what are your business processes, where in the code is your business logic embedded, etc.?
The vast majority of scenarios can be tested very early on, from the unit test stage.
Without a smart (test) automation strategy, you run the risk of wasting time, money and human resources unnecessarily.
No tool is to blame if you don't do your homework! We all know the dog didn't eat it.
2) Correct wrong tests
If your tests don't make sense because they don't test your use case, automation is neither the problem nor the solution.
If your tests are testing the wrong use case because you didn't understand the real one, maybe you should stop communicating asynchronously.
If your tests are wrong because critical paths are not being tested, you should expose your risks and make sure everyone knows them.
If your tests are flaky because your team lacks expertise, you should bring that expertise into the team immediately. And no, you don't always need to hire a dedicated person for this. How about targeted upskilling in the form of coaching or training?
The smartest automation tool alone cannot help you to fix incorrect tests.
Garbarge in, garbage out!
Note: Yes, AI (artificial intelligence) tools can help you with this. But beware! AI is still a machine. This machine doesn't know "I don't know". It will always present you with a statistically probable solution. Unless you have sufficiently fine-tuned your prompt. And even then, please do me a favour and don't blindly trust the AI!
Whenever you use AI or LLMs (Large Language Models), subject the results to a critical, human review.
3) Take ownership of quality
If you don't have a common understanding of quality, you should talk about it.
If your team is not pulling in the same direction to integrate good quality from the outset, you should uncover the reasons for this.
If team members see testing in the remit of others, you should agree on the understanding of responsibilities.
If not everyone takes responsibility for quality, you should ask yourself what your error culture is like.
If testing is a downstream phase rather than an integral part of development, you should rethink your process.
The best automation tool cannot take responsibility for you. Quality is a team responsibility. Change my mind!
4) Replace communication & collaboration
Whether New Work or long-established organisational models, why is there so little and narrow-minded communication? Why is there so little discussion and even less questioning?
Many people think that if they write everything down, define every little detail (which is not possible) or give the team room for manoeuvre (a.k.a. with luck the user story will have a title), everything necessary has been said. It is not!
Every requirement is an invitation to a conversation and can be the basis for cross-functional collaboration.
If your team wants to be agile but still thinks and works in silos, automation will only increase the associated challenges.
You will not be able to avoid actively approaching each other in an interdisciplinary way. Yes, this requires the openness and willingness of each and every individual.
Unless, of course, you favour a culture in which finger-pointing, micromanagement and high staff turnover are the order of the day.
5) Eliminate resistance
Humans are creatures of habit. We don't like change. The pain has to be immense for us to change something. Now imagine a salesperson knocks on your door unsolicited with "the thing" that changes everything. It turns your world upside down, potentially costs you your reason for existence, but hey, it makes everything much better. How do you react?
Most of us react sceptically at best, but most likely rather dismissively. Why should it be any different when it comes to changing working environments or introducing new tools?
If you want to use automation in your organisation, don't forget to take your most valuable asset, your employees, with you. Talk to them openly and take an honest interest in their thoughts and fears. Only together can you get the best out of both worlds, human and machine.
With this in mind, let's see what the automation machine can do when you feed it with the right fuel.
What Automation CAN do for you
1) Faster feedback & automatic quality gates
Smart automation, e.g. in CI/CD pipelines, shortens your feedback cycles to a minimum. You know promptly where your quality stands instead of waiting weeks for test results and bug reports.
Many deployment tools already have fully automated quality checks for code quality, security/vulnerability, accessibility, etc., which you can use directly or customise to your needs.
Many test tools that used to be operated separately from development and deployment can now be seamlessly integrated into the DevOps process. This creates transparency across all test stages.
In addition, you can now start up test environments ad-hoc in virtual containers at the push of a button, run tests automatically and, in the event of an error, have corresponding bugs created directly in your tracking tool.
Manual reporting could thus become a thing of the past. I can hardly wait ;-)
2) Free up resources
Automating the right tests frees up resources in the medium to long term. Your testers no longer have to agonise over clicking through manual tests, but can concentrate on more stimulating tasks, such as exploratory testing. They can also support the team with their special focus on quality by asking critical questions, writing requirements and reviewing them, pair programming, as a quality rubber duck or QA coach. In this way, the entire team and product quality benefit from automation in testing.
Note: Test automation does not necessarily have to be implemented by testers. They specialise in identifying the right tests and prescribing them at the appropriate test level. If they also enjoy coding, excellent.
Personally, I'm happy when software developers take over the implementation as coding professionals. They are simply better at it than I am. It would be tragic if it were the other way round. Low-code and no-code tools aside.
Who defines and who implements is of secondary importance. In the end, there has to be a mix of both disciplines.
3) Reduce sources of error
I have spent countless Sundays in the office on a weekend release, staring in bewilderment at the errors in production. How could this happen? We tested everything rigorously.
9 times out of 10, the error was not due to testing, but to the infrastructure operator's handwritten checklist: "Colleague X overlooked step Y in the deployment log last night. Oops! Sooo, sorry..."
Do yourself a favour and use automatic deployments including the right quality gates and save yourself releases like this.
At that time, I knew all the test cases for my application by heart. I didn't have to click through the manual steps one by one. That's what everyone in my team thought. And then it happened. It was the 1 release out of 10 in which we had overlooked something in testing. Running the same test cases manually hundreds of times had made us tired. A completely normal human error and at the same time avoidable, because it wouldn't have happened to a machine.
4) Parallel test execution
It has never been easier to simulate devices and control browsers, operating systems or real devices.
Countless tools support you in setting up your tests once and then executing them simultaneously in different constellations.
In my opinion, we should do everything we can to utilise the possibilities in this area. They save us valuable time and enable us to utilise existing resources more intelligently.
5) Automation & AI
Automation has become an integral part of our everyday lives and so will AI. The skilful combination of both can enrich and facilitate your daily business enormously. I recommend everyone to familiarise themselves with the basics of AI, LLMs and good prompting. Small things make a big difference here. If you have prompting under control, AI or LLM can support you very well with strategic and operational tasks.
Be it brainstorming on test design or strategy, creating automation scripts or formulating clear acceptance criteria. The possibilities are almost endless, just like the tools that sprout up like mushrooms every day.
My tip: try out different tools and don't be overwhelmed by the endless choice. Your favourite LLM is sure to give you a few good ideas for tools that might suit your use case ;-)
So much for my personal top 5 myths and facts about automation.
As you can see, it's all in the mix.
So let's stop with either-or and focus on the strengths of people and machines in combination.