quality

leadership, Management, quality, Testing

Building a World Class Test Team

I’m starting 2022 in a more senior role, and it has been an aspiration for the last few years to build myself up to this point. I’ve learned so much since I left my first role in management in 2018 and moved towards more senior leadership roles. One of the big things I believe I have learned along the way, through both positive and negative experiences, is the value in building the best team possible and understanding both “what” that means and “how” you get there.

What does a World Class Test Team Look Like?

Obviously, context applies here, but in my experience there are some ways which have helped identify at a high level where we are on our journey as a team to become “World Class”.

A Good Test team could be classed as a cohesive unit which ensures products are released to a high level of quality. They may or may not be embedded within their delivery team, but they are trusted that the testing they do will discover the majority of issues before release. Automation may be in it’s infancy, but what they do have in place provides value.

A Great Test team would be the above and also may help prevent defects being added into the code as well as finding them during any test execution activities. They may also be striving to improve the quality practices within the teams they work closely with. They will have good amounts of automation to compliment their manual testing and are working towards the right balance

A World Class Test team would be all the above. They would also have a positive reputation across the entire organisation for being advocates for quality, thinking about the users and driving the business towards their aim of the best products possible. They would be a diverse and inclusive team who are embedded across the business and not siloed in one area. A constant and trusted part of all product discussions at all levels but not seen as a single sign-off for releases. They have worked with teams to ensure quality is part of everyone’s thinking, but work to continue coaching best practices to improve the confidence and view of quality. Automation is a big part of their testing, but they have a balance to ensure they are automating the right types of tests and compliment this with other forms of investigative testing. Most importantly, the team have a culture of wanting to continuously improve both themselves and the products

It’s obviously not that black and white, but knowing that you can improve the work you do should be seen as a positive to help move the team forward. Of course, the journey never ends, there are always ways to improve.

How Do You Build A World Class Test Team?

As a leader of a team (whether formally or as someone who is pro-active in wanting to drive the team forward), there are some steps you can take to help assess where on the journey you may be.

How outward looking are the team? – I’ve worked in teams in the past who have been very strong “9-5 Testers”, which there is nothing wrong with, but they tend to think the way they test currently is the only way to do it. By understanding where your team are, you know what you are preparing yourself for.
Assess the teams capabilities – knowing whether they are involved in looking at what is going on in the wider test community can help you start to understand their capabilities. Look into the types of testing and activities needed to test to the level needed to provide high quality products and find ways to assess where individuals are in their abilities with these activities. If there are gaps, look at ways to upskill the team members either through external courses, or online material and giving them time to learn.
Break the mould with new hires – You may have a good team of testers already, but are there gaps in their knowledge, are there areas of testing which they lack the skills to perform? These would be good places to start. But for me, the bigger area to focus is on bringing in resources who may evolve the way the team works, ones who may be active in the wider external testing community and hence are able to bring new trends and technologies to the table and encourage learning and sharing within the team. This happened for my team in 2021 and the way we work as a result of these hire has transformed in an incredibly positive way.
Engage them and show them that Testing can be fun – Try and move away from the “9-5 testing” view where you can, of course testing is an activity which needs doing, but it can be more interesting than just ticking a box to say a test has passed. Encourage the team to think outside the box. One way to do this would be to start doing some internal community events, whether it be an external speaker or even a group testing activity like mob testing or splitting into pairs and exploratory testing your product, working with someone you may or may not have worked with before. Doing these kind of activities will ignite an interest that some of the team haven’t had for a while.
Empower them to share outside the team – After getting the team engaged more, the next step is to create platforms for them to share the successes they have with the org outside of the test team. This could be via all hands, team meetings or even just sharing brief accounts of successes over email or slack. Raising awareness of the good work being done by the team will give confidence to the team and empower them to do more. This will help to show the value of good testing to teams across the business.

This is not supposed to be seen as a definitive list of actions to make your team world class, but this helped me get to a place where I had full confidence in what my teams could deliver, how they delivered it and what feedback we got as a team from the wider business. Building a positive reputation for testing/Quality has and always will be one of my goals for my teams.

Do I speak sense, or do you completely disagree? I’d love to hear from you!

leadership, quality, Testing

Defining your Story – Owning Your Quality Narrative

Following up from my previous post, I thought it would be a good idea to dive deeper into the idea of the Quality Narrative. As mentioned in that blog, I first came across the term from the ‘Leading Quality’ book by Ronald Cummings-John (@ronaldcj) and Owais Peer (@owaispeer). The book is a great reference for helping to drive quality in an organisation and it has truly become my go-to reference book to understand how to move forward when you hit those roadblocks to improving processes around testing and quality. Here is Ron and Owais’ definition of a Quality Narrative:

A quality narrative is the way people think and talk about quality in a company.

Sounds simple enough, but when you think about it, there are so many parts to decompose. Let me try and do that now and show you how I use this…

What does quality mean for you and the immediate test/delivery teams? Is this the same view of quality that the wider business has. Quality is very context specific, in some, it could be measured by the number of defects outstanding against a release, each one having an impact on overall Quality, others it could be more about the user’s satisfaction of the system. Here is the definition I drive to and share with my teams:

The measure of whether the software meets the explicit and implicit needs of the customer and their ability to use it successfully
Simon Prior 2020

Having a definition for quality which everyone can agree on and work with, can really help trigger the understanding of where and how Quality fits within an organisation.

People

People would effectively be anyone who you may be trying to pursuade on the importance of quality.

This might seem obvious, but you can split people into different areas of influence and each will have a slightly more removed view of what quality is and how important it is.

You and the test team – You are trying to help drive the culture towards a better focus on quality. To you, it is your main focus.
Your Delivery Team/Department – Quality will be one of the main focus points, not the only one.
Your Organisation – The further removed from quality being the main focus, the less attention given to issues. This is when it starts to become important to be able to map quality to what is important for the company. How does bad quality relate to revenue?

Company

Relating back to the people, the company could mean any level, but ultimately, it should relate to the overall view of quality across the whole organisation.

To the outside world, your company may not talk about quality, until something goes wrong. When it does, quality quickly comes to the forefront because it is then something tangible, whereas good quality doesn’t always have something so visible.

So How Do I Define our Narrative?

So you have an idea of what you are trying to define, the next step would be to start with understanding what the current Quality Narrative is. Start by asking questions to key personnel within every level mentioned above. Ask them about their understanding of Quality, the importance of it, where they believe it fits and how we can improve the quality of our products. You will likely get very different answers for each person you talk to, unless Quality is embedded within the culture already.

This may take time to build up, so don’t expect to be able to get this defined instantly. Once you have pulled this information together, think about the format of defining it. This could be a single slide in a deck or even better, a mindmap. Don’t make it too complicated, keep it succinct and straight to the point on. I would suggest breaking it down to the main 3 areas:

Perceived Quality Ownership	QA Department
Quality Focus	Any discussions around quality start at end of development phase
Value of Quality	Regulatory Requirement and purely confirmation testing (Checking requirements fulfilled)

A very brief example of an “As-Is” Quality Narrative

This should give you an understanding of where you are now, the next step would be to look at where you might want to get to. Understand there will be a journey to get there, but it’s a good time to collaborate with key people again to understand where they would like to focus on quality to be. Quality ultimately doesn’t just mean executing tests, there is far more to ensure the quality is good, everything from ensuring requirements meet what the customer is asking for, all the way through to the right monitoring/observability tools used in production to give feedback effectively and enable the product to be improved based on real usage. Tools such as MetroRetro or Miro can be a great tool (especially during these remote working times) to get people collaborating on this from wherever they are. Bring the key people together, brainstorm the ideas and collaboratively build out a future vision of where you may want to get to. This is then the first step on that journey. Then the hard work starts in moving the culture forward.

leadership, quality, Testing

Growing A Culture of Quality – A Model

Following on from my earlier blog post (Changing the Perception – Working towards a Culture of Quality), I have been lucky enough to have a submission selected for the Test Leadership Congress 2020 which is a virtual conference happening over July-October this year. This got me thinking more about this topic, and it really has become my thing to talk about. At every possible opportunity, I am trying to find ways to change the work culture to have more focus on Quality in any way I can. Whether this be trying to insert myself into Change Review meetings so that someone is representing testing to ask the difficult questions, or even whether it’s starting to define new processes which will help the teams move forward with providing more information about the quality of the system earlier.

With that, I started planning my talk and came up with a model which helps me frame my thoughts and works (for me atleast) when trying to make it a process which teams could follow to improve their working culture.

I’ll add a huge caveat here, that the teams and organisations which this has worked for me with, have not been agile in any particular way, maybe lipservice was paid to ceremonies, but there wasn’t a collaborative culture to any great extent. Testing was generally seen as an expense only considered towards the end of a project and getting involved any earlier was seen as a cost not worth considering.

So let me give you the model, I will then talk through the component parts and how it all plugs together:

So what is this Quality Narrative thing?

I first came across the term “Quality Narrative” , when reading the awesome Leading Quality book. Effectively, the Quality Narrative is how quality is perceived within your organisation. Some of the following questions may help you understand

How important is quality seen when releasing a product? Is it given the right focus? Or Is it an after-thought?
Who ‘own’ quality? Is it a collaborative accountability or do the wider business deem it the responsibility of the test team?
What is the perceived role of the test team? Do you even have a separate test team or do you work more collaboratively?
Are the test team engaged early in the process ?
How is testing done?
What Value does the testing provide?
What is the business’ view of risk?
Does Quality mean more than just testing?

Understanding the answers to these questions for the current state, will give you a fair assessment of the importance of Quality and testing within the organisation. The next step would be to then define what you would like the Quality Narrative to be going forward. Once you have the As-Is and To-be states, it will give you a vision to share and build out the journey to get closer to the ultimate state of Quality being an important consideration in every release.

So you have the vision… what next?

As a leader, you may have driven the definition of this vision, but it should have been a collaborative exercise (with others within the test team and wider delivery teams atleast) in defining the direction and getting buy in at a high level that it is acceptable. Once it’s defined, the next phase is to start engaging the immediate test team, so they can all be advocates for it. Use every opportunity to get them on board with it, give them time to digest it, ask questions and build enthusiasm for working towards the end goal.

One way to do this will be to get the fire lit on their passion for quality. Get an internal community of practice going if there isn’t already one, get passionate speakers to come in and share their ideas and give the team the chance to innovate and change the way they are doing testing too. If they see that their voice is important and you as a leader will listen to them, it will encourage them to stand up when needed, to voice their thoughts.

As small improvements are made to the way testing is done/measured/perceived, celebrate these successes, no matter the size. Seeing that any improvement makes a difference, will help the team feel like what they are doing is worthwhile.

So you have the team on board, now to share wider…

Now you have a passionate, engaged team who are all willing to make a difference and move the business forward, the next stage is to find the opportunities to raise the awareness of Quality around the business. Use different forums internally to share new processes such as metrics which are now being used or a new test strategy which focuses on the vision. Maybe there is an internal blog platform? Are there internal all-hands or departmental meetings which would be good opportunities to discuss such topics and how it could impact those teams? Discussing the test strategy with the customer support teams by framing it around reduced call volumes, would help them buy into the vision in a way that it can help them. By finding ways to discuss quality, it will help get you and your team involved in earlier discussions when projects are initiated. Therefore meaning that what “Good Quality” looks like for a particular project can be defined.

It may also be the opportunity to coach the business to test better themselves too, whether it be asking questions from a quality perspective or for teams which may be working on hot-fixing production or developing internal tools, but don’t have their own test resources, providing support and helping them learn to test effectively, will improve those relationships too.

Sharing is one thing, but can you find Advocates?

When sharing with the business, you may find there are a small group of people who really get it and want to find out more or find out how they can help. This happened to me and this was the time I started getting the TestSphere cards out and invited them to discussions with the test team on all things testing. I’d start getting comments like “I didn’t realise you Testers knew so much!”

Having these voices in wider teams, adds clout to the message. Especially when they are the ones to discuss it with their teams. I had advocates who would be discussing the updated test strategy, or the risk based testing metrics which we had devised and shared with the wider teams to show what we would be reporting on going forward. I actually walked in on a debate in the canteen, with no-one from the QA team involved, but they were discussing how much more confident they felt about the actual quality of the release with the metric I had devised and rolled out. It certainly gave the “warm fuzzys” that we were making a difference.

Improve the Quality Processes and Shout About it!

While also building the relationships with the business, it’s important that they are seeing the value and improvements that you are doing. Of course, alongside the daily expectation of proving the quality of the software through testing, there will be improvements and discussions going on to try and raise the bar and give better indicators of the quality of the software.

As mentioned earlier, this may be working closer with the Customer Support teams to assess call volumes and working on correlating specific customer issues with some additional testing which could be done to find these issues earlier. It may be looking at an approach to Testability which means the system is assessed as testable based on requirements/architecture before any code is develeoped which should mean the systems are of a better quality and less defects found later on. There could be a new approach to automation, maybe it’s been seen as a cost before that wasn’t worthwhile, but by showing the ROI and the feedback cycle reduction, it could be something which can enable earlier releases.

Whatever the initiatives end up being, it’s important to be transparent now that you have built the relationships with the wider business. They should be able to see what you’re doing and collectively reap the rewards of better quality systems.

Reflect, Rinse and Repeat…

This will be a constantly evolving process, a culture is never complete. Enabling it to grow will be important but it will require regular reflection on what could be done to improve and also assess where it isn’t working as well. Maybe there are pockets of the business which don’t see the value, so how could we speak their language and help them buy into it.

Reflecting and re-aligning the vision if necessary will be imperative to ensure it continues to be embedded as a culture. If you stick to your original focus and don’t flex when you need to, you won’t end up with a collaborative culture.

In bigger organisations where there are engineering groups everywhere, it will be important to use shared communities of practice to build the knowledge and share good practices. It will be important to set achievable goals like improving the view of Quality across your engineering group first including all stakeholders outside of engineering. But don’t try to boil the ocean, use regular reflection and feedback to know when to push yourself further or when to focus on honing what you have already achieved.

Let me know your thoughts on this, am I talking rubbish or does it relate? Come and join me at the Test Leadership Congress 2020 to hear me talk more about this. Register here