2022 on Heather Reid's Blog

Is it lonely in edge case land?

Tue, 07 Jun 2022 21:08:38 +0100

As testers, it seems we’re always living in “edge case” land. The risks we see before code is written. The scenarios we think of to test. Even the real use cases we observe in a percentage of our users.

It can sometimes feel like all of our ideas are edge cases. But who decides what an edge case is? And is it just an excuse not to consider something a priority as we develop our products?

Who decides? #

When it comes to a consensus on the definition of an edge case in your context, do you decide this as a team? Or, is it a decision that your team has made for them by another power?

Using the data #

More recently, I have found myself relying on the data a lot to advocate for something not being an edge case. The reason is twofold:

I’m reasonably new to the team so my “gut feeling” isn’t fine tuned quite yet,
The data rarely lies.

My team uses data to see things like:

Number of occurrences of a given error message,
Places where our users get stuck,
Number of people using iOS or Android and what version,
Screen sizes of the devices our users are using, etc.

We rely on the data a lot in our decision making process but also involve a healthy amount of common sense too.

No user would do that #

We’ve all heard this nugget at some point in our careers. Users will always surprise and delight.

What happens if you have data that says 10% of users are in “edge case” land?

What if that 10% equated to a single user? Or to 2 million users? Would you advocate for them both in the same way?

For every “no user would ever do that” comment, there will always be a non zero amount of users who do that.

Questions for you #

Do you use data to decide if something is an edge case? Is it based on a gut feeling? Or does it fall under the “no user would ever do that” mantra?

Do you find it lonely in edge case land? Or are you supported there by the team you have around you?

Concluding thoughts #

With all of this said, I find myself with more questions than answers. Am I on my own in edge case land? Or, do we all find ourselves on our own there from time to time unsure of how to advocate for the problems we see?

Interview Questions

Thu, 19 May 2022 20:29:38 +0100

When I started the interview process with my current company, I wanted to be sure that accepting a role with them, should I be offered one, was the right thing to do.

I missed software testing but I didn’t want to take just any role. I wanted to take a role where I knew I could learn, grow and flourish in my career.

With that in mind, here are the questions I asked throughout my interview process, why I asked them and how that helped me know we would be a good fit for each other.

What does a bad day look like at this company? #

This felt like a risky one to start with but I thought that it would help me frame follow on questions. I asked this of two interviewers, a manager and a team lead.

The managers response was “Down time because it can affect the long term perception of the business”. So, I knew based on this answer that customer perceptions are important, they care about their end user. Not everyone does in my experience.

The team lead responded with “People handing in their notice out of the blue because you feel like you missed some way to help them”. This showed to me a desire to learn from mistakes and an awareness about communication across the team, both things that are very important to me.

What is the most expensive lesson to date for the company in your opinion? #

I used this to drill into the bad side of things more with the company and I asked this in all of my interviews. My strategy with this approach was to see how aligned/aware everyone in the process was with the mistakes made to date.

I got multiple variations of the same answer at all levels of the interview process. “No testers in the early days.” Or “Not enough focus on quality”. It seemed that the company was now in a position where they saw the value testing provides across the development life cycle and how it could have helped them in the early days.

I didn’t want to get too excited about this whole company focus on quality but this was the type of thing I’d heard about at conferences and very much wanted to experience for myself.

What does a typical day look like on the team? #

I wouldn’t have mentioned this question in this post were it not for one stand out answer I got “We’re using the agile principles that make sense in our context”.

So, nobody was shoehorning in agile ceremonies just because everyone else was. Nobody was forcing stand ups or retrospectives or story points just because “that’s agile”.

What challenges are you most proud of overcoming in the role? #

I didn’t want all the questions to be doom and gloom! I wanted to know how they defined success and how they viewed the journey to getting there. Was it all them? Did they acknowledge others? Did they feel like they had been set up for success at all? Did they feel like there was more growth for them in their roles?

Again, the answers here helped to give me valuable insight into the teams I would be working with and helped me to start thinking about how I would communicate with them.

What attracted you to this company? #

This was a fun one to ask because I followed it up with “And is that still the case for you?”

It helped me to get an insight into the journey of everyone so far. Additionally, I was able to gauge if they were actually happy based on their responses to the follow up, i.e. they hadn’t been sold a lie in the interview process 😅

If you could change one thing in the current processes, what would it be? #

On the surface of things, this might seem like a vanilla question to ask. However, the answers I received to this were incredibly honest. I got a completely transparent set of responses that let me know exactly what awaited me should I start in the company and that didn’t scare me.

Running My First 'Bug Bash'

Mon, 28 Mar 2022 18:30:30 +0000

I closed out the first blog in this 3 part series saying that I was going to create a template Miro board in advance of the session. I felt that a well structured board would not only help me with setting the scene for everyone once the bug bash kicked off, but it would also, hopefully, ensure that the session was valuable for everyone.

Lining Everything Up #

I set up a Miro board with the following sections:

1. Feature(s) #

The Jira ticket links relevant to this particular bug bash/feature.

2. Out of Scope #

Any thing we didn’t want people focusing on in the session. E.g. for ours we wanted to explore adding a single thing to a cart. Multiple items is a future feature and so, an out of scope feature was being able to add multiple items to the cart.

3. Risks Identified #

Anywhere we thought there might be a problem but we didn’t get a chance to explore yet or we wanted to explore in more detail. Particularly helpful for mobile specific things if you think you have some device or operating system specific issues.

4. Environment #

The environment everyone will be testing in. Is it a feature branch, production, etc. Also is it on Android or iOS.

5. Account details #

We needed test accounts set up in advance for people to use to help reduce setup time during the session, the details of these accounts were added to the Miro board. Some of the developers who were more familiar with the feature could also set up brand new ones if they wished.

6. Charters #

We set up a few charters based on the risks we’d identified. Not so many as to bias people but enough that if people attended from outside of engineering and didn’t know where to start, they had charters to give them a starting point.

As we’re completely remote, the link to the Miro board and any instructions to complete before attending were sent out in advance.

The Main Event #

When the session officially started, we spent 10 minutes giving a run down of the feature, highlighting the charters and how they could help with inspiration, how we’d like people to report issues, how long we would spend exploring (half an hour) and how we would debrief at the end.

Luisa, the person who developed the feature, and I were available throughout the session for people to unmute and ask us questions when they needed to. We then spent 15-20 minutes debriefing at the end and outlining how people could raise any things they thought about after the session with us.

Learnings #

I’d call this a huge success for my first ever Bug Bash. Next time I’m hoping to:

Be more familiar with the feature before the session
Send the Miro board out further in advance of the session
Refine the Miro board to be leaner, there was a bit too much detail in sections of it.

I hope to blog about applying these learnings when I get a chance. Watch this space!

In an effort to keep my blogs shorter, I’ve split this story into 3 parts. Read part 1 and part 2.

A Bug Bash By Any Other Name...

Mon, 14 Mar 2022 09:03:59 +0000

In my last blog, I set the scene for this blog post. The TLDR is I was going to be running my first bug bash with a new team and I’d never run one before.

Is It Really a Bug Bash? #

Throughout the conversations we’d had so far, one thing constantly jumped out to me. We were calling it a bug bash but the goal of it wasn’t really to find bugs. Was it?

I decided to put the question to the test team.

Hugh suggested “If you call it a bug bash then I would say it is. If you call it something else then it would/could have wider goals.”

Rob followed up with “The goal is to learn and gain a common understanding of what we’re delivering. Part of it is getting to know the feature. Ideally we shouldn’t find many bugs.”

A Bug Bash By Any Other Name #

So, what should we call it instead?

I really liked the idea of calling it a treasure hunt. It felt like a much less aggressive term than “bash” but I was worried that people wouldn’t take it/me seriously if I called it that and I was already nervous about hosting one so that was off the table (for now).

Hugh said that the team had been calling them exploratory testing sessions recently. This felt like a good fit because the goal was to explore and uncover information. If this was a term that people outside of engineering had grown familiar with, they may be more likely to join which would be amazing.

Proposing a Name Change #

In the second pairing session I had with Luisa, I proposed the name change. I suggested that if the goal wasn’t to find bugs and people joined the session with “bugs” in the title and found no bugs, they may feel like they didn’t help us achieve our goal and not want to attend again.

The great thing about my new team is that they’re willing to experiment (they have a session once every two weeks to try experiments! More on that later) and so, the name of the session was changed from bug bash to “exploratory testing session”. Amazingly, as soon as that change happened, 2 people who had RSVP’d no changed to a yes!

Was it a coincidence? Was it the name change? I think I’ll have to try treasure hunt the next time to see!

In an effort to keep my blogs shorter, I’ve split this story into 3 parts. Read part 1 and part 3.

Preparing for My First Bug Bash

Thu, 10 Mar 2022 18:07:15 +0000

I recently started back into software testing and joined the mobile team in Glofox.

One of the first things I got to jump into was running a bug bash remotely for the new team. I’m not going to lie, I was incredibly nervous as I’d never run one before, never mind running one remotely.

Getting the Context #

The team had been working on developing a new feature for clients that was going to be a pretty big release for the business.

One of the developers on my new team, Luisa, suggested that a bug bash would be a good opportunity to help polish the feature before rolling it out.

The team asked if I’d be willing to help them prepare for and host the bug bash. Cue the deer in headlights moment 😳 I had never run or participated in a bug bash! I knew Conor Fitzgerald had blogged about running one though so I made a mental note to track that down. Luisa offered to help me run the session too so I wouldn’t be completely out of my depth.

Luisa also mentioned that the best bug bash she had ever had the opportunity to attend in the company had been run by Julia. So, the first question I asked was “Why?” Followed by “What made it better than other ones you attended?”

With my head spinning full of ideas based on the answers the team gave to those questions, I set about getting the ideas down into a Google doc.

Finding Resources #

The next step was to find Conor’s blog where he talked about the Guidelines for Running a Bug Bash. I pulled out parts of this that I thought would work in our context based on the information I had so far.

Then I asked in the test team Slack channel if anyone had any resources from running one in the company in the past. Again, Julias name popped up! She had been working on a template for bug bashes and had some resources to share with me.

Using Conors blog, Julia’s resources, and the Google doc of ideas, I was starting to form a picture of where I wanted to go with this.

Pairing #

Julia offered to pair with me to get a template together for the session. I jumped at the opportunity! We went through everything she had shared with me and how she structures bug bashes herself along with things to watch out for based on her experiences. I asked a lot of questions to help decide if the ideas I had for the approach would work. We wrapped up with Julia offering to help however she could in advance of the session. I was starting to feel less out of my depth now.

Next, I paired with Luisa. We went through my Google doc of ideas and the tips Julia had shared. We agreed on what we wanted to get from the session, what we didn’t want from the session and divided up tasks to be done in advance of the bug bash.

We agreed that I would set up a template Miro board for the session and we’d pair again the following week to refine it once together and then once more with Julia before we had the actual bug bash.

We wanted to be sure that we were giving the bug bash a good structure so that everyone would find it valuable and we really felt like we were starting to get there.

In an effort to keep my blogs shorter, I’ve split this story into 3 parts. You can now read part 2 and part 3.

Getting Back to Testing With a Shiny New Blog

Tue, 08 Mar 2022 20:44:32 +0000

It’s time to get back into blogging!

Recently I announced that I would be leaving my role as CommunityBoss with Ministry of Testing to become a Test Engineer with Glofox.

It's officially official. I'm saying goodbye to team @ministryoftest to become a Test Engineer with @weareglofox 🎉

A mix of nerves of excitement writing this ❤️https://t.co/1sqC2zB1s0
— Heather Reid (@heather_reiduff) January 18, 2022

I’ve already had a tonne of amazing experiences in the role that I want to blog about so here I am, committing myself to writing up as many of them as I possibly can on my shiny new blog 😎

Watch this space…..