How to involve QA before your app is ready

An illustration of a conveyor belt with metaphorical testing devices and tools.

We talked recently about throwing out our native app and starting over with Electron. But how does that affect me, the one QA Engineer testing Conveyor?

A few months ago our native mac app was full of features and required a lot of testing. Each week was full of iterating on the app and relaying feedback to the team. But the moment we decided to transition to Electron, the number of things available for testing dropped significantly.

The Electron app required new architecture, exploring different technologies, and a good bit of foundational work. Fast forward to today, and we almost have a testable app. So what has our team done to involve me between then and now? Here are a few things that helped.

Test important pieces in isolation

One of the most important features of our infrastructure is syncing. So rather than wait months for the UI to be wired up to begin testing it, Dima spent some time creating a command-line interface that I could use to verify that data was syncing as quickly and reliably as we hoped.

This went well. The bugs we found were easier to track down without having to start at the UI and work backward. And what I learned about the sync strategy will lead to a much better understanding of how to test the app as it matures.

Shorten the feedback loop

We don’t have our continuous integration pipeline fully in place yet. There is some manual work involved for developers to package the latest version of the app and distribute it to me for testing. To provide feedback more quickly, Ilya helped me get my MacBook setup to be able to build the latest version of the app locally.

The intention is to iterate on features while they are still in development. If I find bugs that are a quick fix, then we won’t even need to deal with the overhead of logging things in an issue tracker and sending tickets back and forth. We can handle it quickly in Slack while the developer has the code for that feature in their head and on their screen.

Communication is required

This approach requires clear communication for it to work. Today we have a semi-functioning Electron app. Some features are expected to work. Some features are expected to break. And others are somewhere in between. Before testing anything still in development, I need to know where a specific feature fits along that spectrum. That’s where the communication comes in.

After having a brief conversation with the developer in Slack or a video call, I can explore a specific feature with a solid understanding of what should work and what should be ignored.

Take the opportunity to learn

Even as we try to introduce testing earlier in the development cycle, there are days when nothing is ready to be tested. Those days are a perfect opportunity to learn how to write the automated tests that will be used to speed up our release process and reduce the mind-numbing repetition that would be required otherwise. JavaScript is the obvious choice when determining which language to use for test automation in our Electron app.

One big challenge is that I am new to JavaScript and have struggled to pick it up as quickly as I’d like. I’m tempted to complain that JavaScript has a ton of historical baggage and many of the instructional resources I stumble upon are out of date with modern JavaScript development techniques. But even when I find up to date tutorials, I’ve been slow to master the content.

I tried using Spectron to write some tests, and it didn’t go that well. I ran into errors and didn’t have the know-how to debug them. I had to step back and better understand vanilla JavaScript, ES6, Node.js, Electron, Spectron, and WebdriverIO. I became discouraged by how difficult it was for me to understand everything.

But that feeling vanished when I remembered that I was being given the opportunity to sit at my desk and focus on learning something new.

Wildbit is awesome y’all. Learning is my favorite thing to do at work, so it has been a blast to explore these new (to me) technologies. I plan to follow up with a more technical post on how to do QA for Electron apps. Or if we totally bomb, how not to do QA for Electron apps.

Chris has stated that Conveyor is one of those things that is either going to be brilliantly valuable or a brilliant failure. It’s so true. Our QA process continues to evolve as the app comes to life, but we are investing heavily in the quality of the app from the very beginning. And we’re working hard to make sure that you find it brilliantly valuable.

Get updates and early access