We talked recently about throwing out our native app and starting over with Electron. But how does that affect me, the one QA Engineer testing Conveyor?
A few months ago our native mac app was full of features and required a lot of testing. Each week was full of iterating on the app and relaying feedback to the team. But the moment we decided to transition to Electron, the number of things available for testing dropped significantly.
The Electron app required new architecture, exploring different technologies, and a good bit of foundational work. Fast forward to today, and we almost have a testable app. So what has our team done to involve me between then and now? Here are a few things that helped.
Test important pieces in isolation
One of the most important features of our infrastructure is syncing. So rather than wait months for the UI to be wired up to begin testing it, Dima spent some time creating a command-line interface that I could use to verify that data was syncing as quickly and reliably as we hoped.
This went well. The bugs we found were easier to track down without having to start at the UI and work backward. And what I learned about the sync strategy will lead to a much better understanding of how to test the app as it matures.
Shorten the feedback loop
We don’t have our continuous integration pipeline fully in place yet. There is some manual work involved for developers to package the latest version of the app and distribute it to me for testing. To provide feedback more quickly, Ilya helped me get my MacBook setup to be able to build the latest version of the app locally.
The intention is to iterate on features while they are still in development. If I find bugs that are a quick fix, then we won’t even need to deal with the overhead of logging things in an issue tracker and sending tickets back and forth. We can handle it quickly in Slack while the developer has the code for that feature in their head and on their screen.
Communication is required
This approach requires clear communication for it to work. Today we have a semi-functioning Electron app. Some features are expected to work. Some features are expected to break. And others are somewhere in between. Before testing anything still in development, I need to know where a specific feature fits along that spectrum. That’s where the communication comes in.
After having a brief conversation with the developer in Slack or a video call, I can explore a specific feature with a solid understanding of what should work and what should be ignored.
Take the opportunity to learn
But that feeling vanished when I remembered that I was being given the opportunity to sit at my desk and focus on learning something new.
Wildbit is awesome y’all. Learning is my favorite thing to do at work, so it has been a blast to explore these new (to me) technologies. I plan to follow up with a more technical post on how to do QA for Electron apps. Or if we totally bomb, how not to do QA for Electron apps.
Chris has stated that Conveyor is one of those things that is either going to be brilliantly valuable or a brilliant failure. It’s so true. Our QA process continues to evolve as the app comes to life, but we are investing heavily in the quality of the app from the very beginning. And we’re working hard to make sure that you find it brilliantly valuable.