June 15, 2017

Inside the QA Automation Lab

Bottle Rocket’s Quality Assurance team is on an exciting automation journey. We’ve written the code, trained our manual testers, and are now focusing efforts on the place where it will all be put together. We’ve put a lot of hard work into our automation process and we wanted an automation lab that was just as awesome.

Our lab is an open space that is accessible to all. Shelves containing a collection of devices covering every major type, size, and operating system line the walls. Each device has its own stand and the shelves are backlit to maximize visibility. USB hubs with up to two amps per port are positioned to the side, allowing devices to charge and sync at the same time. The hub is connected to an iMac acting as the server. We anticipate being able to run tests on 32 devices at once per server.

Many of our large projects that have had multiple releases have had automation scripts created for them and can be continuously monitored. Any time a new build is created, a test plan can be triggered to download the build and run a smoke test on selected devices. This can really ease the pain of having to run the same tests on each build over and over, not to mention those server testing sessions in the wee hours of the morning.

Reports are generated and QA on the project are notified. Testers get reports from these runs complete with screenshots of each test outcome. These reports are divided into different sections, such as pages of the app, and the results are clear-cut with color-coded pass/fail icons. Each individual test can pass or fail based on criteria detailed in the test cases. If the tester is unable to see exactly what happened with just the screenshots, they can swing by the automation lab to run the test again and follow what happens in person.

This has been a journey because it wasn’t always easy; there were plenty of issues that had to be worked through along the way. Physical devices were a big problem. They’re definitely the most beneficial for testing because they are the closest we can get to real-world application of an app. The fact is, there are lots of issues that can be missed on a simulator, like things related to performance, battery level, or any kind of hardware component. The more variety we can get on the higher number of devices, the better. But working with this volume of devices has its hardships. Some of them required enhancements to our setup; high-performance devices (namely tablets) require lots of power to even maintain a charge, hence the beefy USB hub mentioned earlier. The less powerful hubs we used initially just could not keep up. Different devices can also have different settings menu navigation, meaning our test cases need to be general enough to apply to different hierarchies. Some devices even disconnect after a period of time.

Our automation engineers solved this one with a program they created called Vadr. Vadr is the interface that allows testers to access the lab and devices remotely. It shows all devices, their connection status (so we know when those difficult devices have disconnected), and allows us to choose which test/test plan to run on each device. This will make it easy for any QA tester to take advantage of our automation tools.

Our goals for an automation lab were accessibility, visibility, and efficiency. We ended up with that and more. As the physical space was created and evolved, our process and understanding of automation testing grew and solidified with it.

March 9, 2017

MWC Barcelona 2017: Jamon, Chaos, and Mobility

For those of you who couldn’t make it to MWC this year, we can catch you up with the first-person experience of Director of Strategy and Design (EMEA), Greg Flory. Here’s Greg’s run down from MWC Barcelona:

It’s Sunday and I’m still recovering from last week in Barcelona at Mobile World Congress. If you haven’t been, I totally get why that sounds a little ridiculous—after all, you’re in one of the nicest cities on the planet, constantly eating jamon Iberico, quaffing tasty Rioja and nibbling on that amazing bread with tomatoes and olive oil. And it’s all lovely. Seriously, you should go. But the reality is that MWC has you running around all day, and most nights, trying to make sense of the chaos created by the collision of technology, vision, globalization, the rapidly advancing future, and the rippling impact of mobile innovation on adjacent industries and technologies. It’s overwhelming. And exhausting.

So, as I gradually emerge from the dreamy Catalan fog, there are several takeaways that I’d like to quickly share:

  • Autonomy is a thing. We tend to think of this in terms of smart objects or connected cars—and there were cars everywhere throughout the exhibition halls—but it’s the impact on human experience that is truly interesting. We have arrived at a point where tools and information can remove uncertainty and the mundane, allowing us to invest our energy in what we care about the most. AI-powered bots will get better, giving us immediate access to precise solutions. Autonomous drones will inspect, map, and deliver to locations quicker, and more safely and efficiently than we can today. Even lighting, championed by Philips, will change dramatically, moving beyond simple illumination to help us heal physically and make sense of our environment in new ways.
  • Data is your business. Or your next business. Investing in mobile ensures that you will have access to information about your customers that you never knew was available. Brands such as Spotify are working with companies to help find better ways to engage their customers. Connected devices, aligned with connected cars, houses, and cities will create even more data, while revealing services and products that we couldn’t have imagined or seen previously. And there’s no excuse for not knowing your customer—the actual people—with names, preferences and an increasing array of options.
  • We all need partners. Now more than ever. Having worked exclusively in mobile for the past six years, I thought I was pretty aware of my limitations. But there are entire parts of the ecosystem that I didn’t know existed. It is expansive and there is opportunity across the spectrum. And wherever you are on the mobility journey, it is an enormous benefit to have the right people to help you manage all of the moving parts. And believe me, there is no shortage. For every one person I saw and/or bumped into on the conference floor, there were 20 trying to get through passport control on Friday morning. And obviously, I think Bottle Rocket is an excellent choice. If you think the partner suggestion contradicts my first point about autonomy, I’d just say that having the right partner allows you to focus on the areas of your unique expertise, ceding certain specialties to people best prepared to manage.

When I wasn’t speaking my unique brand of broken, largely unintelligible Tex-Mex Spanish to patient and accommodating Catlan cab drivers, I was most likely wandering around Halls 8.0, 8.1 and 3 of the Fira Gran Via prepping for and/or leading a technology tour with WPP’s Data Alliance. The tour may have been the best thing that could have happened since it forced me to explore and engage with a lot of people I would normally have avoided. It challenged some of my assumptions and confirmed others.

You can expect a healthy dose of what’s next at the world’s largest mobile gathering, but there seemed to be quite a few brands and manufacturers pushing back against the future, trying their best to pluck our taught little, nostalgic heartstrings. Here are two headliners and one wild card:

  • Nokia, with its 3310, demonstrates that you don’t really need a good reason to dredge up the past (unless this is intended for the developing world) and plenty of people crowded the table to get their paws on the retro hand candy. Looks fun. Feels great in the hand. But the proprietary OS is very much a drill down—endlessly—to take simple actions, then drill back out. This was an instant reminder that UX in the pre-smartphone era was painfully slow and often unrewarding. The 3310 has 22 hours of talk time (not that anyone really does that on their phone anymore), plus about a month of standby between charges. Seems just about right considering how infrequently anyone in the developing world would be likely to use this phone. But easily one of the most crowded stands at the show. I guess it’s kind of like stalking your old crush on Facebook. Nice to see how they’ve done over the years, but probably still pretty happy you’ve moved on.
  • BlackBerry's KEYone brings its CrackBerry heritage to the Android OS, delivering a physical keyboard to the brand's long-suffering addicts. If this quickens your pulse, enjoy, but I found the physical keyboard with its tiny buttons harder and less forgiving than a typical touchscreen. I was never a huge fan and easily moved on almost a decade ago. So, are we facing a resurgent BlackBerry that will draw legions of former obsessives out of the smartphone forest (much like the gobs of zombies in the near certain impending apocalypse)? I wouldn’t bet on it.
  • Moscow-based, Elari, producer of the self-proclaimed “anti-smartphone” Cardphone 3G, has an interesting array of products for people looking to simplify. Their phones are generally small, with the Cardphone looking like a minimalist calculator that can fit in your wallet. I think this is the phone that Walter “Heisenberg” White wishes he'd had as his second “business" phone.

November 21, 2016

TechWeek Recap: from Smartphones to Smart Cities

One of the most interesting panels at this year’s Dallas TechWeek featured Trey Bowles (CEO & Co-Founder of the Dallas Entrepreneur Center), Jen Sanders (Executive Director of the Dallas Innovation Center), Herb Sih (Managing Partner of Think Big Partners in Kansas City), and Tim Fleming (Director of Enterprise Sustainability at AT&T). The panel focused on the aspects of building a connected, smart city that many do not think about – data and efficiency.

When we think of smart cities we usually think of the visible changes, the science fiction stuff that’s coming into reality - drones, autonomous vehicles, robots, etc. This panel however took a different approach and discussed how to leverage data in smarter ways to improve current capabilities and make them more efficient. Trey Bowles gave us a glimpse of what a day might look like in a world where not only people are communicating with technology but machines are talking to each other. For example, what if sensors told the city waste pick-up when your cans are full…or not? What if traffic and route planning were sent directly to the autonomous car you requested since you no longer own a car. Better allocation and use of everything from power to emergency services would be invisible to most of us, but we’d see the benefits in costs and investment. Jen Sanders pointed-out that this isn’t going to happen quickly or to the same degree in every city. Some towns are better candidates for some things than others. Tim Fleming spoke to resources that we don’t consider now when he said “Cities consume 75% of the energy and produce 80% of emissions. What about the emissions of methane from Cows?” The conversation was rounded-out by Herb Sih who said “The city is a platform, much like a smartphone, that can do amazing things given connectivity.”

 

While it may be a while before we see any true smart cities, the world of connected devices continues to grow along with the benefits they can offer – contact us today to find out how to better leverage data and the internet of things for your business.

October 4, 2016

Quality Assurance Automation

In today’s mobile world, quality assurance automation is a new necessary in successful testing. It allows us to be hyper-focused on the quality of our client’s work by checking overlooked things without using team resources. At Bottle Rocket, we’ve defined a formal process for it that allows our QA team to concentrate on more complex testing.

It wasn’t easy developing an automation process from scratch. Deciding to adopt automation brought about a lot of questions: How would automation engineers be allocated within the company and our projects? How much work can one automation engineer handle? What technology would we need to adopt to accommodate them?

Since starting from scratch also meant that there were no design patterns or coding standards in place, there was no testing framework or concrete plan for integrating automation into our current processes and projects. So we started off by selecting our tools of trade – Appium and Postman.

 

Step one: deciding the tools

Appium is an industry-standard, open source quality assurance automation tool based on Selenium's WebDriver. It works by sending commands such as swipe, tap, and long press exactly like a user would. It also includes features such as flexibility to program in multiple languages and a large community surrounding the framework.

For server testing, we selected Postman. This tool sends and receives HTTP messages and can easily test various user flows through API calls. Some benefits of Postman include ease of use, great user experience, and flexibility.

 

Step two: getting the ball rolling

Working out the new processes and workflows of automation comprised of a lot of trial and error. For example, some significant hurdles included provisioning profiles and attempting to install iOS apps from the command line. In the end, Rocketeers discovered a way to create the right build and how to get it onto the device.

After developing a framework from scratch to form automated test plans, automation engineers designed test cases that are general enough to be used for any project. Although reusable test cases take longer to build, they save time in the long run because testers don’t have to write or edit tests for every new project.

Encapsulation, or the explicit bundling of certain parts of code, was also a major focus of our initial automation process. This helps change the target operating system or device for a test run without affecting the test cases themselves. Having two separate code pieces makes the overall framework much more configurable.

 

Step three: integrating automation

The first phase of integrating automation with our current QA process involved training for manual testers. We started off with the basics: programming with Java. We hosted an interactive coding class covering everything from objects to arrays in an approximately eight-week time frame.

After the training, Rocketeers moved on to automation itself with a focus on Appium. We were given some sample code to examine and edit, and after a few weeks, our team wrote scripts and manipulated the app on a physical device.

Since adopting automation, test coverage for both test cases and devices has improved, and testing volume has increased. In the future, we will continue to grow our automation lab with a myriad of test devices to choose from as well as expand our automation team and refine our framework.

 

Automation streamlines the quality assurance process and frees up our testers to focus on tasks that matter most to our clients. Make sure to contact our team of Rocketeers if you’re interested in hearing more about our automation services.

September 23, 2016

Coca-Cola Freestyle App Renovates for iOS 10 With Tasteful iMessage Extension

Fans of Coca-Cola Freestyle are drinking up the latest features of their newly updated iOS app.

Apple’s iOS 10 features extensions for maps, iMessage, and Siri that allow brands to add new touchpoints for their new or established apps. In Version 5.7.1 of the Coca-Cola Freestyle app, users can create and share favorite mixes straight from iMessage platform in a contextually natural way. This addition complements preexisting features such as the Location Finder tool, exclusive offers, and #MyMixMonday.

Make sure to check out these new iOS features by downloading Coca-Cola Freestyle over on the App Store. The app is also available for Google Play.

Discover more about Bottle Rocket’s latest app launches over on our blog.

© 2020 Bottle Rocket. All Rights Reserved.