Ship Late, Ship Infrequently: How Web Development Standards Damage Mobile Apps

By Michael Mace
Mobile Strategist, UserTesting

A few months ago I attended a meetup where mobile developers were talking about their application design challenges. One of the developers discussed her company’s troubles with online criticism from customers. Just a few negative reviews can sour the sales for an application permanently, she said. An Android staffer attending the meetup agreed: “The three things that drive an app’s success in the store are price, icon design, and the number of stars in the reviews.”

“Everyone tells us ‘ship early and often.’ But if we do that, we end up shipping an unoptimized app and we get bad reviews. What are we supposed to do?”

That made the developer frustrated. “Everyone tells us ‘ship early and often,’ ” she complained. “But if we do that, we end up shipping an unoptimized app and we get bad reviews. What are we supposed to do?”

It’s a good question, and one that I hear a lot from developers. The conventional wisdom in the web app world is that you should release your app to the public as soon as it’s barely functional, so you can start learning and iterating. “If your first version isn’t embarrassingly bad, you’re doing it wrong,” many consultants and VCs say.

Developers run into trouble when they apply that advice to mobile apps. In web development, you can iterate to your heart’s content — run endless A/B experiments, and change the features every day. Because there’s no app store to centralize reviews, there’s not a big downside to making an early version public. As long as you don’t publicize the early versions of the site heavily, first impressions don’t spread to the rest of your customer base.

But mobile apps work completely differently. On iOS, Apple limits how often you can release new versions of an app. Android allows more frequent updates, but customers don’t necessarily appreciate the frequent delays involved in loading new versions. And the app stores on all platforms have long memories. If your first version stinks, the smell from the bad reviews may linger for longer than you can afford to wait.

We need new ground rules for mobile developers

In some ways, mobile app development is a throwback to the old world of packaged PC software. In the old world, software couldn’t be updated frequently because it was sold through stores, on CDs or floppy disks. There was no practical way to update all of those disks, so one version stayed on the market until all the boxes sold through. An app was being updated frequently if it had one new version a year. Sales were driven by reviewers working for the tech magazines and buyer’s guides. An app that got bad reviews at launch usually would never get a second review, so developers had to make sure their first release was great.

That didn’t mean keeping a product in development forever. Like today, you couldn’t make money until you shipped, so it was important to get your product out quickly. But you didn’t compromise on your core features, and you sweated through multiple rounds of private beta testing to make sure everything worked right before the “Golden Master” moment when an application would be released.

We need to adapt some of those old business practices for the mobile app world. Specifically:

Be brutally clear about your target customer

In the web world, traffic is king. You want to bring as many people as possible to your site, and then sift them to find the customers. In mobile apps, attracting the wrong customers can be disastrous. If someone who’s not the target customer for your app tries it and doesn’t like it, they may well post a negative review even though your app is great for the people it was targeted at.  This means you need to actively discourage the wrong sort of customer from installing your app. In your marketing, your app store listing, and in any introductory panels you put at the front of your app, you should be brutally clear about what it does, who it’s for, and who it’s not for. You should actively discourage the wrong people from using the app.  It’s better to have a hundred downloads by the right sort of customer than a thousand downloads by the wrong ones.

“You should actively discourage the wrong people from using the app.  It’s better to have a hundred downloads by the right sort of customer than a thousand downloads by the wrong ones.”

Is this release really necessary?

We shouldn’t go back to the days of 12-month product releases, but hummingbird-quick updates are also not ideal in mobile. Every time you update your app, even if it’s an automatic background push, you potentially upset your users by creating delays and extra data traffic to their device. If you’ve made visible changes to the usage flow, that’s potentially even more disruptive to the user. Remember always that mobile users have short attention spans, and if they become disenchanted with your app they’ll probably just move on to something else – and won’t return.

Never ship your beta

In contrast to the web mantra of “ship early, ship often,” in mobile the mantra should be “test early, test often.” Because there’s a big downside to doing public releases, you need to incorporate frequent private feedback into your development process, from the earliest stages of development. Instead of shipping the earliest usable version of your app, you show it to your testers, get their feedback, make changes, and test again ­– all in private. Only when these tests are successful should you consider releasing to the public.

To put it another way, if you feel obligated to label your mobile app “beta,” you shouldn’t be shipping it.

Old-school PC application developers did testing by assembling a big panel of beta testers, often paid and signed to NDAs, who tested pre-release apps and documented their results. While this process was valuable, it was also expensive and slow. We can do better today.

How to get feedback

There are many ways to get private app feedback, ranging from easy and cheap (give it to your friends and see what they say) to complex and expensive (rent a focus group facility and pay strangers to come in and use your app). Any testing process introduces biases, so be aware of yours, so you can correct for them. I know someone at one very large mobile software company who told me that they do much of their testing internally, on other employees. “We design for ourselves,” he said. That makes it easy to recruit testers, but the company is notorious for creating obscure user interfaces that only an engineer could love. No wonder.

Ideally, your testing process should target your sort of customer. If you’re located in Silicon Valley, your testers should live outside the area. (People in the Valley tend to be influenced by the thinking of the industry, just as everyone in southern California thinks like a film industry executive.)

Structure your tests so they’re as close to real life as possible. The more artificial your testing process, the less meaningful your results will be. And because so much of mobile success is driven by how people feel about your app, be sure you’re testing for affinity to the app, not just usability. Get verbal feedback from your testers, and ask how they’re feeling, not just what they’re doing. Just looking at analytics and server logs is not sufficient; you need to understand what your users are thinking, in addition to what they’re doing.

At UserTesting, we’d be glad to help you with your testing needs, but the most important message is that however you do it, you need to find a way to test your app on real users before it hits the app store. Test early, test often – and do it before you ship.

This post was originally published on Usertesting.com

About the Author

Michael Mace is Mobile Strategist at UserTesting.com. A longtime veteran of Silicon Valley, he co-founded two software startups, worked as an executive at Apple and Palm, and consulted on strategy and product planning to many of the tech industrybs leading firms. Hebs a well-known tech industry speaker, with keynotes at the Apple Worldwide Developer Conference, speeches at conferences including CES and CTIA, and appearances on CNN, BBC, and Bloomberg TV.

Michael previously served as vice president of product planning and chief competitive officer at Palm, where he worked closely with app developers, network operators, and mobile phone manufacturers to plan for the future of mobile data. At Apple he held a variety of leadership positions, including director of Mac platform marketing, director of marketing for the Home and Education Division, and director of worldwide Customer & Competitive Analysis. He is the author of Map the Future, a book on planning strategy in fast-changing markets.

See Michael Live!

Join Michael in his session on “The Four Mobile & Responsive Traps” at Conversion Conference San Francisco 2014, March 17-19. Follow Michael on Twitter and ask for a promo code to save more on your pass.