My Projects

September 8, 2009 78 comments so far

Apple's App Store Approval Process - Suggestions for Improvements

Let's take a slight detour and take a look at Apple's App Store approval process. The approval process is what each application update needs to go through in order to become available in Apple's iPhone application online store. As each review can be seen as a small project I would like to analyze how it might work today and why I believe that it could work much better by adopting a couple of improvements.

Until recently, nobody really knew how the App Store approval process really works. Since Apple's open letter to the FCC (answering the FFC's questions) we have at least an idea and to be honest, for me personally it confirms some of my worst fears. Since I was personally a "victim" of the process once in a while I will still try to be as objective as possible, but of course I cannot guarantee it 100% ;-).

The major problem with this process as it is outlined by Apple is simply that it does not scale (anymore). It never worked really smoothly, since there have been problems and inconsistencies from day one. However, since a couple of months now there are really fundamental problems and major delays that are not surprising if you simply do the math based on the information in Apple's letter: They are getting 8,500 updates per week and they are reviewing each update two times with 40 full time reviewers. Given that it would be "nice" that updates are again approved within a week's time frame it would mean that every reviewer would have to review 425 app updates per week or 85 per day or more than 10 applications per hour on an ideal 8 hour productive work day.

This fact together with the in-depth type of review they insist to do cannot work with more than 70,000 apps in the App Store (at least in my opinion and based on my experience in project and process management). However, I have a number of ideas how the process could be improved in order to speed things up and to reduce some bottlenecks. Please note that I only have the same information than you do and I only can guess about the rest, but here are my improvement suggestions:

  • Focus on the important stuff. Do not do excessive feature reviews anymore. Instead, focus on discovering real bugs and security problems. Especially, try not to judge what could be "confusing to the user". I had two cases myself where a reviewer was "confused", but not a single business user (which is not surprising given the numbers above: if the reviewers are human beings they cannot have enough time to really understand what is going on in a more complex or innovative app...)
  • One in-depth review is sufficient. Having two independent reviews for each update is nice in an ideal world, but I believe it creates a major bottleneck in the process. Review the review if you need to, but do not double or triple the effort. In my opinion, this should work well in combination with improvement suggestion #1
  • Differentiate new features from bug fixes. Not a single quality assurance/control team I know checks a bug-fix update the same way as a feature-loaded release. Still, when submitting an update there is no possibility to state whether it is a bug fix release or a feature release. My improvement suggestion: let the developer choose the type of update such as "Major Feature Release", "Minor Feature Release" and "Bug Fix Release". Bug fix releases should have priority over new feature releases in the approval process
  • Provide a "Hot Fix" update type. Sometimes it is very important to get a critical bug fix out of the door as quickly as possible. I experienced this first hand when iPhone OS 3 was released. I found an incompatibility that only occurred with SDK Beta 5 and fixed it ASAP, but I (and my users) needed to wait for this critical bug fix for nearly three weeks. By having a separate "Hot Fix" update that again has priority over all other update types Apple could assure that hot fixes really get released as fast as possible
  • Automate the process (as far as it makes sense). I guess this is something which is most probably done already anyway. Especially for a sand-boxed environment such as an iPhone OS application it should be quite easy to create automatic test scripts that check for usage of undocumented APIs and for common security problems. If automating the process is not yet done extensively I would strongly suggest looking into it. We are seeing the benefits for similar processes that we need to support internally every day (at Onepoint Software)

As you probably already concluded from my argumentation above, in my opinion it "deteriorates" the iPhone user experience far more if a critical bug fix needs weeks to get reviewed than if a single user on the planet gets maybe confused by an app. But maybe it's just me ;-). Don't get me wrong, I am a big fan of Apple, the iPhone and the App Store. But I believe if Apple does not fix this review process in time they might loose their competitive advantage faster than they think (which I would not like to see). What do you think?