Infrequently Noted

TL;DR: App stores exist to provide overpowered and easily abused native app platforms a halo of safety. Pre-publication gates were valuable when better answers weren't available, but commentators should update their priors to account for hardware and software progress of the past 13 years. Policies built for a different era don't make sense today, and we no longer need to accept sweeping restrictions in the name of safety. The idea that preventing browser innovation is pro-user is particularly risible, leading to entirely avoidable catch-22 scenarios for developers and users. If we're going to get to better outcomes from stores on OSes without store diversity or side-loading, it's worth re-grounding our understanding of stores.

Contemporary debates about app stores start from premises that could use review. This post reviews these from a technical perspective. For the business side, this post by Ben Thompson is useful context.

...Only Try To Realize The Truth: There Is No "App" #

OSes differ widely in the details of how "apps" are built and delivered. The differences point to a core truth: technically speaking, being "an app" is merely meeting a set of arbitrary and changeable OS conventions.

The closer one looks, the less a definition of "appiness" can be pinned down to specific technologies. Even "toy" apps with identical functionality are very different under the covers when built on each OSes preferred stack. Why are iOS apps "direct metal" binaries while Android apps use Java runtimes? Is one of these more (or less) "an app"? The lack of portability highlights the absence of clear technical underpinnings for what it means to "be an app".

To be sure, there are platonic technical ideals of applications as judged by any OS's framework team^[1], but real-world needs and behaviour universally betray these ideals.^[2] Native platforms always provide ways to dynamically load code and content from the network, both natively or via webview and choice of code platform is not dispositive in identifying if an experience is "an app".

The solid-land definition "an app" is best understood through UI requirements to communicate "appiness" to end-users: metadata to identify the publisher and provide app branding, plus bootstrap code to render initial screens. Store vendors might want to deliver the full user experience through the binary they vend from their stores, but that's not how things work in practice.

Industry commentators often write as though the transition to mobile rigidly aligned OSes with specific technology platforms. At a technical level, this is not correct.

Since day one, iOS has supported app affordances for websites with proprietary metadata, and Android has similar features. "Appiness" is an illusion, an arbitrary line drawn to include programs built on some platforms but not others, and all of today's OSes allow multiple platforms to expose "real apps."

Platforms, while plural, are not equal.

Apple could unveil iOS with support for web apps without the need for an app store because web applications are safe by default. Circa '07, those web apps were notably slower than native code could be. The world didn't yet have pervasive multi-process browser sandboxing, JS JITs, or metal-speed binary isolation. Their arrival, e.g. via WASM, has opened up entirely new classes of applications on a platform (the web) that demands memory safety and secure-by-default behavior as a legacy of 90s browser battles.

In '07, safety implied an unacceptable performance hit on slow single-core devices with 128MiB of RAM. Many applications couldn't be delivered if strict protection was required using the tools of the day. This challenging situation gave rise to app stores: OS vendors understood that fast-enough-for-games needed dangerous trade-offs. How could the bad Windows security experience be prevented? A priori restraint on publishing allowed vendors to set and enforce policy quickly.

Fast forward a decade, and both the software and hardware situations have changed dramatically. New low-end devices are 4-to-8 core, 2GHz systems with 2GiB of RAM. Sandboxing is now universal (tho quality may differ) and we collectively got savvy about running code quickly while retaining memory safety. The wisdom of runtime permission grants (the web's model) has come to be widely accepted.

For applications that need peak compute performance, the effective tax rate of excellent runtime security is now in the 5-10% range, rather than the order of magnitude cost a decade ago. Safety is within our budget, assuming platforms don't make exotic and dangerous APIs available to all programs — more on that in a second.

So that's apps: they're whatever the OS says they are, that definition can change at any moment, and both reasonably constrained and superpowered programs can "be apps." Portability is very much a possibility if only OSes deign to allow it^[3].

What Is An "App Store" Technically? #

App stores, as we know them today, are a fusion of several functions:

1. Security screens to prevent malign developer behavior on overpowered native platforms
2. Discovery mechanisms to help users find content, e.g., search
3. App distribution tools; i.e., a CDN for binaries
4. Low-friction payment clearing houses

Because app stores rose to prominence on platforms that could not afford good safety using runtime guards, security is the core value proposition of a store. When working well — efficiently applying unobjectionable policy — store screening is invisible to everyone but developers. Other roles played by stores may have value to users but primarily serve to build a proprietary moat around commodity (or lightly differentiable) proprietary, vertically integrated platforms.

Amazon and search engines demonstrate that neither payment nor discovery requires stores. If users are aware of security in app stores, it's in the breach, e.g., the semi-occasional article noting how a lousy app got through. The desired effect of pre-screening is to ensure that everything discovered in the store is safe to try, despite being built on native platforms that dole out over-broad permissions. It's exceedingly difficult to convince folks to use devices heavily if apps can brick the device or drain ones bank account.^[4]

The Stratechery piece linked at the top notes how important this is:

It is essential to note that this forward integration has had huge benefits for everyone involved. While Apple pretends like the Internet never existed as a distribution channel, the truth is it was a channel that wasn’t great for a lot of users: people were scared to install apps, convinced they would mess up their computers, get ripped off, or accidentally install a virus.

Benedict Evans adopts similar framing:

Specifically, Apple tried to solve three kinds of problem.

• Putting apps in a sandbox, where they can only do things that Apple allows and cannot ask (or persuade, or trick) the user for permission to do ‘dangerous’ things, means that apps become completely safe. A horoscope app can’t break your computer, or silt it up, or run your battery down, or watch your web browser and steal your bank details.

• An app store is a much better way to distribute software. Users don’t have to mess around with installers and file management to put a program onto their computer - they just press ‘Get’. If you (or your customers) were technical this didn’t seem like a problem, but for everyone else with 15 copies of the installer in their download folder, baffled at what to do next, this was a huge step forward.

• Asking for a credit card to buy an app online created both a friction barrier and a safety barrier - ‘can I trust this company with my card?’ Apple added frictionless, safe payment.

These takes highlight the value of safety but skip right past the late desktop experience: we didn't primarily achieve security on Windows by erecting software distribution choke points. Instead, we got protection by deciding not to download (unsafe) apps, moving computing to a safe-by-default platform: the web.^[5]

In contrast with the web, native OSes have historically blessed all programs with far too much ambient authority. They offer over-broad access to predictably terrifying misfeatures (per-device identifiers, long-running background code execution and location reporting, full access to contacts, the ability to read all SMS messages, pervasive access to clipboard state, etc.).

Native platform generosity with capabilities ensures that every native app is a vast ocean of abuse potential. Pervasive "SDK" misbehavior converts this potential to dollars. Data available to SDK developers make fears about web privacy look quaint. Because native apps are not portable or standards-based, the usual web-era solutions of switching browsers or installing an extension are unavailable. To accept any part of the native app ecosystem proposition is to buy the whole thing.

Don't like the consequences? The cost of changing one's mind now starts at several hundred dollars. The moat of proprietary APIs and distribution channels is sold as a security necessity, but it's primary role is to protect margins. The lack interoperability is designed to prevent users from leaving. The web, meanwhile, doesn't need stores because browsers are safer.

This situation is improving at the pace of hardware replacement, which is to say, glacially.^[6] The sluggish pace of mobile OSes in reducing the over-powered permissions and APIs they give native apps demonstrates the tension at the heart of proprietary ecosystems — they must put up with abuse of users because the alternative would reduce the need for their gatekeeping.

The security pantomime of app stores is important to the perception by users and regulators that they are somehow necessary. Mobile platforms won't provide real safety because they recall what happened to Windows. When the web got to parity in enough areas, interest in the proprietary platform receded to specialist niches (e.g., AAA gaming, CAD). Portable, safe apps returned massive benefits to users who no longer needed to LARP as systems administrators.

The web's safety transformed software distribution so thoroughly that many users stopped thinking of applications as such. Nothing is as terrifying for a vertically integrated hardware vendor as the thought that developers might leave their walled garden, leading users to a lower-tax-rate jurisdiction.

Safety enables low friction, and the security theatre of app stores doesn't indicate safety, but mismanagement. Windows, Android, and ChromeOS allow web apps in their stores, so long as the apps conform to content policies, showing that there's nothing "non native" about the web. Today, developers can reach users of all these devices on their native app stores from a single codebase. Only one vendor prevents developers from meeting user needs with open technology.

This prejudice against open, unowned, safe alternatives exists to enable differentiated integrations. Why? To preserve the ability to charge much more over the device's life than differences in hardware quality might justify. The downsides now receiving scrutiny are a direct consequence of this strategic tax on users and developers. Restrictions, once justifiably required to maintain safety, are now antique. What developer would knowingly accept such a one-sided offer if made today?

Compound Disinterest #

Two critical facts about Apple's store policies bear repeating:

1. Apple explicitly prevents web content within their store
2. Apple prevents browser makers from shipping better engines in alternative iOS browsers^[7]

It is uniquely perverse that these policies ensure the web on iOS can only ever be as competent, safe, and attractive to developers as Apple allows.

Were Apple producing a demonstrably modern and capable browser, it might not be a crisis, but Apple's browser and the engine they force others to use is years behind.

The continuing choice to under-invest in Safari and WebKit has an ecosystem-wide impact, despite iOS's modest reach. iOS developers who attempt to take Apple up on their generous offer to reach users through the web are directly harmed. The low rate of progress in Apple's browsers all but guarantees that any developer who tries will fail.

But that's not the biggest impact. Because the web's advantage is portability, web developers view features not available "everywhere" as entirely unavailable. The lowest common denominator sets a cap on developer ambitions, and Apple knows it. App Store policies taht restrict meaningful browser competition ensure the cap is set low so as to prevent real competition with Apple's proprietary tools and distribution channel.

Apple doesn't need a majority of web usage to come from browsers without critical features to keep capabilities from being perceived skeptically; they only need to keep them out of the hands of, say, 10% of users. So much the better if users who cannot perceive or experience the web delivering great experiences are wealthy developers and users. Money talks.

Thirteen years of disinterest in a competitive mobile web by Apple has produced a crisis for the web just as we are free of hardware shackles. The limits legitimated the architecture of app store control are gone, but the rules have not changed. The web was a lifeboat away from native apps for Windows XP users. That lifeboat won't come for iPhone owners because Apple won't allow it. Legitimately putting user's interests first is good marketing, but terrible for rent extraction.

The Mobile Web: MIA

Arguments against expanding web capabilities to reach minimum-mobile-viability sometimes come wrapped in pro-privacy language from the very parties that have flogged anti-privacy native app platforms^[8]. These claims are suspect. Detractors (perhaps unintentionally) conflate wide and narrow grants of permission. The idea that a browser may grant some sites a capability is, of course, not equivalent to giving superpowers to all sites with low friction. Nobody's proposing to make native's mistakes.

Regardless, consistent misrepresentations are effective in stoking fear. Native apps are so dangerous they require app store gatekeepers, after all. Without looking closely at the details, it's easy to believe that expansions of web capability to be similarly broad. Thoughtful, limited expansions of heavily cabined capabilities take time and effort to explain. The nuances of careful security design are common casualties in FUD-slinging fights.

A cynic might suggest that these misrepresentations deflect questions about the shocking foreclosure of opportunities for the web that Apple has engineered — and benefits from directly.

Apple's defenders offer contradictory arguments:

1. Browsers are essential to modern operating systems, and so iOS includes a good browser. To remain "a good browser," it continually adds and markets new features.
2. Browsers are wildly unsafe because they load untrusted code and, therefore, they must use only Apple's (safe?) engine.^[9]
3. App stores ensure superpowered apps are trustworthy because Apple approves all the code.
4. Apple doesn't allow web apps into the store as they might change at runtime to subvert policy.

Taken in isolation, the last two present a simple, coherent solution: ban browsers, including Safari. Also, webviews and other systems for dynamically pushing code to the client (e.g., React Native). No ads that aren't pre-packaged with the binary, thanks. No web content — you never can tell where bits if of a web page might be coming from, after all! This solution also aligns with current anti-web store policies.

The first two points are coherent if one ignores the last decade of software and hardware progress. They also tell the story of iOS's history: 13 years ago, smartphones were niche, and access to the web corpus was important. The web was a critical bridge in the era before native mobile-first development. Strong browser sandboxing was only then becoming A Thing (TM), and resource limits delayed mobile browsers from providing multi-process protection. OS rootings in the low-memory, 1-2 core-count era perhaps confirmed Cupertino's suspicions.^[10]

Time, software, and hardware reality have all moved forward. The most recent weaksauce served to justify this aging policy has been the need for JIT in modern browsers. Browser vendors might be happy to live under JIT-less rules should it come to that — the tech is widely available — and yet alternative engines remain banned. Why?

Whatever the reason, in 2020, it isn't security.

Deadweight Losses #

Undoubtedly, web engines face elevated risks from untrusted, third-party content. Mitigating these risks has played a defining role in evolving browsers for more than twenty years.

Browsers compete on security, privacy, and other sensitive aspects of the user's experience. That competition has driven two decades of incredible security innovation without resorting to prior-restraint gatekeeping. The webkit-wrapper browsers on iOS are just as superpowered as other apps. They can easily subvert user intent and leak sensitive information (e.g., bank passwords). Trustworthiness is not down to Apple's engine restrictions; the success of secure browsers on every OS shows this to be the case. These browsers provide more power to users and developers while maintaining an even more aggressive security posture than any browser on iOS is allowed to implement.

But let's say the idea of "drive-by" web content accessing expanded capabilities unsettled Apple, so much so that they wanted to set a policy to prevent powerful features from being available to websites in mere tabs. Could they? Certainly. Apple's Add to Home Screen feature puts installed sites into little jails distinct from a user's primary browsing context. Apple could trivially enforce a policy that unlocked powerful features only upon completion of an installation ceremony, safe in the knowledge that they will not bleed out into "regular" web use.^[11] We designed web APIs to accommodate a diversity of views about exposing features, and developers guard against variable support already.

Further, it does not take imagination to revoke features should they be abused by a site. Safari, like all recent-vintage browsers, includes an extensible, list-based abuse mitigation system.

The preclusion of better iOS browsers hinges on opacity. Control drives from developer policy rather than restrictions visible to users, rendering browser switching moot. Browser makers can either lend their brands to products they do not think of as "real browsers" or be cut off from the world's wealthiest users. No other OS forces this choice. But the drama is tucked behind the curtains, as is Apple's preference.

Most folks aren't technical enough to understand they can never leave Hotel Cupertino, and none of the alternative browsers they download even tell them they're trapped. A more capable, more competitive web might allow users to move away from native apps, leaving Apple without the ability to hold developers over the barrel. Who precludes this possibility? Apple, naturally.

A lack of real browser competition coupled with a trailing-edge engine creates deadweight losses, and these losses aren't constrained to the web. Apple's creative reliance on outdated policy explicitly favors the proprietary over the open, harming consumers along the way.

Suppose Apple wishes us to believe their app store policies fair, while simultaneously claiming the web is an outlet for categories of apps they do not wish to host in their store. In that case, our pundit class should at least query why Apple will (uniquely) neither allow nor build a credible, modern version of the web for iOS users.

Update: my colleague Emily Stark has just written an essential-reading post on this topic. We probably need to write something about all of the non-prompt controls that Chrome has put in place behind the patterns discussed in this post, but Emily's post really drives home the case for flexibility with practical examples.

From time to time, working groups (particularly at the W3C) are asked to adopt requirements on browser-presented UI (pixels drawn outside or over-top the page content) in the normative text of a specification. For example, someone might advocate for text like along the lines of "before proceeding to Step 5, conforming User Agents MUST request user consent". Less frequently, advocates will push for the inclusion of specific text to be presented, or in the most extreme cases, specific UI layouts.

The most intense of these debates occur in the context of security-sensitive moments: warning users of potential risks, requesting permission to access sensitive features, etc. Thankfully, these efforts usually fail in the face of bristling browser feedback.

What's going on here? Don't browsers care about providing consistent UI? Don't they want the best possible security?

The answer to both questions is an emphatic yes. Counter-intuitively, it is because browsers care so much about security, privacy, and consistency (in that order) that we err on the side of rejecting constraints on security UI. In the modern era, that is paired with API design patterns that maximize flexibility about these moments.

Bit Shifter Priorities #

Web Standards are voluntary. The force that most powerfully compels their adoption is competition, rather than regulation. In practice, this means that vendors of integrated browsers^[1] have final say over what their browsers do. Those who ship the bits, rather than those who write the spec text, are ultimately responsible for the properties of the subsequent experience. This is the basis for functioning browser competition.^[2] Those who shift the bits call the shots.

This is an inherent property of modern browsers. Vendors participate in standards processes not because they need anyone else to tell them what to do, and not because they are somehow subject to the dictates of standards bodies, but rather to learn from developers and find agreement with competitors in a problem space where compatibility returns outsized gains.

Web Standards can be understood as liberally licensed, meticulously ventilated documentation about the detailed workings of a system that implementers agree to potentially be wrong about together.

Normative restrictions in a well-built spec should only cover aspects of a design where benefits of consistency and compatibility reliably outweigh the costs to getting some detail wrong.^[3] That is, if an API is found to be wanting in some way, the specified bits of the design are limited to aspects vendors can take time to address through an iterative, deliberative process.

Of course, there are classes of errors that vendors are not willing to be wrong about for the time it takes to forge new consensus. Chief among them are security vulnerabilities. Pervasive, abusive behaviour is not far behind. There are dozens of instances over the decades of browsers unilaterally breaking spec'd behaviours in the interest of guarding users. UI prescriptions in specifications that intersect with these concerns will be blithely ignored by vendors looking to protect users and their own reputations. This flows directly from the frequently-cited priority of constituencies.

Normative text about properties that are neither developer exposed nor are likely to be upheld in the breech create long-term liabilities. Over time, predictable conformance breakage — either from incident response or iterative exploration toward better solutions — has the effect of putting asterisks over otherwise-implementable specifications. This creates credibility problems for specs, their editors, working groups that back them, reviewers who do not flag unenforceable text, and SDOs that ratify toothless language. In a world of voluntary standards, reputation matters, and bad spec language can create a lasting negative impression.

Hitchen's Razor #

One of the strongest reasons to avoid making MUST (or even SHOULD)-strength statements about what browsers present is the vast depth of our collective ignorance.

Security UX is a relatively young field, and the set of organisations funding research into it is relatively small. We know from practice that understandability of systems is a major challenge and that users are poor at judging risk. This often causes those concerned about specific risks to demand prompts or user confirmation, but evidence also teaches that we risk "dialog blindness" through repeated re-confirmations.

The most successful examples of guiding users to safety in recent years involve a great deal of trial-and-error and follow-up research. Our best tools, collectively, are studies of variations and watching behaviour at scale. There are no models that I'm aware of that consistently maximize user understanding of choice over time while reducing decision burdens.

Consequently, as API designers, we owe it to our UXR peers to maximize their ability to iterate on potential presentations in service of those dual goals. Failing to acknowledge our own ignorance is a surefire way to paint ourselves into a bad corner.

If the case for a specific UI treatment on the basis of reckons is suspect, the case for consistency between browsers is doubly so. There is some evidence that a not-inconsiderable subset of desktop users may encounter multiple browsers in a week, but it is not a majority. Consistency between them should not then be a more important goal than internal consistency within a single browser's UI surfaces and/or consistency with underlying OS affordances (which also differ). Unlike the APIs that might invoke them, these UIs are not developer defined or exposed, removing the usual argument for consistency in a web standard.

Frustratingly, intense demands are often paired with generalized statements of the form "I think..." and "users don't know X...", without associated research. These reckons can take up a great deal of time when they should fail-fast against Hitchen's Razor, particularly given the browser community's history of iteration on these surfaces. Should future proposals for normative browser security surfaces come with peer-reviewed, wide-scale UXR attached then, perhaps, it may be worth revisiting blanket push-back.

Until then, and given our vast lack of understanding, it seems prudent to design defensively while expanding each spec's Privacy and Security Considerations sections to exhaustively enumerate risks that UX researchers will be asked to navigate. Designing for maximum iteration potential in this area has been, thus far, the best way to be least-cumulatively-wrong.^[4]