Infrequently Noted

Alex Russell on browsers, standards, and the process of progress.

Misdirection

As the over-heated CSS vendor prefix debate rages, I can't help but note the mounting pile of logical fallacies and downright poor reasoning being deployed. Some context is in order.

Your Moment Of Zen

The backdrop to this debate is that CSS is absolutely the worst, least productive part of the web platform. Apps teams at Google are fond of citing the meme that "CSS is good for documents, but not for apps". I push back on this, noting the myriad ways in which CSS is abysmal for documents. That isn't to minimize the pain it causes when building apps, it's just that the common wisdom is that CSS surely must be fantastic for somebody else. Once we find that person, I'll be sure to let you know. In the mean time we should contemplate how very, very far behind the web platform is in making it delightful to build the sorts of things that are work-a-day in native environments.

But it's worse than simply being under-powered: CSS has the weakest escape hatches to imperative code and demands the most world-view contortion to understand its various layout modes and their interactions. Imagining a more congruent system isn't hard -- there are many in the literature, might I humbly suggest that now might be a good time to read Badros & Borning? -- and even CSS could be repaired were we able to iterate quickly enough. Things have been moving faster lately, but fast enough to catch up with the yawning gap in platform capabilities? We'll come back to the speed/scale of change later.

For now, consider that the debate (as captured by Nate Vaughn) is about the retrospective implications of the few things that have already gotten better for some set of developers in some situations. That this sort of meaningful progress (corners, gradients, animations, transitions, flexing boxes, etc.) is rare makes the debate all the more bone-chilling to me. We finally got a few of the long list of things we've been needing for a decade or more, and now because the future is unevenly distributed, we're about to blow up the process that enabled even that modicum of progress? How is it we're extrapolating causality about engine usage from this unevenness, anyhow? None of this is obvious to me. The credible possibility of losing prefixes as a way to launch-and-iterate is mind-exploding when you realize that the salient competition isn't other browsers, it's other platforms. Either the proponents of squatting other vendor's prefixes haven't thought this through very hard or they're bad strategists on behalf of the web as a platform...not to mention their own products. The analysis that we're being asked to accept rests on an entire series of poor arguments. Lets start with the...

Uncomfortable Assumptions

In an interview out yesterday with Eric Meyer, Tantek Çelik of Mozilla tried to present this debate as a question of barriers to the adoption of non-WebKit based browsers, specifically Firefox Mobile. Opera has made a similar case. What they ommit is that the only platforms where they can credibly ship such browsers are Android and S60 (a dead end). That's a large (and growing) chunk of the world's handsets -- good news for me, as I now work on Chrome for Android here in London -- but for whatever reason it appears that iOS users surf a whole lot more.

Let that sink in: on the devices that are the source of most mobile web traffic today, it's not even possible to install a browser based on a different engine, at least not without a proxy architecture like the one used in the excellent Opera Mini or Amazon's Silk. iOS and Windows Phone are both locked-down platforms that come with only one choice of engine (if not browser shell). When folks from the vendors who want to appropriate others' prefixes talk about "not being able to compete", remember that competition isn't even an option for the most active users of mobile browsers. And it's prefixes that are keeping us down? We must go deeper.

The tension comes into focus when we talk in terms of conversion, retention, and attrition. Conversions are users who, if able, switch to a new product from an old one. Retention is a measure of how many of those users continue to use the product after some period of time. Today (and since Windows first included a browser), the single largest factor in the conversion of users to new browsers is distribution with an OS. This is the entire rationale behind the EU's browser choice UI, mandated on first start of new Windows installs. Attrition is the rate at which users stop choosing to use your product day after day, and for most desktop installed software, attrition is shockingly high after 3 to 6 months. The attrition rate is usually measured by cohorts over time; users who installed on the same day/week/month to measure what % of that group continue to use the product over increasingly long periods of time. The rate of decay falls, but the overall attrition invariably continues to rise. You might not get un-installed, but that doesn't mean you'll still be the go-to icon on the user's home screen or desktop. Eventually every device is recycled, wiped, or left for history in a drafty warehouse and along with it, your software. A key factor in getting attrition under control for Chrome has been evangelism to improve site compatibility, e.g. "I'm not using your browser because my bank website doesn't work with it". That argument -- that site compatibility is key to ensuring a competitive landscape for what otherwise are substitutes -- puts the entire thing in some perspective. Attrition isn't the same thing as conversion, and conversion is driven primarily by integrations and advertising. Implicit in the arguments by Tantek and others is a sub-argument of the form:

Our product would have measurably more users if sites were more compatible.

Thanks to what we know about what drives conversions, in the short run this is simply false. Long term, what invariably gives you more users is starting with more users. The set of things that are most effective at convincing users to swap browsers, even for a day, include: advertising, word-of-mouth, a superior product, distribution/bundling, and developer advocacy. Depressingly, only one of those involves actually being a better product, and the prerequisite for all of them is the ability to switch (thanks, Android team!). There's a similar dynamic at work when doing advocacy to web developers: if you're nowhere in their browser stats, they're adding support for a standard or worse a second prefix in order to do service to a cause, not because it's measurably good for them. Clearly, that's going to be somewhat less effective. Where, then, is the multi-million dollar advertising campaign for Fennec? The carrier and OEM rev-share deals for bundling on new Android phones? Hmm. To hear Tantek et. al. tell it, non-WebKit-based browsers would be prevalent on mobile if only it weren't for those damned browser prefixes causing users of other browsers to receive different experiences! Also, those kids and that damned dog.

Over the long haul compatibility can have a dramatic effect on the rate of attrition by changing the slope of the curve -- which, remember, is a rate with decay and not a single % -- but it begs the next uncomfortable question: what do we mean by "compatibility" here? What sorts of incompatibility cause attrition? Is it content that looks slightly worse but still essentially works (think grey PNG backgrounds on IE6) or does it simply turn you away, not allow you to play in any way, and generally fails (think the ActiveX dependencies of yore)?

Inaccessible or Ugly?

Eric was good enough to call out what I view as a key point in this debate: what sort of "broken" are we talking about? Tantek responded with a link to side-by-side screenshots of various properties rendered on Chrome for Android's Beta and current Fennec. In some of these cases we may be looking at Fennec bugs. Wordpress.com serves the same content to Fennec which seems to bork what float: left; means. That, or some media query is preventing the main blocks from being floated; it's hard to tell which from a quick view-source:. For the versions of google.* included in the list, the front end is simply serving the desktop version to Fennec which makes the wonky button rendering even stranger. Is there room to improve what gets sent to Fennec? You bet, but that's not what's being argued in the main. Ask yourself this: is what you see on that page worth destroying the prefix system for? 'Cause that's what the advocates of prefix-squatting would have you believe. In effect, they're suggesting that nothing will cause developers to test on non-pervasive engines, a deeply fascinating assertion. Even if we accept it, it doesn't point clearly to a single tactic to resolve the tension. It certainly doesn't argue most strongly for prefix-squatting.

An important point Eric failed to follow up on was Tantek's assertion that Mozilla will be cloaking user-agent strings. Does he imagine that the only thing that might be cause someone to send different content is CSS support? API support for things like touch events differs, the performance characteristics of device classes and browsers vary wildly, and application developers are keen to deliver known-good, focused experiences. The endless saga of position: fixed; as plumbed by Google teams and others is a story of competing constraints: browser vendors optimize for content, content fights back. Repeat. What does Mozilla imagine is going to happen here? Maintained content will react to the browser usage of end-users (and as we've covered, compat != conversions). Unmaintained content, well, that's what fallback is all about. And bad things deserve to lose. Assuming that your browser is 100% compatible with developer expectations and testing if you only switch the UA and implement a few prefixes is NPAPI-level crazy all over again, and it's entirely avoidable. Tantek and Brendan, of all people, should be able to reason that far. I guess they'll find out soon enough -- although we will have all been made poorer for it.

Now, what about the related argument that Mozilla & Co. are only going to be doing this for properties which are "stable" (nevermind their actual standardization status)? The argument says that because something hasn't changed in another vendor's prefixed version in a while, it must be safe to squat on. Not only is this (again) incredibly short-sighted, it says that instead of forcing a standard over the line and clobbering both developers and other vendors with the dreaded label of "proprietary" (the usual and effective tactic in this situation), they're instead willing to claim ownership and therefore blame for the spread of this soon-to-be-proprietary stuff, all the while punting on having an independent opinion about how the features should be structured and giving up on the standards process...and all for what?

Product vs. Platform

Perhaps there wasn't space in Tantek's interview with Eric, but both of them chose not to be introspective about the causes of WebKits use in so many mobile browsers, with Tantek merely flagging the use of a single engine by multiple products as "a warning sign." But a warning of what, exactly? Eric didn't challenge him on this point, but I sorely wish he had. Why did Safari, the Android Browser, Chrome, Silk, Black Berry, and many others all pick WebKit as the basis for their mobile browsers?

WebKit isn't a browser. It's just not. To make a browser based on WebKit, one might bring along at least the following bits of infrastructure which WebKit treats as bits to be plugged in:

And that's a minimum. Competitive ports tend to include WebSQL, LocalStorage, and Indexed DB back-ends, video codecs, 3D infrastructure (deeply non-trivial), perhaps an alternative JavaScript engine (V8 or other), and alternative/additional image decoders (e.g., WebP). All of that is in addition to needing to create your own UI for bookmarking, navigation, etc. WebKit is an engine, not a fully-functioning vehicle. Therein may lay some of the difference in the relative success of the WebKit and Gecko on mobile to date. Want to build a Gecko-based browser? Great, first clone the entire Firefox codebase from Mercurial, then layer your stuff on top/around. Oh, wait, things might not cleanly be factored to allow you to plug in your own X, Y, or Z? Your builds take forever? Welcome to life in the Mozilla universe where your product is always second fiddle to Firefox. Now, that's not by way of criticism, mind you. It is entirely reasonable for a product like Firefox not to pay coordination costs with other vendors/users of their code. God knows the overhead over here in WebKit land of trying to keep the menagerie of ports building against all changes is downright daunting. Mozilla (the organization) has made choices that prioritized the success of their product (Firefox) over their codebase (Gecko). WebKit was structured as platform only (no product), both forcing enormous costs onto every port while also freeing them from swimming upstream against a single product's imperatives in the codebase.

What we're witnessing isn't open vs. closed, it's differences in initial cost of adoption. In JS terms, it's jQuery (focused core, plugins for everything else) vs. Sencha or Dojo (kitchen sink). Entirely different target users, and both will find their place. Nobody should be shocked to see smaller, more focused bits of code with good plugin characteristics spreading as the basis for new projects. The Mozilla Foundation wants to help prevent monoculture? In addition to making the Firefox product a success, there are concrete engineering things they can do to make Gecko more attractive to the next embedder, Firefox-branded or not. I haven't heard of progress or prioritization along those lines, but I'm out of the loop. Perhaps such an effort is underway, if so, I applaud it. Whatever the future for Gecko, Product success isn't related to platform success as a first approximation. Having a good, portable, pluggable core increases the odds of success in evolutionary terms, but it's absolutely not determinant; see MSIE.

Speaking of IE...I respect those guys a lot, but the logical leap they're asking us to swallow is that the reason people return Windows Mobile phones is that some CSS doesn't work. That's what attrition means on a platform where they're the only native runtime. Data would change my mind, but it's a hell of a lot to accept without proof.

The Time Component

Lets take a step back and consider Tantek's claim that Mozilla has gotten very little traction in evangelizing multi-prefix or prefix-free development for the past year: Firefox for Android has been available since Oct. 2010 and stable for just 6 months. Opera Mobile on Android has been stable for just over a year. IE 9 (the only IE for mobile you could ever seriously consider not serving fallback experiences to) only appeared with Windows Phone 7.5 (aka "Mango"), shipped to users an entire 6 months ago.

And we expect webdevs to have updated all their (maintained) content? Never mind the tenuous correlation between the sorts of soft incompatibilities we're seeing at the hands of CSS and user attrition; the argument that even this lesser form of harm hasn't been blunted by evangelism appears suspect. Taking the incompatibilities seriously, I can quickly think of several other measures which are preferable to destroying the positive incentives towards standardization the prefix system creates (from least to most drastic):

All of these are less blow-up-the-world than what MSFT, Mozilla, and Opera are proposing. It's not even an exhaustive list. I'm sure you can think of more. Why these have either been not considered or dismissed remains a mystery.

It's More Complicated Than That

In all of this, we're faced with an existential question: what right do web developers have to shoot themselves in the foot? Is there a limit to that right? What sort of damage do we allow when some aspect of the system fails or falls out of kilter for some period of time? It's a question with interesting parallels in other walks of life (for a flavor, substitute "web developers" with "banks").

Can we show active harm to other browsers from the use of prefixes? The data is at best unclear. Arguing that any harm rises to a level that would justifies destroying the prefixes system entirely is rash. I argued many of the reasons for this in my last post, but lets assume in our mental model that developers respond to incentives in some measure. If, concurrently with achieving as-yet un-managed distribution, Mozilla et. al. implement others' prefixes, what should we expect developers to do in response? After all, they will have reduced whatever tension might have been created by content that "looked wonky" and, where standards exist, will have reduced the incentive to switch to the standard version.

Now lets play the model one more turn of the wheel forward too: assume that Chrome or Safari (or both!) act in good faith and contemplate removing the -webkit-* prefix support for standardized features at a quick clip...and Mozilla doesn't. You see how this quickly leads to a Mexican standoff: web developers won't stop using prefixed versions because those are the way you get 100% support (thanks to Mozilla's support for them); vendors won't un-prefix things because others who squat their prefixes will then have a compatibility advantage; and nobody will be keen to add new things behind prefixes because they can no longer be assumed to be experiments that can change. Lose, lose, lose.

Some on the other side of the debate are keen to cite game theory as a support for their course of action, but the only conclusion I can draw is that their analysis must be predicated on a set of user and developer motivations that are entirely unlike the observable world we inhabit.

A Call To Reason, Not Arms

Based on a better understanding of the landscape, what should the various parties do to make the world better for themselves now and in the long run and for the web as a platform?

Endnotes

I've left a lot out of this post, but it's too long already. I do truly hope it's the last I write on prefixes because, as I said up front, we have much bigger fish to fry. Stat. Prefixes do work, they're essential to delivering a future platform that can compete, and yes, they should be torn down at the same pace they're erected.

A few things that folks have asked about as tangents to this debate:

Update: Michael Mullany of Sencha adds epicly good, epicly long context about what causes developers to target UAs and what the incentives that'll change their minds about supporting a given browser really are.

Thanks to Frances Berriman, Jake Archibald, Tony Gentilcore, and Tab Atkins for reviewing earlier versions of this post. Errors of fact and form are all mine, however. Updated occasionally for clarity and to remove typos.

Vendor Prefixes Are A Rousing Success

tl;dr version: Henri Sivonen's arguments against vendor prefixing for CSS properties focus on harm without considering value, which in turn has caused him to come to a non-sensical set of conclusions and recommendations. Progress is a process, and vendor prefixes have been critical in accelerating that process for CSS.

For a while now I've been hearing the meme resurface from CSS standards folks and a few implementers that "vendor prefixes have failed". I'd assumed this was either a (bad) joke or that it was one of those things that web developers would scoff at loudly enough to turn the meme recessive. I was wrong.

Henri Sivonen, Mozilla hacker extrordinare, has made the case directly and at length. Daniel Glazman, co-chair of the CSS WG posted a point-by-point response. If you have the patience, you should read both.

Lost in the debate between "browser people" and "spec people" is the the essential nature of what has happened with prefixes: they worked. From the perspective of a web developer, any first approximation of the history of vendor prefixes are pure win, even if only a fraction of the value that has been delivered behind them is attributable to prefixes un-blocking vendors from taking risks and shipping early.

Daniel's rebuttal to Henri gets a lot of things right, but he gives in on an essential point; by agreeing with Henri that vendor prefixes are "hurting web authors" he wites off the benefits that they've delivered -- namely the ability of vendors to get things out to devs in a provisional way that has good fallback and future-proofing properties and the ability for devs to build with/for the future in an opt-in, degradable way.

Rounded corners, gradients, animations, flex box, etc. are all design and experience enablers that developers have been able to take advantage of while waiting for the standards dust to settle, and thanks to W3C process, it takes a LONG time to to settle. Yes, that has some costs associated with it. Henri is very worried that browsers that aren't keeping up quickly will be "left behind" by webdevs who use only one vendor's prefix. But surely that's a lesser harm than not getting new features and not having the ability to iterate. And it provides incentive for following browsers to try to make a standard happen. What's not to love? More to the point, I just don't believe that this is a serious problem in practice. What front-ender in 2011 doesn't test on at least two browsers? Yes, yes, i'm sure such a retrograde creature exists, but they were going to be making non-portable content regardless of prefixes. Assuming you're testing fallback at all (e.g., by testing on more than one browser), prefixes not appearing on some browser are just the fallback case. CSS FTW! Webdevs who don't test on more than one browser...well, they're the ones hanging the noose around the neck of their own apps. Vendor prefixes no more enable this stupidity than the existence of the User-Agent header. Compatibility is a joint responsibility and the best each side (browser, webdev) can hope of the other is some respect and some competence. Cherry picking egregious examples and claiming "it's hurting the web" seems, at a minimum, premature.

And how did we think we'd get a good standard, anyway? By sitting in a room in a conference center more often and thinking about it harder? Waiting on a handfull of early adopters to try something out in a tech demo and never stress it in practice? That's not a market test (see: XHTML2), it doesn't expose developers to the opportunities and tradeoffs that come with a new feature, and doesn't do anything to address the inevitable need to integrate feedback at some point.

Yes, we could go with Henri's suggestion that the first person to ship wins by default, never iterate on any designs, and avoid any/all first-mover disadvantage situations, but who among the browser vendors is perfect? And what would the predictable consequences be? I can only assume that Henri thinks that we'll end up in a situation where vendors coordinate with the CSS WG early to add new stuff, will design things more-or-less in the open, and will only ship features to stable (no flag) when they're sure of their design. That could happen at the limit, but I doubt it. Instead, the already fraught process of adding new features to the platform will be attempted by even fewer engineers. Who wants the responsibility for having to be perfect lest you screw the web over entirely? Fuck that noise, I'm gonna go work on a new database back-end or tune something to go faster. Browsers are made by smart people who have a choice of things to be working on, and any time you see a new platform feature, it probably came about as the result of an engineer taking a risk. Many times the engineers in a position to take those risks don't have a great sense for what good, idiomatic web platform features might be designed, so they'll need to tweak/iterate based on feedback. And feedback is painfully hard to extract from webdevs unless you've made something available in a tangible way such that they can use it and discover the limitations. Shipping things only to dev is perhaps a good idea for other aspects of the platform where we can't count on CSS's forgiving parsing behavior (the basis for prefixes). Syntax changes for JS and CSS seem like good examples. But for features that are primarily new CSS properties? Oy. Making the stakes even higher, reducing the ability to get feedback and iterate isn't going to lead to a harmonious world of good, fast standards creation. It's going to predictably reduce the amount of new awesome that shows up in the platform.

Prefixes are an enabler in allowing the necessary process of use, iteration, and consensus building to take place. Want fewer messes? There's an easy way to achieve that: try less stuff, add fewer features, and make each one more risky to add. That's Henri's prescription, wether or not he knows it, and the predictable result is a lower rate of progress -- advocating this sort of thing is much worse for the web and for developers than any of the harm that either Henri or Daniel perceive.

Which brings me to Henri's undifferentiated view of harm. His post doesn't acknowledge the good being done by prefixed implementations -- I get the sense he doesn't build apps with this stuff or it'd be obvious how valuable prefixed implementations are for work-a-day web app building -- instead focusing on how various aspects of the process of prefixed competition can be negative. So what? Everything worth having costs something. Saying that things "hurt the web" or "hurt web developers" without talking in terms of relative harm is just throwing up a rhetorical smoke screen to hide behind. If you focus only on the costs but write the benefits out of the story of course the conclusion will be negative. In many cases, the costs that Henri points out are correctly aligned with getting to a better world: having to type things out many times sucks, creating demand among webdevs for there to be a single, standardized winner. Having multiple implementations in your engine sucks, creating demand from vendors to settle the question and get the standards-based solution out to users quickly. Those are good incentives, driven by prices that are low but add up over time in ways that encourage a good outcome: a single standard implemented widely.

And as Sylvain Galineau pointed out, what looks like pure cost to one party might be huge value to another. I think there's a lot of that going on here, and we shouldn't let it go un-contextualized. The things that Henri sees as down-sides are the predictable, relatively minor, costs inherent in a process that allows us to make progress faster and distribute the benefits quickly, all while minimizing the harm. That he's not paying the price for not having features available to build with doesn't mean those opportunity costs aren't real and aren't being borne by webdevs every day. Being able to kill table and image based hacks for rounded corners is providing HUGE value, well ahead of the spec. Same for gradients, transitions, and all the rest. Calling prefixed implementations in the wild a bad thing needs to argue that the harm is greater than all of that value. I don't think Henri could make that case, nor has he tried.

I think the thing that most shocks me about Henri's point of view is that he's arguing against a process when in fact the motivating examples (transforms, gradients) have been sub-optimal in exactly the better-than-before ways we might have hoped for! Gradients, for example, saw a lot of changes and browsers had different ideas about what the syntax should be. Yes, it's harder to get a consistent result when you're trying to triangulate multiple competing syntaxes, but we got to use this stuff, get our hands dirty, and get most of the benefits of the feature while the dust settled. Huzzah! This is exactly> the way a functioning market figures out what's good! Prefixes help developers understand that stuff can and will change, and they clear the way for competition of ideas without burdening the eventual feature's users with legacy bagage tied to a single identifier.

So what about the argument that there might be content that doesn't (quickly?) adopt the non-prefixed version, or that vendors can't remove their prefixed implementations because content depends on it?

To the first, I say: show me a world where 90+% of users have browsers support the standard feature and I'll show you a world in which nobody (functionally) continues to include prefixes. That process is gated in part by the WG's ability to agree to a spec, and here I think there's real opportunity for the CSS WG to go faster. The glacial pace of CSS WG in getting things to a final, ratified spec is in part due to amazingly drawn-out W3C process, and in part a cultural decision on the part of the WG members to go slow. My view is that they should be questioning both of these and working to change them, not blaming prefixes for whatever messes are created in the interim.

As for removing prefixes, this is about vendors just doing it, and quickly. But the definition of "quickly" matters here. My view is that vendors should be given at least as long as it took to get a standard finalized from the introduction of their prefixed version for the removal process to be complete. So if Opera adds an amazing feature behind a -o- prefix in early 2012 and the standard is finalized in 2014, the deprecation and eventual removal should be expected to take 2 years (2016). This has the nice symmetry of incentives that punish the WG for going slow (want to kill prefix impls? get the standard done) while allowing the vendors who took the biggest risks to provide the softest landings for their users. And it doesn't require that we simply go all-in on the first person's design to ship. Yes, there will be mounting pressure to get something done, but that's good too!

The standards process needs to lag implementations, which means that we need spaces for implementations to lead in. CSS vendor prefixes are one of the few shining examples of this working in practice. It's short-term thinking in the extreme to either flag the costs associated with them as either justifying their removal or even suggesting that the costs are too high.

And webdevs, always be skeptical when someone working on an implementation or a spec tells you that something is "hurting the web" when your experience tells you otherwise. The process of progress needs more ways to effectively gauge webdev interest, collect feedback, and test ideas. Not fewer or narrower channels.

Function-ality

I'm sitting here in Derek Featherstone's amazing a11y talk at Fronteers and I feel like I need to follow up the last post with a quick primer on the zen of function for (both of) the spec authors who read this blog.

The reason it's offensive to the JS hacker for WebIDL to disallow new against DOM types -- any of them -- is that it means that it's no longer specifying how you'd describe these types in the primitive we use over here in JS for this sort of thing: functions. This might sound nuts to folks who come from C++ or who spend their time in spec-ese, but there's no difference between a plain-old function, a "constructor", and a "partial" or "mixin" in JS semantics. We use functions for all of them. You can say new function(){ ... } and create an instance of an anonymous "class" in JS. You can take the same function and invoke it as a "regular function" -- (function(){ ... })(); -- and you can use that sort of function as a mixin or partial interface too: new function(){ (function(){ ... }).call(this); ... }. The exact same function object can even act in all of these capacities (although it's rare). People use them as they need to, but they all boil down to functions.

What, then, does it mean for something to disallow new against some type for which you can in some otherwise get an instance in JS? The same thing when you can't .call() it: it's alien. It's not a class as we know it, which means that it's not a function, and if it's not a function...well, it doesn't belong. Fundamentally, it's smuggling static semantics into a language that has perfectly good dynamic semantics for the same thing. This strikes at the very heart of what WebIDL is supposed to be for: describing JS types for things implemented somewhere else. By not allowing new and .call() WebIDL is giving JS semantics the bird, asserting that the fact that these things aren't JS types makes them better in some way...and that is a bug, either in the perspective of the spec authors or of the specs themselves.

Luckily, the fix is easy: all WebIDL types should de-sugar to functions. All of them. All the time. No questions asked. That you will be able to use new and .call() and all the rest isn't a bug, and it's not something to guard against. It's just how JavaScript rolls...and how JavaScript's largest, most important library should roll too.

Real Constructors & WebIDL Last Call

For those who haven't been following the progress of WebIDL -- and really, how could you not? An IDL? For the web? I'd like to subscribe to your newsletter... -- the standard is now in last call, which is W3C for "alllllllllllmost done".

Which it is not.

Before I get to why, let me first say some nice, well-earned things about WebIDL: first, it has helped us out of the ad-hoc IDL sludge that used to be how APIs for JavaScript have been exposed in the past. It has shaved off many sharp edges and is giving spec authors a single dialect in which to write their API descriptions. From a browser perspective, this is a Very Good Thing (TM). Next, the draft in question contains some wonderful changes from the status quo, particularly the addition of a sane prototype to all WebIDL-specified objects.

That all sounds good, so what's missing?

In a word, constructors.

Well, a lot more than that, but I'd settle for constructors. Functionally speaking, it boils down to the fact that WebIDL makes spec authors do extra work to make something like this sane:

new HTMLDivElement();

Why doesn't this work today? Funny story...see, HTML defines HTMLDivElement as a regular WebIDL interface. WebIDL doesn't really have the notion of concrete classes, just interfaces with and without constructors. Since the HTML spec is just doing what most specs will do -- adding the smallest IDL you can get away with -- the JS usability of this API is left in limbo; neither clearly HTML5's responsibility nor WebIDL's.

So what should a contentious version of HTML5 do? One answer is to specify a constructor, turning the IDL from this:

interface HTMLDivElement : HTMLElement {};

to this:

[Constructor]
interface HTMLDivElement : HTMLElement {};

Repeat ad-infinitum for each and every interface that should be constructable in every single spec that browser vendors ever implement. Don't miss any! And please make sure that all your spec editors are on-board with good JS APIs as a goal! As it stands today, WebIDL doesn't even force most spec authors to consider the question "do I need a constructor here?" -- spoiler: yes -- let alone the obvious follow-ups like "what arguments should one take?".

The obvious better answer here is to flip the default on interfaces, causing them to generate constructors by default unless turned off with [NoConstructor] attributes or specified as partial interfaces (i.e., mixins or traits).

Cameron McCormack who is heading up the WebIDL effort tweeted in response to my exasperation that:

I think a "W3C Web API design guidelines" document would be a perfect place for such a recommendation.

For serious? Such a document might be useful (and I'm working on something that might pass as a first draft), but what's the argument against flipping the default here? This isn't a dissent on the facts of the situation: most WebIDL "interfaces" that are exposed to JS are things that could be easily new'd up to useful ends. Most specs flub this in spectacular style. Most spec authors seem entirely ignorant of the problem and the design language of WebIDL continues to lead down a primrose copy-and-paste path that has little overlap with sanity. So why punt the decision? And why did it take and act of coordination with TC39 to get the prototype thing fixed?

And Why Are We Having This Discussion Anyway?

WebIDL, for all of its virtues, is deeply confused.

If you're reading any of the stuff in the HTML5 spec that's describing its API this way, it's hard to see how it would have any sane relationship to JavaScript. Sure, you could argue that there might be other languages that matter, other languages for which you'd need to be able to generate some API, but none of them rise to anything like the importance of JavaScript. It is the programming language of the web, so if WebIDL has any animating force at all, it's JS. Then there's the "accident of history" aspect. Early DOM was specified as a form of IDL in part because there was some expectation that other languages would need to consume it and IDL was how C++ hackers (who still make up the entire set of people working on browser engines) are/were comfortable in describing their FFIs thanks to the legacy of COM/CORBA. Hilarious examples of multi-language-ism still persist in the WebIDL spec for no apparent reason whatsoever, warping the entire design around the altar of an ideal that is either quixotic or vestigial depending on which argument you give more weight.

Since the debate was re-kindled thanks to a debate at a TC39 meeting in July, I've been on the receiving end of more than one webdev's rant about DOM's JS incoherence, generally taking the form:

Why the *#!*?^@$ isn't DOM just #@!*@ing specified in JavaScript?

To be honest, I have no answer aside from pointing to the IDL history, the fact that browser hackers don't tend to write JS so don't feel the pain, and noting that WebIDL is better in some important ways. Certainly these interfaces could be specified in a subset of JS with annotations for where native behavior is required. But their larger lament has merit too: seamlessness with JS is the bar WebIDL should be judged by. I.e. does it help spec authors do the right thing by JS devs? Or does it lead them down paths that make their generated APIs stick out like sore thumbs, full of anti-social/alien behavior such that you can't think of them as "regular" JS?

Yes, constructors are only one minor step toward reaching this aspiration, but the fact that WebIDL has gotten to last-call without a reasonable solution to them speak volumes. If WebIDL isn't animated by the need of JS developers, it would be good if that could be loudly called out somewhere so that the community can have the spirited debate that this point demands. If it is, can we please get on discussing how best to ensure that most "interfaces" generate constructors and stop punting?

Either way, WebIDL isn't done yet.

Update: It occurred to me, as part of the discussion in the comments, that the provision against new with a class or type of any type is completely non-sensical in JS, as is the lack of call() and apply() methods on them. Idiomatic subclassing requires that the mixin-style be available, which uses ClassName.call(this). This is what you'd do with things that are "virtual" or "partial" interfaces if you're describing them in actual JS. And there's no real problem with folks new-ing them up. Doesn't happen, doesn't matter. Strong restrictions against it are, to quote Andrew DuPont, anti-social. Long story short: there is absolutely no reason whatsoever to disable construction on any WebIDL interface. It's deeply non-sensical from the JavaScript perspective.

Standards Are Insurance

I keep getting distracted from writing this long thing by responding to the discussion created by the last short-ish thing, but I wanted to explicitly call out one aspect, namely that standards are a form of insurance.

More correctly -- and apologies if this sounds like a Planet Money episode -- vendors sell derivatives contracts (insurance) against the proprietary nature of technologies. I.e., as a hedge against winner-take-all dynamics of network effects and the potential for monopoly rent extraction. Adopters of that technology often refuse to buy it at all without the availability of a derivative to cover their risk of lock-in.

The nice bit about these contracts is that the results (price) are free to the public -- ITU and other old-skool orgs are exceptions -- meaning anyone who wants to implement can assess what it'll cost, and if they can provide a solution at a lower to-end-user price, they can compete under the same terms. The public nature of the derivative contract can have the effect of lowering the price of the good itself in markets that clear and have many participants.

Standards organizations are correctly understood as middle-men or market-makers for these contracts.

Update: It may seem strange to some that derivatives contracts (insurance) can drive prices down, but it happens in many industries. E.g., satellite insurance has done wonders for the commercial launch vehicle industry. And who would fund the construction of new cargo ships if you couldn't convince enough people to ship their precious goods? Insurance matters.

Older Posts

Newer Posts