One of the things I've poured myself into this year -- with a merry band of contributors including Domenic Denicola, Anne van Kesteren, Jake Archibald, Mark Miller, Erik Arvidsson, and many others -- has been a design for Promises that DOM and JS can both adopt.
There's a (very) long history of Promises, Deferreds, and various other Promise-ish things in JS which I won't bore you with here except to note that there are very few client-side libraries which don't include such a thing and use it as a core idiom for dealing with async behvaior (e.g., XHR). jQuery, Dojo, Q, WinJS, Cujo, Closure, YUI, Ember (via RSVP), and all the rest use this style of contract pervasively and have for years. In fact, it's so common that Domenic Denicola and others have gone as far to rustle up a community standard for how they should interop under the banner of Promises/A+. The major libraries are coalescing around that contract and so it seems time, finally, to make our biggest and most important library -- DOM -- savvy to them too.
The recent history starts (arbitrarily) a couple of years ago and ends 2 weeks ago. In that time, a single design has evolved that not only DOM could get behind, but also which TC39 has agreed in principle to endorse and support going forward, thanks in large part to Mark Miller's analysis of the competing styles of use which proves strongly that the A+-compatible API we've designed need not upset anybody's applecart.
The TC39 meeting was a key turning point: up until 2 weeks ago, DOM had a version of this design under the name
Future. I made the decision to not use the name
Promise for that work because without TC39's agreement on a design, the DOM variant could at some point find itself both camping on a global name and disagreeing with JS about semantics or naming of particular APIs. That sort of thing might have lead to the suspicion by DOM folks that TC39 was out of touch and slow, and by TC39 that DOM rushed in like fools into a space that's pretty clearly something the std lib should include (even if it couldn't do so for multiple years due to publication and spec timing issues).
Meanwhile, in the background, several DOM APIs have started to adopt Futu...er...Promises, notably Web Crypto and Web MIDI. There has also been lively discussion about other APIs that can benefit from moving to a standard mechanism for describing async operations.
It seems, in each individual case, like this shouldn't be such a big deal. Some APIs have callbacks, some use events...what's the fuss?
The big reason to spend months of my life on this problem, and to harass other very busy people to do the same, is to me the core value of web standards: when they're working well, they create a uniform surface area that describes a coherent platform. We are the beneficiaries of this uniformity today regarding events, and they are a major piece of the design language which DOM API authors can reliably use to help describe bits of their design. Promises, like Events, are yet another tool in the box that DOM APIs authors can use, and thanks to sane constructors and the ability to subclass built into the design, it's possible for end-user code to eventually put down the custom implementations of Promise-like things and simply rely on the platform to do what platforms should do: make cheap and easy what previously was common but expensive.
As of this week, the WHATWG DOM spec has changed its naming to reflect the consensus design, substituting
fulfill(), and removing a few of the methods that didn't have consensus or were agreed to be unnecessary in a 1.0.
Thanks to this broad consensus over the design, both Mozilla and Google have begun to implement Promises in our respective engines. Further, the W3C TAG agreed at last week's meeting to recommend to spec authors that they adopt Promises for asynchronous, single-valued operations. This is also great news because the TAG has gone from being a body which is largely reactive to one that has begun to become pro-active, taking a more active role in API oversight and integration across Working Groups to help ensure the coherence of the overall platform's architecture and design.
The job of moving many of today's APIs which use ad-hoc callback systems or vend Promise-like-but-not-quite objects is far from over, much the way providing constructors for DOM operations is a work in progress...but I have many reasons to hope, not least of all because folks like Mark, Tab, Domenic, Yehuda, and Anne are working together in good faith to help make it so.
This, then, is how we can collectively add new primitive types to the platform: begin with community and evidence, build consensus around a design slowly (starting with the key stakeholders), and eventually work across the entire platform to integrate these primitives pervasively.
It takes people who are willing to put down an us-vs-them perspective and collaborate honestly and openly to make it happen, but moving the web forward always does. Promises are budding proof that such a thing isn't beyond our institutions, vendors, and platform leaders to do. Collaboration across the spectrum from users, to spec organizations, to vendors can happen. The platform can be reformed and rationalized, and even the most recalcitrant of DOM spec authors are willing to listen when presented with evidence that their APIs aren't idiomatic or could be improved to help make the commons better, even when it comes at some risk and cost to their own APIs.
A reminder: I approve comments manually.
I don't mind vulgarity, wrongness, or even the mild ad-homenim...as long as it contributes to the discussion. Should you post a comment here that does not contribute, expect it to be binned or treated as spam -- without warning. Be witty, be carefree, be data-driven, be on point, but most of all, write something that will try to convince someone else you're right. The elation of asserting something you feel is self-evident is a joy that falls into despair far too quickly and is too easily dashed by argument and evidence.
Some weeks back I lobbed an overly terse "noooooooooo!!!!!" at the W3C Web Application Security Working Group over revisions to the CSP 1.1 API; specifically a proposed reduction of the surface area to include only bits for which they could think of direct use-cases in a face-to-face meeting. At the time I didn't have the bandwidth to fully justify my objections. Better late than never.
For those who aren't following the minutiae of the CSP spec, it began life as a Mozilla-driven effort to enable page authors to control the runtime behavior of their documents via an HTTP header. It was brought to the W3C, polished, and shipped last year as CSP 1.0; a much better declarative spec than it started but without much in the way of API. This is a very good way for any spec to get off the ground. Having a high-level declarative form gives implementers something they can ship and prove interop with very quickly. The obvious next step is to add an API.
Late last year I started digging into CSP, both for a personal project to implement a "user CSP" extension for Chrome, and to work with the spec authors to see what state the proposed API was in and how it could be improved. The short form of my analysis of the original CSP proposal was that it was pretty good, but missed a few notes. The new proposal, however, is a reflection of the declarative machinery, not an explanation of that machine.
For example, the previously proposed API provided methods on a
SecurityPolicy class like
allowsConnectionTo(url) which outline an API that the browser might plausibly need to enforce a policy at runtime. The new API includes no such methods. As someone working with CSP, you suspect the browser indeed has such a method on just such an object, but the ability to use it yourself to compose new and useful behaviors is now entirely curtailed. This is the extreme version of the previous issues: an API that explains would make an attempt to show how the parser is invoked -- presumably as a string argument to a constructor for
SecurityPolicy. Similarly, showing how multiple policies combine to form a single effective policy would have lead away from
document.securityPolicy as something that appears to be a single de-serialized
SecurityPolicy and instead be written in terms of a list of
SecurityPolicy instances which might have static methods that are one-liners for the
.forEach(...) iteration that yeilds the aggregate answer.
So why should any WG bother with what I just described?
First, because they'll have to eventually, and by showing only enough skin to claim to have an API in this round, the question will be raised: how does one implement new rules without waiting for the spec to evolve? The Extend The Web Forward idea that flows naturally from p(r)ollyfills has shown real power which this new design puts further from reach...but it won't keep people from doing it. What about implementing something at runtime using other primitives like a Navigation Controller? Indeed, the spec might get easier to reason about if it considered itself a declarative layer on top of something like the Navigation Controller design for all of the aspects that interact with the network.
There are dozens of things that no over-worked spec author in a face-to-face will think to do with each of the platform primitives we create that are made either easier or harder for the amount of re-invention that's needed to augment each layer. Consider CSS vs. HTML's respective parsing and object models: both accept things they don't understand, but CSS throws that data away. HTML, by contrast, keeps that data around and reflects it in attributes, meaning that it has been possible for more than a decade to write behavioral extensions to HTML that don't require re-parsing documents, only looking at the results of parsing. CSS has resisted all such attempts at gradual runtime augmentation in part because of the practical difficulties in getting that parser-removed data back and it's a poorer system for it. CSP can either enable these sorts of rich extensions (with obvious caveats!) or it can assume its committee knows best. This ends predictably: with people on the committee re-implementing large bits of the algorithms over and over and over again for lack of extension points, only to try to play with new variations. This robs CSP and its hoped-for user-base of momentum.
document.securityPolicy object doesn't conceptually de-sugar to anything reasonable except a list of policy objects...but that more primitive
SecurityPolicy object type doesn't appear anywhere in the description. This means that if anyone wants to later extend or change the policy in a page, a new mechanism will need to be invented for showing how that happens: meta tags parsed via DOM, not objects created in script and added to a collection. All of which is objectionable on the basis that all that will happen is that some objects will be created and added to the collection that everyone suspects is back there anyway. This is like only having
There is, of course, another option: CSP 1.1 could punt on an API entirely. That's a coherent position that eliminates these frictions, and I'm not sure it's a bad option given how badly the API has been mangled recently. But it's not the best solution.
I've got huge hope for CSP; I think it's one of the best things to happen to webappsec ever. What happens about the API will always be overshadowed by the value that it is already delivering. But as a design microcosm, its API is a petri-dish sized version of scenario-solving vs. layering, and a great example of how layering can deliver value over time; particularly to people who aren't in the room when the design is being considered. An API that explains by showing how declarative layers on top of imperative is one that satisfies use case zero: show your work.
...but if it were, there would be time to cover the W3C Advisory Board election. This is truly inside-baseball stuff, as most of the AB's work happens in member-only areas of the W3C website and most of what they effect is W3C process.
So why care? Because spec licensing matters, sadly.
Let me first outline my view: in an ideal world, specification language for web standards would be in the public domain or dedicated to it. The body of intellectual property that is non-copyright is fought over via an independent process, and to the extent that it accumulates in a standards organization, it should also be possible for the group of members that have contributed to take their ball and start again somewhere else.
Why does this matter? Competition.
Standards bodies should not be insulated from the pressure to try to deliver better results. There are, of course, pathologies around this; some of which are common enough to have names: "pay for play", "venue shopping", etc. But in general, to the extent that many bodies can produce substitue goods, it gives them a reason to differentiate. The concrete example here is WHATWG vs. W3C. I don't think it's controversial to assert that without the WHATWG, the current W3C would be f'd. Competition makes everyone better, even for products that are "free" for consumers and are the product of community effort.
This, then, is why it's such a terrible idea for the W3C's Advisory Committee (the people who have some power) to elect representatives to the Advisory Board (who have even more power) that are willing to take the self-interested side of the W3C against liberal licensing of specs over the competition-enabling position that liberal licensing makes everyone better off (to a first approximation).
If you are an AC rep, the time is now to quiz candidates on this point. If you truly think that the W3C is a unique community, it's important to realize that what makes it that special is a set of shared values, not a death-grip on legacy intellectual property rights. Fixating on that ownership is the fast-path to making everyone worse off; by the time it truly becomes important, it's so likely that the organization that cares about it needs competition to right the ship or get out of the way.
Several times in the past few months I've been presented with this question: is it good or useful to provide synchronous APIs to web workers?
Having considered the question at some length, it seems to me the answer must now be "no".
Consider IndexedDB. It's not implemented in any browser yet, but some hard-working soul did the arduous work to specify a second synchronous version of its API which is meant to be available only in Workers where it can't lock up the UI thread. That spec work was probably done because it was thought that a synchronous API would be nicer to use than the async version. As a result, the API is now double the size, but only in some contexts. I came across this while attempting to rework the IDB API to use Futures in order to improve usability in a backwards-compatible way.
So why not the sync version? At least 2 reasons:
- The async version doesn't have to be thorny as the current IDB API is. The current IDB API has its issues: it uses events for one-time operations (not 0-through-N times operations). To do so it has to create non-obvious control flow contracts to give developers a place to set up listeners. Another legacy of using events is that delivered values are always wrapped in event objects and must be dug out with boilerplate. Some API conveniences can be added, and some things in the API are implicit that perhaps shouldn't be (e.g. transactions). All of these things can be repaired in a backwards-compatible way that improves async IDB to the point of being nearly as nice to use as sync IDB. Futures can help, as can some reworking of optional arguments.
- There's very little a worker will do that is really long running and truly synchronous. Even things that look like they're a good fit at first blush break down quickly. Just think of the user experience: who wants their laptop fan spinning up, their lap getting warm, and their precious battery indicator slipping towards a powerless coma without being able to either find what is causing it or, better, some UI that communicates what work is being done on their behalf and a way to cancel it if necessary. Getting work off the main thread is what allows apps to do work for users without locking up those UI indicators, but to provide any ability to interrupt the task or broadcast progress, it must yield to the main event loop.
It's this second concern that I think it truly fatal to the cause of sync worker APIs: assuming they work and are popular, they will create a world in which it's necessary to put limits on their overall running time...limits that will be circumvented by breaking up the work into smaller chunks and dealing with it asynchronously inside the worker. Likewise, anyone building a serious app that's trying to do the right thing by the user will factor their worker's tasks into small enough chunks that they can both service "stop" messages and distribute progress notifications to the UI. There might be scenarios where such messages aren't necessary and where users aren't coveting CPUs and batteries...where sending SIGHUP doesn't matter. But the intersection of those scenarios and the client-side web seems mostly to be a happy accident: your code might not have encountered enough data to create the problems. Yet.
This is particularly clear in the IDB cases: upgrading, iterating over, and updating hundreds of thousands of items of data is the sort of thing that will take a while, and is likely in response to some application semantic the user cares about: synchronizing mail, migrating to a faster schema layout in the process of some upgrade, etc. A blind
for loop is asking for trouble. All of this might work fine in a dev environment with a (small) staging set of data...but it's recipe for disaster when power-users with tons of data encounter it. What then? If the APIs an app depends on are all synchronous, it's a huge boulder to roll up a hill to provide notifications, chunk up work, and refactor around async-ish patterns that chunk work up. If the work was async in the first place, the burden is much lower. So even apps that aren't Doing It Right (TM) are likely to reap some benefit down the line from thinking in terms of async first.
There are other arguments that you can field against these sorts of APIs, particularly ones that double-up API surface area, but it doesn't seem to me that they're necessary. The person attempting to justify synchronous worker APIs who provides a good argument for ergonomics and learnability still has all their work ahead of them: they must show that these APIs are not harmful to the user experience. After all, Workers were added to the platform as a way of improving UX (by moving work off the main thread). And I fear they cannot do so without violating core JS semantics.
So let's pour one out for our sync API dreams: we're gonna miss you, control flow integration. But not for too long.
yield will see you avenged.