So why care? Because spec licensing matters, sadly.

Let me first outline my view: in an ideal world, specification language for web standards would be in the public domain or dedicated to it. The body of intellectual property that is non-copyright is fought over via an independent process, and to the extent that it accumulates in a standards organization, it should also be possible for the group of members that have contributed to take their ball and start again somewhere else.

Why does this matter? Competition.

Standards bodies should not be insulated from the pressure to try to deliver better results. There are, of course, pathologies around this; some of which are common enough to have names: “pay for play”, “venue shopping”, etc. But in general, to the extent that many bodies can produce substitue goods, it gives them a reason to differentiate. The concrete example here is WHATWG vs. W3C. I don’t think it’s controversial to assert that without the WHATWG, the current W3C would be f’d. Competition makes everyone better, even for products that are “free” for consumers and are the product of community effort.

This, then, is why it’s such a terrible idea for the W3C’s Advisory Committee (the people who have some power) to elect representatives to the Advisory Board (who have even more power) that are willing to take the self-interested side of the W3C against liberal licensing of specs over the competition-enabling position that liberal licensing makes everyone better off (to a first approximation).

If you are an AC rep, the time is now to quiz candidates on this point. If you truly think that the W3C is a unique community, it’s important to realize that what makes it that special is a set of shared values, not a death-grip on legacy intellectual property rights. Fixating on that ownership is the fast-path to making everyone worse off; by the time it truly becomes important, it’s so likely that the organization that cares about it needs competition to right the ship or get out of the way.

Having considered the question at some length, it seems to me the answer must now be “no“.

Consider IndexedDB. It’s not implemented in any browser yet, but some hard-working sould did the arduous work to specify a second synchronous version of its API which is meant to be available only in Workers where it can’t lock up the UI thread. That spec work was probably done because it was thought that a synchronous API would be nicer to use than the async version. As a result, the API is now double the size, but only in some contexts. I came across this while attempting to rework the IDB API to use Futures in order to improve usability in a backwards-compatible way.

So why not the sync version? At least 2 reasons:

• The async version doesn’t have to be thorny as the current IDB API is. The current IDB API has its issues: it uses events for one-time operations (not 0-through-N times operations). To do so it has to create non-obvious control flow contracts to give developers a place to set up listeners. Another legacy of using events is that delivered values are always wrapped in event objects and must be dug out with boilerplate. Some API conveniences can be added, and some things in the API are implicit that perhaps shouldn’t be (e.g. transactions). All of these things can be repaired in a backwards-compatible way that improves async IDB to the point of being nearly as nice to use as sync IDB. Futures can help, as can some reworking of optional arguments.
• There’s very little a worker will do that is really long running and truly synchronous. Even things that look like they’re a good fit at first blush break down quickly. Just think of the user experience: who wants their laptop fan spinning up, their lap getting warm, and their precious battery indicator slipping towards a powerless coma without being able to either find what is causing it or, better, some UI that communicates what work is being done on their behalf and a way to cancel it if necessary. Getting work off the main thread is what allows apps to do work for users without locking up those UI indicators, but to provide any ability to interrupt the task or broadcast progress, it must yield to the main event loop.

It’s this second concern that I think it truly fatal to the cause of sync worker APIs: assuming they work and are popular, they will create a world in which it’s necessary to put limits on their overall running time…limits that will be circumvented by breaking up the work into smaller chunks and dealing with it asynchronously inside the worker. Likewise, anyone building a serious app that’s trying to do the right thing by the user will factor their worker’s tasks into small enough chunks that they can both service “stop” messages and distribute progress notifications to the UI. There might be scenarios where such messages aren’t necessary and where users aren’t coveting CPUs and batteries…where sending SIGHUP doesn’t matter. But the intersection of those scenarios and the client-side web seems mostly to be a happy accident: your code might not have encountered enough data to create the problems. Yet.

This is particularly clear in the IDB cases: upgrading, iterating over, and updating hundreds of thousands of items of data is the sort of thing that will take a while, and is likely in response to some application semantic the user cares about: synchronizing mail, migrating to a faster schema layout in the process of some upgrade, etc. A blind for loop is asking for trouble. All of this might work fine in a dev environment with a (small) staging set of data…but it’s recipe for disaster when power-users with tons of data encounter it. What then? If the APIs an app depends on are all synchronous, it’s a huge boulder to roll up a hill to provide notifications, chunk up work, and refactor around async-ish patterns that chunk work up. If the work was async in the first place, the burden is much lower. So even apps that aren’t Doing It Right (TM) are likely to reap some benefit down the line from thinking in terms of async first.

There are other arguments that you can field against these sorts of APIs, particularly ones that double-up API surface area, but it doesn’t seem to me that they’re necessary. The person attempting to justify synchronous worker APIs who provides a good argument for ergonomics and learnability still has all their work ahead of them: they must show that these APIs are not harmful to the user experience. After all, Workers were added to the platform as a way of improving UX (by moving work off the main thread). And I fear they cannot do so without violating core JS semantics.

So let’s pour one out for our sync API dreams: we’re gonna miss you, control flow integration. But not for too long. Generators, iterators, and yield will see you avenged.

May the deities we’ve invented forgive us for the tripe we’re about to sell each other as “news”.

What’s bound to be missing in most of this coverage is what’s plainly said, if not in so many words, in the official blog post: going faster matters.

Not (just) code execution, but cycle times: how long does it take you to build a thing you can try out, poke at, improve, or demolish? We mere humans do better when we have directness of action. This is what Bret Victor points us towards — the inevitable constraints of our ape-derived brains. Directness of action matters, and when you’re swimming through build files for dozens of platforms you don’t work on, that’s a step away from directness. When you’re working to fix or prevent regressions you can’t test against, that’s a step away. When compiles and checkouts take too long, that’s a step away. When landing a patch in both WebKit and Chromium stretches into a multi-day dance of flags, stub implementations, and dep-rolls, that’s many steps away. And each step hurts by a more-than-constant factor.

This hit home for me when I got my first workstation refresh. I’d been working on Chrome on Windows for nearly a year in preparation for the Chrome Frame release, and all the while I’d been hesitant to ask for one of the shiny new boxes that the systems people were peddling like good-for-you-crack — who the hell was I to ask for new hardware? They just gave me this shiny many-core thing a year ago, after all. And I had a linux box besides. And a 30″ monitor. What sort of unthankful bastard asks for more? Besides, as the junior member of the team, surely somebody else should get the allocation first.

Months later they gave me one anyway. Not ungrateful, I viewed the new system with trepidation: it’d take a while to set up and I was in the middle of a marathon weekend debugging session over a crazy-tastic re-entracy bug in a GCF interaction with urlmon.dll that was blocking the GCF launch. If there was a wrong time to change horses, surely this was it. At some point it dawned that 5-10 minute link times provided enough time to start staging/configuring at the shiny i7 box.

A couple of hours later the old box was still force-heating the eerily dark, silent, 80-degree floor of the SF office — it wasn’t until a couple of weeks later that I mastered the after-hours A/C — when my new, even hotter workstation had an OS, a checkout, compiler, and WinDBG + cargo-culted symserver config. One build on the new box and I was hooked.

5-10 minute links went to 1-2…and less in many cases because I could now enable incremental linking! And HT really worked on the i7′s, cutting build times further. Hot damn! In what felt like no-time at all, my drudgery turned to sleuthing/debugging bliss (if there is such a thing). I could make code changes, compile them, and be working with the results in less time than it took to make coffee. Being able to make changes and then feel them near-instantly turned the tide, keeping me in the loop longer, letting me explore faster, and making me less afraid to change things for fear of the time it would take to roll back to a previous state. It wasn’t the webdev nirvana of ctrl-r, but it was so liberating that it nearly felt that way. What had been a week-long investigation was wrapped up in a day. The launch was un-blocked (at least by that bug) and the world seemed new.

The difference was directness.

The same story repeats itself over and over again throughout the history of Chrome: shared-library builds, ever-faster workstations, trybots and then faster trybots, gyp (instead of Scons), many different forms of distributed builds, make builds for gyp (courtesy of Evan Martin), clang, and of course ninja (also Evan…dude’s a frickin hero). Did I mention faster workstations? They’ve made all the same sort of liberating difference. Truly and honestly, in ways I cannot describe to someone who has not felt the difference between ctrl-r development and the traditional Visual Studio build of a massive project, these are the things that change your life for the better when you’re lashed to the mast of a massive C++ behemoth.

If there is wisdom in the Chrome team, it is that these projects are not only recognized as important, but the very best engineers volunteer to take them on. They seem thankless, but Chrome is an environment that rewards this sort of group-adaptive behavior: the highest good you can do as an engineer is to make your fellow engineers more productive.

And that’s what you’re missing from everything else you’re reading about this announcement today. To make a better platform faster, you must be able to iterate faster. Steps away from that are steps away from a better platform. Today’s WebKit defeats that imperative in ways large and small. It’s not anybody’s fault, but it does need to change. And changing it will allow us to iterate faster, working through the annealing process that takes a good idea from drawing board to API to refined feature. We’ve always enjoyed this freedom in the Chromey bits of Chrome, and unleashing Chrome’s Web Platform team will deliver the same sorts of benefits to the web platform that faster iteration and cycle times have enabled at the application level in Chrome.

Why couldn’t those cycle-time-improving changes happen inside WebKit? After all, much work has happened in the past 4 years (often by Googlers) to improve the directness of WebKit work: EWS bots, better code review flow, improved scripts and tools for managing checkins, the commit queue itself. The results have been impressive and have enabled huge growth and adoption by porters. WebKit now supports multiple multi-process architecture designs, something like a half-dozen network stack plug-ins, and similar diversity at every point where the engine calls back to outside systems for low-level implementation (GPU, network, storage, databases, fonts…you name it). The community is now committed to enabling porters, and due to WebKit’s low-ish level of abstraction each new port raises the tax paid by every other port. As James Robinson has observed, this diversity creates an ongoing drag when the dependencies are intertwined with core APIs in such a way that they can bite you every time you go to make a change. The Content API boundary is Blink’s higher-level “embedding” layer and encapsulates all of those concerns, enabling much cleaner lines of sight through the codebase and the removal of abstractions that seek only to triangulate between opaque constraints of other ports. Blink gives developers much more assurance that when they change something, it’s only affecting the things they think it’s affecting. Moving without fear is the secret of all good programming. Putting your team in a position to move with more surety and less fear is hugely enabling.

Yes, there are losses. Separating ourselves from a community of hugely talented people who have worked with us for years to build a web engine is not easy. The decision was wrenching. We’ll miss their insight, intelligence, and experience. In all honesty, we may have paid too high a price for too long because of this desire to stay close to WebKit. But whatever the “right” timing may have been, the good that will come from this outweighs the ill in my mind.

Others will cover better than I can how this won’t affect your day-to-day experience of WebKit-derived browser testing, or how it won’t change the feature-set of Chrome over-night, or how the new feature governance process is more open and transparent. But the most important thing is that we’ll all be going faster, either directly via Blink-embedding browsers or via benchmarks and standards conformance shaming. You won’t feel it overnight, but it’s the sort of change in model that enables concrete changes in architecture and performance and that is something to cheer about — change is the predicate for positive change, after all.

It seems entirely rational to be skeptical about governments doing things well. My personal life as a migrant to the UK is mediated by heaving bureaucracies, lawyer-greased wheels, and stupefyingly arbitrary rules. Which is to say nothing of the plights of my friends and co-workers — Google engineers, mostly — whose migration stories to the US are horrifying in their idiocy. Both the US and UK seem beholden to big-stupid: instead of trying to attract and keep the best engineers, both countries seem hell-bent to keep them out. Heaven forbid they make things of value here (and pay taxes, contribute to society, etc.)! It takes no imagination whatsoever for me to conjure the banality and cruelty that are the predictable outcomes of inflexible, anachronistic, badly-formulated policy.

You see it perhaps most clearly when this banality is translated to purely transactional mediums. PDFs that you must fax — only to have a human re-type the results on the other end. Crazy use of phones (of course, only during business hours). Physical mail — the slowest and worst thing for a migrant like myself — might be the most humane form of the existing bureaucracy in the US. Your expectations are set at “paper”, and physical checks and “processing periods” measured in weeks feel somehow of a piece. It has always “worked” thus.

It’s befuddling then to have been a near witness to nothing short of the wholesale re-building of government services here in the UK to be easiest to navigate by these here newfangled computrons. And also just flat-out easy. The mantra is “digital by default”, and they seem to be actually pulling it off. Let me count the ways:

1. High-level policy support for the effort
2. Working in the open. Does your government do its development on github?
3. Designing services for the humans that use them, not the ones who run them
4. Making many processes that were either only-physical or damn infuriating near-trivial to do online
5. Making key information understandable. Give the UK “limited corporation” page a view. Now compare to the California version. Day, meet night.
6. Saving both government and citizens massive amounts of money in the process

They even have a progress bar for how many of the ministries have been transformed in this way.

Over the same timeframe I’ve known a few of the good folks who have put themselves in the position of trying to effect changes like this at Code for America. It’s anathema in the valley to say anything less than effusive about CFA — anything but how they’re doing such good, important work. How CFA has the potential to truly transform the way citizens and government interact. Etc, etc. And it’s all true. But while CFA has helped many in the US understand the idea that things could be better, the UK’s Government Digital Service has gone out and done it.

So what separates them?

First, the sizes of the challenges need to be compared. The US has 5x the population, an economy that’s 6x larger, and a federalist structure that makes fixing many problems more daunting than most UK citizens can possibly imagine. Next, it should be noted that London is a better place to try to hire the Right People (TM). Yes, it’s much more expensive to live here, but software salaries are also much lower (both in relative and absolute terms). There wasn’t as much tech going on here as in the valley to start with, and the gold-rush to produce shiny but less competent versions of existing websites for world+dog (aka: “the app ruse”) hasn’t created the engineering hiring frenzy here that it has stateside. There’s also a general distrust in the American psyche about the core proposition of the government doing good things. Public-spiritedness seems to so many of my generation a sort of dusty nostalgia that went the way of hemp and tie-dye. Close encounters with modern American government do little to repair the image.

But all of those seem surmountable. The US has more of everything, including the Right People (TM). Indeed, the UK is managing an entire first-world’s set of services on a smaller GDP. Why then do US public services, to be blunt, largely still suck?

The largest differences I’ve observed are about model. Instead of having a mandate to change things from the inside, the organizational clout to do it, and enough budget to make a big dent out of the gates (e.g. gov.uk) CFA is in the painful position of needing to be invited while at the same time trying to convince talented and civic-minded engineers and designers to work for well below industry pay for a limited time on projects that don’t exist yet.

Put yourselves in the shoes of a CFA Fellow: and your compatriots are meant to help change something important in the lives of citizens of a municipality that has “invited” you but which is under no real pressure to change, has likely moved no laws or contracts out of the way to prepare for your arrival, and they know you’re short-timers. Short-timers that someone else is taking all the risk on and paying for? What lasting change will you try to effect when you know that you’ve got a year (tops) and that whatever you deliver must be politically palatable to entrenched interests? And what about next year’s Fellows? What will they be handed to build on? What lasting bit of high-leverage infrastructure and service-design will they be contributing to?

The contrast between that and the uncomfortably-named “civil servants” of the GDS could not be more stark. I don’t get the sense that any of them think their job is a lifetime assignment — most assume they’ll be back at agencies any day now, and some of the early crop have already moved on in the way nerds tend to do — but at the pub they talk in terms of building for a generation, doing work that will last, and changing the entire ethos of the way services are produced and consumed. Doing more human work. And then they wake up the next morning and have the authority and responsibility to go do it.

I don’t want to be down on CFA. Indeed, it feels very much like the outside-government precursor to the GDS: mySociety. mySociety was put together by many of the same public-spirited folks who initially built the Alpha of what would a year later become gov.uk and the GDS. Like CFA, mySociety spent years pleading from the outside, making wins where it could — and in the process refining ideas of what needed to change and how. But it was only once the model changed and they grabbed real leverage that they were able to make lasting change for the better. I fear CFA and the good, smart, hard-working people who are pouring their lives into it aren’t missing anything but leverage — and won’t make the sort of lasting change they want without it. CFA as an organization doesn’t seem to understand that’s the missing ingredient. America desperately needs for its public services to make the same sort of quantum leap that the UK’s are making now. It is such an important project, in fact, that it cannot be left to soft-golved, rose-tinted idealism. People’s lives are being made worse by mis-placed public spending, badly executed projects, and government services that seem to treat service as an afterthought. CFA could be changing this, and we owe it to ourselves and our friends there to ask clearly why that change hasn’t been forthcoming yet.

The CFA Fellows model has no large wins under its belt, no leverage, and no outward signs of introspection regarding its ability to deliver vs. the GDS model. Lets hope something larger is afoot beneath that placid surface.

Update: I failed to mention in the first version of this post that the one of the largest philosophical differences between the two countries is the respective comfort levels with technocratic competence. There exists a strain of fatalism about government in the US that suggests that because government doesn’t often do things well, it shouldn’t try. It’s a distillation of the stunted worldviews of the libertarian and liberal-tarian elite and it pervades the valley. Of course governments that nobody expects anything of will deliver crappy service; how could it be otherwise?

What one witnesses here in the UK is the belief that regardless of what some theory says, it’s a problem when government does its job badly. To a lesser degree than I sense in years past, but still markedly moreso than in the US, the debate here isn’t about can the government get good at something, but why isn’t it better at the things the people have given it responsibility for?

As a result, the question quickly turns how one can expect a government to manage procurement of technical, intricate products for which it’s the only buyer (or supplier) without the competence to evaluate those products — let alone manage operations of them. Outsourcing’s proponents had their go here and enormous, open-ended, multi-year contracts yielded boondoggle after boondoggle. By putting contractors in a position of power over complexity, and starving the in-house experts of staffing and resources to match, the government forfeited it’s ability to change its own services to meet the needs of citizens. What changed with gov.uk was that the government decided that it had to get good at the nuts and bolts of delivering the services, outsourcing bits and pieces of small work, but owning the whole and managing it in-house. Having the Right People (TM) working on your team matters. If they’re at a contractor, they have a different responsibility and fiduciary duty. When the ownership of the product is mostly in-house, ambiguities borne of incomplete contract theory are settled in favor of the citizen (or worst case, government) interest, not the profit motive.

The gov.uk folks say “government should only do what only government can do”, but my observation has been that that’s not the end of the discussion: doing it well and doing it badly are still differentiable quantities. And doing better by citizens is good. Clearing space to do good is the essential challenge.

After doing much reading of TAG meeting minutes, f2f notes, issues, delivered products, and findings I’ve come to a sobering conclusion: the TAG isn’t focused on eliminating the biggest sources of developer pain today. Now, you can argue that this might not be their job, but I won’t agree. It’s the TAGs job to help ensure the health of the platform, both for publishers and search engines, but also for authors. And the web as a platform is in some real trouble today.

There doesn’t seem to be a sense of urgency in the TAG about the corrosive effects of poor design and integration on the overall usability and health of the system. The Web to the TAG, as I can understand it through the meeting minutes and notes, is a collection of documents that represent internally-referential collections of data which are are linked to other documents, not a series of applications that are attempting ever more impressive feats of richness on a platform that stymies them every time you hit one of the seams. In reality it is (aspirationally) both things but the very real tensions between them don’t appear in the TAG’s work, leading me to believe that it doesn’t comprehend the latter aspect of web development today and what that means for the future health and viability of the platform.

I drone on and on and on about layering because explaining each bit of the platform in terms of the one below it relieves much of the tension created by disconnected declarative forms and APIs. This matters because in today’s web when you go ever so slightly off the path paved by a spec’s use-cases, the drop-off is impossibly steep, and the only way to keep from risking life-threatening abstraction level transitions is to flood the entire canyon with JavaScript and hope you can still swim in the resulting inland sea of complexity. This is what our biggest, “best” webapps do today, relying on ENORMOUS piles of JavaScript that largely serve to re-create what browsers already do in the hopes of marginally extending that capability. It’s simply nuts, but the TAG doesn’t seem to acknowledge the threat this poses to everything it holds dear: linking, declarative forms, data…it’s all about to be lost beneath the waves, and because the TAG doesn’t understand the growing importance of JS, it seemingly doesn’t see the threat. Declarative forms disappear first beneath imperatively-delivered complexity; lingua-franca APIs next. Without ways of getting what your app needs while keeping one foot on the declarative path, app developers do what they must; declarative data and relationships become “nice to haves” not “the way you do it”. Layering provides easy steps between the levels of abstraction, avoiding the need to re-create what the platform was already doing for you along with whatever custom thing you need — and it’s not the TAG’s current agenda.

If elected, I will work to fix that. The TAG is the right group to formulate and articulate a theory of good layering in the web platform’s architecture and it’s the only group at the W3C whose mission is to help spec authors wrestle with large-scale design and integration problems like this. My background is in apps, and JS, and browsers, and I work at one of the few places deeply invested in ensuring that we maintain a healthy, declarative web for the future. I care tremendously about the viability of the largely-declarative web. Through my work with Dimitry Glazkov and many others on Web Components I’ve done as much as anyone in the last decade to help build a bridge between the JS and declarative worlds. Dimitry and I created and led the team here at Google that have put Shadow DOM, CSS Variables, Custom Elements, Mutation Observers (and Object.observe) into specs and on the platform agenda, all with the explicit goal of creating better layering; explaining the magic in today’s platform and drawing connections between the bits that had none before. And I think we need to keep building more of those bridges, but it’s hard when W3C culture views that agenda with suspicion. Why would any WG concern itself with integration with specs outside its charter? It’s the TAGs job to inject that global perspective. I believe the TAG should pursue the following work as a way of filling its charter:

• Getting reconnected to web developers: today’s TAG isn’t composed of web developers (Sir Tim excepted) and the general level of familiarity on the committee with the pressing issues in web development seems low. As a member I’ll press to ensure that at least one of the face-to-face meetings each year overlaps with an industry web development conference (not an academic symposium). Having the TAG simply go and listen, and perhaps answer questions in such a forum, would do much to illuminate the gulf in understanding. I also support Marco’s agenda of direct developer outreach (G+, AMA, IRC, etc.)
• Forming and expressing an opinion about idiomatic JS APIs: the W3C is the body that specifies JavaScript’s largest, most important standard library yet it doesn’t seem to treat that task with any care. Poor integration with JS types, idioms, library conventions, and calling conventions are the calling card of W3C APIs. The detrimental effects are pervasive and corrosive. The TAG should produce a guide on designing idiomatic JS APIs for WGs to use. Additionally, the TAG should use its spec review perch to call out poorly designed JS APIs as such and propose changes that improve full-stack integration. Lastly, the TAG should be open to facilitating (or teaching!) classes on how to design good APIs for the web. It’s a skill that can be learned like any other, and the TAG has a unique responsibility towards ensuring that the next generation of APIs is better than the last.
• Bridge building to TC39: I’ve served on ECMA TC39 — the standards body for JavaScript — since 2007 and in that time I’ve seen how the dysfunctional relationship with the W3C enables terrible API design. Needless to say, the results haven’t been great. For a small taste, go look at some IndexedDB samples. There are dozens of terrible DOM APIs (from the perspective of JS) and JS fails to help describe interfaces well enough (from the perspective of DOM). The impasse has even been codified. For the sake of the platform and the developers who must use it, we need to do better. I will push to ensure that at least one of the face-to-face meetings of TC39 each year overlaps with a TAG meeting. I’m hopeful that with my fellow TC39 delegate (Yehuda Katz) and the maintainer of DOM (Anne van Kesteren) also on the TAG, we can make serious progress on this front, actively building a culture of understanding between TC39 and the W3C and furthering the common goal of a web that’s competitive and well integrated.
• Advocate for layering: it’s the TAGs natural role to advocate for coherence in the platform, not only at the markup level, but also inside the app runtime bits that take up so very much of the time in W3C’s most active WGs. If the TAG doesn’t do this, asking the important questions at the right time and working to show the benefits of collaboration and cross-API thinking, who will? To combat this, I propose that the TAG take as an open issue the lack of coherence in the client side platform and work to identify the largest developer pain-points in findings that work to set an agenda for WGs in the future. There is no hope for a well-integrated, layered platform without every WG accepting some responsibility for the usability and layering commons, and if elected I will work to ensure the TAG is a tireless advocate for that commons.

If that sounds like meaningful progress to you, I’d appreciate your organization’s vote; along with your consideration to vote for my fellow reformers: Yehuda Katz, Anne van Kesteren, Peter Linss, and Marcos Cáceres. AC reps for each organization can vote here and have 4 votes to allocate in this election. Voting closes near the end of the month, and it’s also holiday season, so if you work at a member organization and aren’t the AC rep, please, find out who that person in your organization is and make sure they vote. The TAG can’t fix the web or the W3C, but I believe that with the right people involved it can do a lot more to help the well-intentioned people who are hard at work in the WGs to build in smarter ways that pay all of us back in the long run.

Origin(al) Sins – Alex Russell from OWASP AppSec USA on Vimeo.

I made some pretty glaring errors in the talk: you can’t combine sandboxing with seamlessness for cross-origin content. It’s something I’m hoping we can improve on in the browsers/specs, but it’s not the current state of play. I also failed to mention that CSP is Mozilla’s brain-child and they deserve huge credit for pushing it down the field. Similarly, I failed to mention that IE 10′s CSP support is actually quite shite, supporting only the sandbox directive. Lastly, my objections to the OCAP worldview may have been oblique so, before the flames arrive in the comments, let me try again:

I think that you’ll always end up with some variant of “object capabilities”, if only through wrapper objects that hide protocol details. The OCAP world calls these “membranes” for particularly high-fidelity versions of this. When you have a word for it, it’s probably common. Vending objects that represent capabilities is natural at some level, but I strongly resist the urge to take the pun too far. Don’t get me wrong, I’m a huge fan of capabilities as the model for building apps that have real security by default; and I hope their pervasive use combined with more restrictive, separated execution contexts creates the world we all want to be in. My only quibble is on the developer ergonomics front. OCAP encourages the (perhaps naive) programmer to build many things inside the same “vat” (heap, context, whatever), leading to brittleness whereas protocols and true sandboxing can create secure boundaries that, importantly, look like boundaries. Systems that provide big “this is a security boundary!” disclaimers on their APIs while making it hard to screw them up stand a better chance of weathering the storms of human imperfection. Can you do OCAP right? Sure, it’s possible, but as I argue in the talk, betting on humans to do the right thing is eventually and inevitably a loser. So I favor large, knobby interfaces over which you can send messages and no more. Wrap ‘em up with an RPC layer…cool…whatever. But design a protocol with side-effects that are straight-forward to reason about on both sides — not an object which can be easily intertwined with many others in a dense graph — and you’ve got my vote. I’m even in favor of building those protocols and then wrapping them with easier-to-use APIs (call it “CAP->O”, if you will), but eliding the step of a large boundary over which you can only send data, and making it relatively hard to cross? Nah, I’ll be over here with the easy-to-reason about solutions that don’t make my small-ish brain melt, thanks.

Also, I wasn’t sure to what extent Doug was joking about “incompetent” programmers in his talk, so our disagreements may not be what they seem. Caveat emptor.

Thanks to Jeremiah and the other organizers for taking a risk and inviting someone who’s been out of the security space for so long to give a talk. I promise to them that should they be so foolish in the future, I’ll be sure to duck out and get one of my betters on the Chrome Security Team to stand in instead ;-)

…I don’t really share his vision. I think JS is an important part of the Web, but I don’t see it as any more core to the Web than CSS, HTML, the DOM APIs, HTTP, etc. Most of the proposals I’ve seen to take JS and build a platform on it tend to fall down on #3, #4, and #5. In particular, I think fundamentally #3 and #4 basically force the platform to have a declarative part (the declarative XAML part of .NET is how it managed to address #4, for instance, and the declarative design of XHTML2 and RDF are how they manage to address #3). That doesn’t have to mean that the code part is a secondary part, though. I think the Web has reached a point where you can think of the DOM as the core, with JS and the HTML syntax layered atop that (much like how in .NET there’s objects that can be represented in XAML or generated from C#), which puts the JS part on par with the HTML part in terms of importance to the platform.

This is either me not articulating my “vision” clearly, Hixie mis-understanding my point(s), or some combination thereof. So I’ll try again in 4 paragraphs:

I make no claims about the relative importance of HTML, CSS, JS, DOM or any of the rest. It honestly feels silly to talk about the platform that way. I argue something very, very different: HTML, DOM, and (to a lesser extent) CSS should all conceptually bottom out at JS. I’m not saying “go write a browser in JS”…that’s also non-sequitur. JS must, of course, appeal in turn to C/C++ so a “browser written in JS” would need privileged APIs provided by C/C++ or something lower level.

What I’m saying is something very direct: there’s no other ubiquitous programming language for the web. JS is the only game in town. As a result, when we describe imperative behavior in prose (as the HTML5 spec does) that behavior could, in theory, be implemented in JS (or some sub/super-set). Since HTML, DOM, and CSS aren’t Turing-complete, they naturally belong at a “higher level” in the web platform stack.

Now, all of this sounds dangerously academic and perhaps nuts…but consider that we have to expose APIs for nearly all of this stuff. Whenever we add something to HTML, CSS, etc., it always comes with some sort of API. Now, that could be a good API, or it could be an obtuse turn around the parser after some terrible string concatenation.

I’m claiming that you get better APIs when you design your imperative explanation of the declarative stuff — which you MUST do for specs to be sane — in JS. This means designing new APIs to help you accomplish the task at hand as you go. Doing it that way ends you up with the types, APIs, and security boundaries you’re going to need in the end anyway. Doing it the other way, patching crap onto the side later without explaining how the declarative stuff “works”, just leads to the pain and constant need for case-by-case revision that we see in today’s web APIs.

That’s the whole argument in 4 ‘grafs. It’s subtle, but I hope not terribly controversial. Importantly, nothing about it demands giving up on any of Hixie’s 5 arguments. He can keep ‘em all and still get a well-layered platform. We’ve got enough examples of doing it wrong under our belt to see how painful the badly-layered web is today. We can end this insanity — not by claiming that any part of the platform is “more important” — by explaining the platform in layers, higher levels in terms of the lower levels.

Anyhow, as a result of that ever-growing design/analysis/writing hairball, I haven’t blogged when I might have. Notably, videos of some of my recent talks are now up, including my Fronteers closing talk:

Alex Russell | The Web Platform & the Process of Progress | Fronteers 2012 from Fronteers on Vimeo.

Special thanks to Peter and the Fronteers volunteers for managing to find a pink boozy cake (a culinary revelation!), simultaneously meeting my only (joke) demand as a speaker and trolling me in person. WELL PLAYED.

Also, give the rest of the excellent talks a gander over on Vimeo . Personal favorites of mine include the ever-brillaint Phil Hawksworth, Marcin Wichary’s stunning Doodles talk, Lea Verou, and Rebecca Murphey….actually, there were too many to list. Seriously, go check it out.

I also recently spoke at the London Ajax User’s Group, and video is also available (but not embeddable). Apologies for both videos including a lot of me rambling. I need to get better at this whole “public speaking” thing.

On a related note, FFJS was once again brilliant. Best one day conference going this side of the world, hands down. Congrats to Julie and Remy on a stonkingly good day.

The answer is that I’m now in the running for one of the TAG’s 4 open seats. At last week’s TPAC I was talked talked myself into running. Google has nominated me, so we’re off to the proverbial races. Given the number of condolences I’ve received since deciding to run, you might wonder why I’d do such at thing at all; particularly since the general consensus is that the TAG’s impact is somewhere between “mostly harmless” and “harmless”. I’ve never heard it described as “valuable” or “productive” in its modern incarnation, so why run? Could it be for the warmth of human contact that a weekly conference call could provide? A deep-seated but as yet unfulfilled desire to weigh in on what really irks me about how people use HTTP? Indeed not.

I’m running to try to turn the TAG into an organization that has something to say about the important problems facing devs building apps today; particularly how new specs either address or exacerbate those challenges.

Notionally that’s what the TAG does now; review specs in development and provide feedback. If you look at the current work items, however, there’s nothing about JavaScript. There’s similarly little on DOM or the sorry state of API design at the W3C and how it is enabled by WebIDL and a broad cultural ignorance of JS. The TAG could provide advice on API design to WGs, how to think about creating a well layered platform, and how to keep webdevs in mind at all times.

State management gets a lengthy exposition that doesn’t do much to uncover the challenges in designing apps that need to keep and manage state, synchronize that state in local data models, and map state onto URLs. Its a hard problem that most JS-driven apps struggle with and the TAG has no view beyond motherhood, apple pie, and pushState(). It can do better.

For webdevs, the TAG is a natural ally — it carries the weight of the W3Cs reputation in ways that vendor-motivated WGs don’t. It can be an advocate for building things well to reluctant managers and organizations, helping you make the “environmentalist” argument; “the web is for everyone, why are you making it bad?” It can also be a go-between for agitated webdevs and the WGs, a voice of reason and patient explanation.

As a talking shop, the TAG can never be expected to have power over implementations or webdevs, but it can be opinionated. It can take a stand on behalf of webdevs and call out huge problems in specs, missing work items for WGs, and build bridges to the framework authors that are doing the heavy lifting when the browsers don’t. It can also build bridges to other standards bodies that specify hugely important parts of the web platform, bringing a webdev perspective to bear.

I have worked inside a browser, represented Google at TC39, and have built large apps and frameworks on the web stack. I have seen firsthand how the TAG isn’t doing this important work of building bridges and jawboning on behalf of web developers. And I plan to fix it, if elected.

If you work at a company that is a member of the W3C, I hope to get your AC rep’s vote. If not, I would still love your support in letting AC reps from the Member organizations know that you support the goal of turning the TAG into an advocacy organization for the interests of webdevs.

If you’re particularly interested in this problem, I’d love your help — I’m looking for reformist-minded candidates to join me in running for the TAG. If you don’t have a sponsoring or nominating organization, send me mail or a tweet.

Voting nitty-gritty, largely cribbed from Ian Jacob’s mail to a Members-only W3C mailing list:

• Nominations: any Member organization may nominate one individual to serve on the TAG. That nominating organization is expected to cover their nominee’s costs for the period of service, so don’t nominate in haste. Nominations are currently open and close on Nov 30th.
• Voting: only Advisory Committee members from Member organizations can vote on the TAG elections. It’s not clear when voting starts but it ends on January 31st, 2013. So if you have an opinion about this, find your AC rep (or one from a company you think should be voting here) and let them know how you feel.

“That’s just a browser caching problem.”

Look for this to be deployed along standards-wonk classics such as “our job should just be to provide low-level primitives”, “we don’t know enough to solve this”, and “people will just write tiny libraries to provide that”.

“It’s a caching problem” is a common refrain of those who don’t build apps and aren’t judged on their latency. And it’s transparently bullshit in the web context. It relies on you forgetting — for just that split second while it sounds possible to duck the work at hand — that we’re talking about a platform whose primary use-case is sending content across a narrow, high-latency pipe. If you work in web standards and don’t acknowledge this constraint, you’re a menace to others.

Recent history alone should be enough to invalidate the caching argument; remember Applets? Yeah, Java had lots of problems on the client side, but what you saw with client-side Java were the assumptions of server-side engineers (who get to control things like their deployment VM versions, their hardware and spindle speeds, etc.) imported to distributed code environments where you weren’t pulling from a fast local disk in those critical moments when you first introduced your site/app/whatever to the users. Instead, you saw the piles upon piles of JARs that get created when you assume that that next byte is nearly free, that disk is only 10ms away, and that cold startups are the rare case. It worked out predictably and Java-like systems succeed on the client when their cultural assumptions do align with deployment constraints — Android and iOS are great examples. Their mediated install processes see to it.

Back out here on the wolly web, caching is something that’s under user control. It must be for privacy reasons, and we know from long experience that users clear their caches. The “cache” they can’t clear is the baseline set of functionality the platform provides — i.e., the built in stuff on all runtimes developer cares about…which is what specs can effect (obliquely and with some delay).

By the time you’re having a serious discussion about adding a thing to a spec among participants who aren’t obvious bozos, you can bet your sweet ass that the reason it was brought up in the first place is a clear and obvious replication of effort among existing users bordering on the ubiquitous — often in libraries they use. Saying that someone can write a library rises no higher than mere truism, and saying that a standards body shouldn’t provide some part of the common features found among those existing libraries because caching will make the cost of those libraries moot is ignorance or standards-jujitsu in an attempt to get a particular proposal shelved for some other reason.

As bad as the above is, consider the (historically prevalent) use of this argument regarding “why we don’t need no stinking” markup features — just “fix” browser caching and “give me a low-level thing and I’ll build [rounded-corners, gradients, new layout mechanisms, CSS3d xforms, etc.] myself”. The subtle bias towards JavaScript and away from a declarative web is one of the worst, most insidious biases of the already enfranchised upper-class of JavaScript-savvy web developers, perpetuating a two-tier web in which “real engineers” can do better every year but wherein those same expressiveness and performance gains aren’t transmitted to folks without CS degrees (yes, I’m looking straight at you, WebGL). You almost want to give it to them, though, so they’ll finally come to terms with how wrong they really are about the ability for caching to be “fixed”.

We’ve spent a decade on this — remember that we’re all using CDNs, pulling our JS libraries from edge-cached servers with stable URLs for optimal LRU behavior, running minifiers, setting far-forward expires, etc. And that’s just what webdevs are doing: meanwhile browser vendors have been working day and night to increase the sizes of caches, ensure full caches where possible, and implement sometimes crazy specs that promise to help. It’s not for lack of trying, but we still don’t collectively know how to “fix” caching.

Think libraries are free? Show me how you’ll make ‘em free and I’ll start taking you seriously. Until then, bozo bit flipped.

“It should have exactly the same API as library X”

Similar arguments include “it must be pollyfillable”, etc.

Why is this bullshit? Not because it represents an aversion to being caught in an implementation/deployment dead zone — that’s a serious concern. Nobody wants a better world dangling out there just beyond reach, waiting for Vendor X to pick it up and implement or for Version Y of Old Browser to die so that 90% of your clients can acces the feature. That’s where libraries provide great value and will continue to, no matter what the eventual feature looks like. Remember, the dominant libraries in use by developers don’t have Firefox, Chrome, IE, and Opera-specific versions that you serve up based on client. The platonic ideal is 180-degrees in the other direction: one code base, feature detection, polyfills, progressive enhancement — basically anything within (and often well outside of) reason to keep from serving differential content. So your library is going to have all those versions anyway until all of your clients have the new-spec version. Optimizing based on existing API design because “now we can polyfill it without extra effort!” misunderstands both the role of spec designers and of libraries and serves users very poorly indeed.

Where this argument becomes truly inadmissible, though, is when it seeks to define the problem to be “what our API already does”. Turns out that the language you can design features with when all you have is JavaScript and HTML/CSS patterns is…well…the JS, HTML, and CSS you have today. Powerful yes. Expressive? Hrm. If your job, however, is to evolve an integrated platform, taking on a constraint predicated on the current form of your systems is nuts. There are some hard constraints like that (backwards compatibility), but adopting some form of hack as the “blessed” way of doing something without looking around and going “how can we do this better given that we have more powerful and expressive language to solve the problem with?” is nothing but lost opportunity.

Yes, a standards group might look around and go “nope, we can’t do better than that polyfill/library” and adopt the solution wholesale. That’s not a bad thing. What is bad, though, is advocacy about far-future solutions to current problems based solely on the idea that some library totally nailed it and we shouldn’t be asking for anything more — e.g.: Microdata cribbed from RDFa and Microformats when what you really wanted was Web Components. Rote recital of existing practice is predictably weak sauce that robs the platform of its generative spark. We should all be hoping for enhancements that make the future web stronger than today’s, and you only find those opportunities by taking off the blinders and giving yourself free reign to do things that libraries, polyfills, and hack just can’t.

What to do about the developers and users caught in the crossfire? Well, we can advocate for degradable, polyfill-friendly designs, but there are limits to that. The most hopeful, powerful way to make the pain disappear is to ensure that more of our users year-over-year are on browsers/platforms that auto-update and won’t ever be structurally left-behind again. And yes, that’s something that every web developer should be working towards; prompting users to update to current-version auto-upgrading — “evergreen”, if you will — browsers. Remember: you get the web you ask your users for.

“Just tell us the use cases”

This is the one I’m most sick of hearing from the mouths of standards wonks and authors. What they’re trying to say is some well-meaning combination of:

• If you tell us exactly how you’re doing whatever it is you’re accomplishing today, we’ll run the risk of just adopting some API that was designed with some bogus constraints (see above).
• We need to communicate to a lot of people involved in this effort why they should care, ’cause putting stuff into browsers is a shitload of work.

What it often winds up doing, however, is serving as a way for a standards body or author to shut up unproductive folks; folks who aren’t willing to do the work of helping them come to enlightenment about the architecture of the current solution, the constraints it was built under, and the deficiencies it leaves you with. Or it can be used to avoid that process entirely — which is where we’re getting towards bullshit territory. Taken to the extreme (and it too often is) the “use-cases, not details” approach infantilizes standards body participants, setting a bar too high for the folks who are crying out for help and setting the bar too low for the standards body because, inevitably, the tortured text of the “use cases” is an over-terse substitue for a process, a way of building, and an architectural approach. Use-cases (as you see them for HTML and CSS in particular) become architecture-free statements, no more informative than a line plucked at random from Henry V. Suggestive, yes. Useful? Nope. The very idea of building standards without a shared concept of architecture is bogus, and standards participants need to stop hiding behind this language. If you don’t understand something, say so. If many people work hard to explain it and can’t, ask for a sample application or snippet to hack on so you can do a mind-meld with their code and can ask questions about it in context.

Yes, having the use-cases matters, but nobody should be allowed to pretend that they’re a substitue for understanding the challenges of an architecture.

While I’ve only covered a few of the very worst spec-building tropes, it’s by no means the end of the rhetorical shit list. Perhaps this will be a continuing series — although I’m sure this post will offend enough folks to make me think twice about it. If we put an end to just these, though, a lot of good could be accomplished. Here’s to hoping.

At the last TC39 meeting, after spending what felt like an eternity on the important topic of UTF-16 encoding, the last day hastily ended with two topics that simply could not be any more urgent: can we get something done about data mutation observation — the underpinnings for every data binding system worth a damn — and classes.

As you might imagine, classes got at least one cold shoulder, but this time we were able to suss out the roots of the objection: the latest majority-agreed proposal (“Maximally Minimal Classes“) isn’t “high water mark” classes and, since so much has been given up to get something moved down the field, classes are no longer worth having. In this post I’ll outline a bit of the debate so far, current state, and hopefully convince you that the anti-classicist’s arguments are weak.

But first: why classes at all?

There’s an ambient argument in the JS world that “we don’t need no stinking classes”. That all we must do to save the language is teach people the zen of function objects and closures and all will be well with the world…and if we lose some people, so what? I don’t want to tar everyone who sympathizes with the beginning of that sentiment with the obvious negatives of the last bit, but that’s the overall gist I get. In this view, classes are an unnecessary and lead people to consider JS code the “wrong” way. I’ve written before about how in ES6 class means function, but that doesn’t mollify the discontent.

But then why any language feature at all? Why isn’t assembler good enough for everyone? Or C? Or C++?

Turns out the answer is “human frailty”. Or put a different way, the process of cognition depends on very limited amounts of short-term stack space in our wetware, and computing languages are about making the computer hospitable for the human, not about telling the system what to do. Our tradeoffs in languages would look much different if we could all easily recall 20 or 30 things at a time instead of 5-10. Languages are tools for freeing creative people from registers and stacks and heaps and memory management and all the rest; all while trying to keep the creative process grounded in the reality that it’s memory words in a Von Neumann architecture; without that grounding we’d end up too disconnected from the system to deliver anything practical.

Abstractions, high-level semantics, types…they’re all pivots, ways of considering sets of things while working with an individual example of that thing mentally. You don’t consider every case all at once and don’t remember everything about the system at one time; instead you focus on the arguments to the current function which creates it’s own scope. All of these mechanisms create ways of limiting what you need to think about. They ease the burden of associating some things with other things by allowing you to consider smaller cases and allowing you to create or break limitations as they become variously helpful and harmful to getting something done. And we have lots of words to describe the things we find painful about dealing with the overhead of remembering; of a lack of directness and locality: “spaghetti code”, “unmaintainable”, “badly factored”, “tightly coupled”, etc. Where there’s a new fad in programming, odds are high that it is being touted as a solution to some aggregate behavior which causes a reasonable local decision to become globally sub-optimal. This is what language features are for and why, when you see yourself using the same patterns over and over again, something is deeply wrong.

I’ll say this again (and imagine me saying it sloooooowly): languages evolve in response to the pain caused by complexity. The corollary is that language which don’t evolve to remove complexity, or which add it in day-to-day usage cause pain. Pain is almost always bad.

Complexity takes many forms, but the process of disambiguation when looking at a syntactic form that relies on a pattern is one of the easiest to take off the table. You’ll see this in my work in lots of areas: Web Components make the relationship between markup and eventual behavior less ambiguous when you read the HTML, Shadow DOM creates an isolation boundary for CSS and HTML that make it easier to consider a “thing” in isolation when building and using them, classes in JS make it easier to read some bit of code and not be tripped up by the many meanings of the word function, and many of the features I’ve advocated for in CSS (mixins, hierarchy, variables) are factoring mechanisms that make it simpler to pivot between repeated stuff and the ad-hoc visual semantics of the document that mean so much to the design language.

Not only should we write off the luddites, we should consider every step towards lower complexity good and steps towards complexity and slower understanding to be bad and actively harmful. Tortured claims about how “you aren’t going to need those things” need to be met with evidence; in this case, what do the pervasive libraries in the ecosystem do? Yeah, those are complexity to be taken off the table. The agenda is clear.

But back to classes and JS.

We’ve been debating various forms of classes in JS since the late ’90s. That’s right; we’ve been circling this drain for more than a decade. In that time, nearly every permutation of class system that I’m familiar with has been debated as the rightful heir to the reserved class keyword. From a parallel inheritance structure (non-prototypal) to “just prototypes like we have them today”, from “classes as constructor body” to “classes as object literals with constructors”, from frozen to gooey and unbaked, from classes as nominal types to totally untyped, we’ve seen it all. In all of this, the only constant has been failure. TC39 has continued, for more than a decade, to fail to field any class syntax (yes, I have my slice of blame-cake right here). In that time, JS has gone from being important to being critical to the construction of large systems; both on the client and the server. As this has happened, we’ve seen function pressed into service not only as lambda and method, but module, class, and pretty much every other form of ad-hoc composition system that’s needed at any point. We’re overdue for adding language-level support for all of these things. The committee has shown great ability to extend and expand upon idioms already in the language, taking syntax as a starting point and using it create new room for reducing pain (the spread operator and destructuring are great examples). Yet still many on the committee, notably Luke Hoban of Microsoft and Waldemar Horwat of Google, worry that a form of class in ES6 that doesn’t make coffee for you while also shooting rainbows out of it’s behind won’t be worth the syntactic space it takes up. I would have worried about this too back in ’08 when I first attended a TC39 meeting, but no longer. This is a group that is good at incremental change. Rainbow-shooting technology takes a while to build; meanwhile, wouldn’t it be great if we could get a cup of joe around here? We’ve waited long enough to get something meaningful to build on and with. I want a hell of a lot more than what the Maximally Minimal Classes proposal adds, but they’re a starting point, and there will be more ES versions.

So that’s Argument #1: people will hate us if class doesn’t do enough.

True, perhaps, but they will hate us anyway. Add classes and Mikeal Rogers personally organizes the pitchfork and torch distribution points and leads the march. Don’t add classes and the world at large thinks of JS as a joke and a toy — where’d that giant library/transpiler get to, anyway? You need something after all. Add them and Mikeal can keep not using classes to his heart’s content, and if we add classes in the style I prefer (and which Max/Min Classes embody), he can use them as old-skool functions and never care how they were declared; all the while making JS more productive for folks who don’t want to buy into a whole lifestyle to get a job done.

What’s Argument #2? That instances created from classes aren’t frozen. That is to say, it’s still possible to add new properties to them later. You know, like every other JS object.

If you’re thinking “yes, but isn’t there Object.freeze() already?”, you’re not alone. What you’re witnessing here is a debate about what’s considered good, not what’s possible. You can manually freeze/seal objects to your heart’s content, but what the (tiny) minority that is strenuously making argument #2 is demanding is that nobody be allowed to use the word class to create a constructor function body unless the resulting object is automatically frozen. They are willing to hold up the entire train until this preference is mollifed, and are unhappy with any short syntax which does not bless their preferred style.

In fairness, I also would be grumpy to see things go a direction I don’t prefer, but consider the following syntaxes that could be deployed in a future version to create frozen classes, none of which are precluded by the Max/Min proposal:

// "const" causes instances to have their properties 
// frozen on exiting the constructor
const class Point {
  constructor() { ... }
}
 
// Declaring a const properties member causes the
// constructor to check for those only, freezing exiting
// the constructor
class Point {
  constructor() { ... }
  const properties {
     // Object literal
     ...
  }
}

And I’m sure you can think of a couple of others. The essential distinguishing feature, and the thing that is holding the entire train up, is that the word class without any embellishment whatsoever doesn’t work this way. Yes, I know it sounds crazy, but that’s where we’re at. You can help, though. Erik Arvidsson has implemented Max/Min classes in Traceur and you can try it out in the live repl. If you like/hate it, won’t you please let the es-discuss mailing list know? Or just post in the comments here for posterity. We know that these classes need many more features, but your thoughts about this syntax as a starting point couldn’t be more timely.

”every year we’re throwing more and more JS on top of the web”

The way things are going in my world, we are looking at replacing the web with JS, not simply layering. At a certain point you look at it all and say “why bother”. Browsers render the DOM not markup. They parse markup. Just cut out the middle man and send out DOM – in the form of JS constructs.

The second part is to stop generating this markup which then must be parsed on a server at the other end of a slow and high latency prone communication channel. Instead send small compact instructions to the browser/client that tells it how to render and when to render. Later you send over the data, when it’s needed…

This is a clear distillation of what scares me about the road we’re headed down because for each layer you throw out and decide to re-build in JS, you end up only doing what you must, and that’s often a deadline-driven must. Accessibility went to hell? Latency isn’t great? Doesn’t work at all without script? Can’t be searched? When you use built-ins, those things are more-or-less taken care of. When we make them optional by seizing the reigns with script, not only do we wind up playing them off against each other (which matters more, a11y or latency?) we often find that developers ignore the bits that aren’t flashy. Think a11y on the web isn’t great now? Just wait ’till it’s all JS driven.

It doesn’t have to be this way. When we get Model Driven Views into the browser we’ll have the powerful “just send data and template it on the client side” system everyone’s looking for but without threatening the searchability, a11y, and fallback behaviors that make the web so great. And this is indicative of a particularly strong property of markup: it’s about relationships. “This thing references that thing over there and does something with it” is hard for a search engine to tease out if it’s hidden in code, but if you put it in markup, well, you’ve got a future web that continues to be great for users, the current crop of developers, and whoever builds and uses systems constructed on top of it all later. That last group, BTW, is you if you use a search engine.

But it wasn’t all clarity and light in the comments. Austin Cheney commented on the last post to say:

This article seems to misunderstand the intention of these technologies. HTML is a data structure and nothing more. JavaScript is an interpreted language whose interpreter is supplied with many of the most common HTML parsers. That is as deep as that relationship goes and has little or nothing to do with DOM.

…It would be safe to say that DOM was created because of JavaScript, but standard DOM has little or nothing to do with JavaScript explicitly. Since the release of standard DOM it can be said that DOM is the primary means by which XML/HTML is parsed suggesting an intention to serve as a parse model more than a JavaScript helper.

Types in DOM have little or nothing to do with types in JavaScript. There is absolutely no relationship here and there shouldn’t be…You cannot claim to understand the design intentions around DOM without experience working on either parsers or schema language design, but its operation and design have little or nothing to do with JavaScript. JavaScript is just an interconnecting technology like Java and this is specifically addressed in the specification in Appendix G and H respectively.

And, after I tried to make the case that noting how it is today is no replacement for a vision for how it should be, Austin responds:

The problem with today’s web is that it is so focused on empowering the people that it is forgetting the technology along the way. One school of thought suggests the people would be better empowered if their world were less abstract, cost radically less to build and maintain, and is generally more expressive. One way to achieve such objectives is alter where costs exist in the current software life cycle of the web. If, for instance, the majority of costs were moved from maintenance to initial build then it could be argued that more time is spent being creative instead of maintaining.

I have found that when working in HTML alone that I save incredible amounts of time when I develop only in XHTML 1.1, because the browser tells you where your errors are. … Challenges are removed and costs are reduced by pushing the largest cost challenges to the front of development.

… The typical wisdom is that people need to be empowered. If you would not think this way in your strategic software planning then why would it make sense to think this way about strategic application of web technologies? …

This might all sound very rational on one level, but a few points need to be made:

• If we’re not building technology for people, WTF are we doing with these CPU cycles exactly?
• Speed of iteration is a key enabler of progress in any technology stack I’ve ever worked in
• Strictness fails in the wild

I think Austin’s point about moving costs from maintenance to build is supposed to suggest that if we were only more strict about things, we’d have less expensive maintenance of systems, but it’s not clear to me that this has anything to do with strictness. My observation from building systems is that this has a lot more to do with being able to build modular, isolated systems that compose well. Combine that with systems that let you iterate fast, and you can grow very large things that can evolve in response to user needs without turning into spaghetti quite so quickly. Yes, the web isn’t great for that today, but strictness is orthogonal. Nothing about Web Components demands strictness to make maintainability infinitely better.

And the last point isn’t news. Postel’s Law isn’t a plea about what you, dear software designer, should be doing, it’s an insightful clue into the economics of systems at scale. XML tried being strict and it didn’t work. Not even for RSS. Mark Pilgrim’s famously heroic attempts at building a reliable feed parser match the war stories I’ve heard out of the builders of every large RSS system I’ve ever talked to. It’s not that it’s a nice idea to be forgiving about what you accept, it’s that there’s no way around it if you want scale. What Austin has done is the classic bait-and-switch: he has rhetorically substituted what works in his organization (and works well!) for what’s good for the whole world, or even what’s plausible. I see this common logical error in many a standards adherent/advocate. They imagine some world in which it’s possible to be strict about what you accept. I think that world might be possible, but the population would need to be less than the size of a small city. Such a population would never have ever created any of the technology we have, and real-world laws would be how we’d adjudicate disputes. As soon as your systems and contracts deal with orders of magnitude more people, it pays to be reliable. You’ll win if you do and lose if you don’t. It’s inescapable. So lets banish this sort of small-town thinking to the mental backwaters where it belongs and get on with building things for everyone. After all, this is about people. Helping sentient beings achieve their goals in ways that are both plausible and effective.

If helping people is not what this is about, I want out.

This is why I get so exercised about WebIDL and the way it breaks the mental model of JS’s “it’s just extensible objects and callable functions”. It’s also why my discussions with folks at last year’s TPAC were so bleakly depressing. I’ve been meaning to write about TPAC ever since it happened, but the time and context never presented themselves. Now that I got some of my words out about layering in the platform, the time seems right.

Let me start by trying to present the argument I heard from multiple sources, most likely from (in my feeble memory) Anne van Kestern Jonas Sicking(?):

ECMAScript is not fully self-describing. Chapter 8 drives a hole right through the semantics, allowing host objects to whatever they want and more to the point, there’s no way in JS to describe e.g. list accessor semantics. You can’t subclass an Array in JS meaningfully. JS doesn’t follow it’s own rules, so why should we? DOM is just host objects and all of DOM, therefore, is Chapter 8 territory.

Brain asploded.

Consider the disconnect: they’re not saying “oh, it sure would be nice if our types played better with JS”, they’re saying “you and what army are gonna make us?” Remember, WebIDL isn’t just a shorthand for describing JavaScript classes, it’s an entirely parallel type hierarchy.

Many of the Chapter 8 properties and operations are still in the realm of magic from JS today, and we’re working to open more of them up over time by giving them API — in particular I’m hopeful about Allen Wirfs-Brock’s work on making array accessors something that we can treat as a protocol — but it’s magic that DOM is appealing to and even specifying itself in terms of. Put this in the back of your brain: DOM’s authors have declared that they can and will do magic.

Ok, that’s regrettable, but you can sort of understand where it comes from. Browsers are largely C/C++ enterprises and DOM started in most of the successful runtimes as an FFI call from JS to an underlying set of objects which are owned by C/C++. The truth of the document’s state was not owned by the JS heap, meaning every API you expose is a conversation with a C++ object, not a call into a fellow JS traveler, and this has profound implications. While we have one type for strings in JS, your C++ side might have bstr, cstring, wstring, std::string, and/or some variant of string16.

JS, likewise, has Number while C++ has char, short int, int, long int, float, double, long double, long long int…you get the idea. If you’ve got storage, C++ has about 12 names for it. Don’t even get me started on Array.

It’s natural, then, for DOM to just make up it’s own types so long as its raison d’être is to front for C++ and not to be a standard library for JS. Not because it’s malicious, but because that’s just what one does in C++. Can’t count on a particular platform/endianness/compiler/stdlib? Macro that baby into submission. WTF, indeed.

This is the same dynamic that gives rise to the tussle over constructable constructors. To recap, there is no way in JS to create a function which cannot have new on the left-hand-side. Yes, that might return something other than an instance of the function-object on the right-hand side. It might even throw an exception or do something entirely non-sensical, but because function is a JavaScript concept and because all JS classes are just functions, the idea of an unconstructable constructor is entirely alien. It’s not that you shouldn’t do it…the moment to have an opinion about that particular question never arises in JS. That’s not true if you’re using magic to front for a C/C++ object graph, though. You can have that moment of introspection, and you can choose to say “no, JS is wrong”. And they do, over and over.

What we’re witnessing here isn’t “right” or “wrong”-ness. It’s entirely conflicting world views that wind up in tension because from the perspective of some implementations and all spec authors, the world looks like this:

Not to go all Jeff Foxworthy on you, but if this looks reasonable to you, you might be a browser developer. In this worldview, JS is just a growth protruding from the side of an otherwise healthy platform. But that’s not how webdevs think of it. True or not, this is the mental model of someone scripting the browser:

The parser, DOM, and rendering system are browser-provided, but they’re just JS libraries in some sense. With <canvas>‘s 2D and 3D contexts, we’re even punching all the way up to the rendering stack with JS, and it gets ever-more awkward the more our implementations look like the first diagram and not the second.

To get from parser to DOM in the layered world, you have to describe your objects as JS objects. This is the disconnect. Today’s spec hackers don’t think of their task as the work of describing the imperative bits of the platform in the platform’s imperative language. Instead, their mental model (when it includes JS at all) pushes it to the side as a mere consumer in an ecosystem that it is not a coherent part of. No wonder they’re unwilling to deploy the magic they hold dear to help get to better platform layering; it’s just not something that would ever occur to them.

Luckily, at least on the implementation side, this is changing. Mozilla’s work on dom.js is but one of several projects looking to move the source of truth for the rendering system out of the C++ heap and into the JS heap. Practice is moving on. It’s time for us to get our ritual lined up with the new reality.

Which brings me too David Flanagan who last fall asked to read my manifesto on how the platform should be structured. Here it is, then:

The network is our bottleneck and markup is our lingua-franca. To deny these facts is to design for failure. Because the network is our bottleneck, there is incredible power in growing the platform to cover our common use cases. To the extent possible, we should attempt to grow the platform through markup first, since markup provides the most value to the largest set of people and provides a coherent way to expose APIs via DOM.

Markup begets JS objects via a parser. DOM, therefore, is merely the largest built-in JS library.

Any place where you cannot draw a line from browser-provided behavior from a tag to the JS API which describes it is magical. The job of Web API designers is first to introduce new power through markup and second to banish magic, replacing it with understanding. There may continue to be things which exist outside of our understanding, but that is a challenge to be met by cataloging and describing them in our language, not an excuse for why we cannot or should not.

The ground below our feet is moving and alignment throughout the platform, while not inevitable, is clearly desirable and absolutely doable in a portable and interoperable way. Time, then, to start making Chapter 8 excuses in the service of being more idiomatic and better layered. Not less and worse.