This is why I get so exercised about WebIDL and the way it breaks the mental model of JS’s “it’s just extensible objects and callable functions”. It’s also why my discussions with folks at last year’s TPAC were so bleakly depressing. I’ve been meaning to write about TPAC ever since it happened, but the time and context never presented themselves. Now that I got some of my words out about layering in the platform, the time seems right.

Let me start by trying to present the argument I heard from multiple sources, most likely from (in my feeble memory) Anne van Kestern Jonas Sicking(?):

ECMAScript is not fully self-describing. Chapter 8 drives a hole right through the semantics, allowing host objects to whatever they want and more to the point, there’s no way in JS to describe e.g. list accessor semantics. You can’t subclass an Array in JS meaningfully. JS doesn’t follow it’s own rules, so why should we? DOM is just host objects and all of DOM, therefore, is Chapter 8 territory.

Brain asploded.

Consider the disconnect: they’re not saying “oh, it sure would be nice if our types played better with JS”, they’re saying “you and what army are gonna make us?” Remember, WebIDL isn’t just a shorthand for describing JavaScript classes, it’s an entirely parallel type hierarchy.

Many of the Chapter 8 properties and operations are still in the realm of magic from JS today, and we’re working to open more of them up over time by giving them API — in particular I’m hopeful about Allen Wirfs-Brock’s work on making array accessors something that we can treat as a protocol — but it’s magic that DOM is appealing to and even specifying itself in terms of. Put this in the back of your brain: DOM’s authors have declared that they can and will do magic.

Ok, that’s regrettable, but you can sort of understand where it comes from. Browsers are largely C/C++ enterprises and DOM started in most of the successful runtimes as an FFI call from JS to an underlying set of objects which are owned by C/C++. The truth of the document’s state was not owned by the JS heap, meaning every API you expose is a conversation with a C++ object, not a call into a fellow JS traveler, and this has profound implications. While we have one type for strings in JS, your C++ side might have bstr, cstring, wstring, std::string, and/or some variant of string16.

JS, likewise, has Number while C++ has char, short int, int, long int, float, double, long double, long long int…you get the idea. If you’ve got storage, C++ has about 12 names for it. Don’t even get me started on Array.

It’s natural, then, for DOM to just make up it’s own types so long as its raison d’être is to front for C++ and not to be a standard library for JS. Not because it’s malicious, but because that’s just what one does in C++. Can’t count on a particular platform/endianness/compiler/stdlib? Macro that baby into submission. WTF, indeed.

This is the same dynamic that gives rise to the tussle over constructable constructors. To recap, there is no way in JS to create a function which cannot have new on the left-hand-side. Yes, that might return something other than an instance of the function-object on the right-hand side. It might even throw an exception or do something entirely non-sensical, but because function is a JavaScript concept and because all JS classes are just functions, the idea of an unconstructable constructor is entirely alien. It’s not that you shouldn’t do it…the moment to have an opinion about that particular question never arises in JS. That’s not true if you’re using magic to front for a C/C++ object graph, though. You can have that moment of introspection, and you can choose to say “no, JS is wrong”. And they do, over and over.

What we’re witnessing here isn’t “right” or “wrong”-ness. It’s entirely conflicting world views that wind up in tension because from the perspective of some implementations and all spec authors, the world looks like this:

Not to go all Jeff Foxworthy on you, but if this looks reasonable to you, you might be a browser developer. In this worldview, JS is just a growth protruding from the side of an otherwise healthy platform. But that’s not how webdevs think of it. True or not, this is the mental model of someone scripting the browser:

The parser, DOM, and rendering system are browser-provided, but they’re just JS libraries in some sense. With <canvas>‘s 2D and 3D contexts, we’re even punching all the way up to the rendering stack with JS, and it gets ever-more awkward the more our implementations look like the first diagram and not the second.

To get from parser to DOM in the layered world, you have to describe your objects as JS objects. This is the disconnect. Today’s spec hackers don’t think of their task as the work of describing the imperative bits of the platform in the platform’s imperative language. Instead, their mental model (when it includes JS at all) pushes it to the side as a mere consumer in an ecosystem that it is not a coherent part of. No wonder they’re unwilling to deploy the magic they hold dear to help get to better platform layering; it’s just not something that would ever occur to them.

Luckily, at least on the implementation side, this is changing. Mozilla’s work on dom.js is but one of several projects looking to move the source of truth for the rendering system out of the C++ heap and into the JS heap. Practice is moving on. It’s time for us to get our ritual lined up with the new reality.

Which brings me too David Flanagan who last fall asked to read my manifesto on how the platform should be structured. Here it is, then:

The network is our bottleneck and markup is our lingua-franca. To deny these facts is to design for failure. Because the network is our bottleneck, there is incredible power in growing the platform to cover our common use cases. To the extent possible, we should attempt to grow the platform through markup first, since markup provides the most value to the largest set of people and provides a coherent way to expose APIs via DOM.

Markup begets JS objects via a parser. DOM, therefore, is merely the largest built-in JS library.

Any place where you cannot draw a line from browser-provided behavior from a tag to the JS API which describes it is magical. The job of Web API designers is first to introduce new power through markup and second to banish magic, replacing it with understanding. There may continue to be things which exist outside of our understanding, but that is a challenge to be met by cataloging and describing them in our language, not an excuse for why we cannot or should not.

The ground below our feet is moving and alignment throughout the platform, while not inevitable, is clearly desirable and absolutely doable in a portable and interoperable way. Time, then, to start making Chapter 8 excuses in the service of being more idiomatic and better layered. Not less and worse.

Your Moment Of Zen

The backdrop to this debate is that CSS is absolutely the worst, least productive part of the web platform. Apps teams at Google are fond of citing the meme that “CSS is good for documents, but not for apps”. I push back on this, noting the myriad ways in which CSS is abysmal for documents. That isn’t to minimize the pain it causes when building apps, it’s just that the common wisdom is that CSS surely must be fantastic for somebody else. Once we find that person, I’ll be sure to let you know. In the mean time we should contemplate how very, very far behind the web platform is in making it delightful to build the sorts of things that are work-a-day in native environments.

But it’s worse than simply being under-powered: CSS has the weakest escape hatches to imperative code and demands the most world-view contortion to understand its various layout modes and their interactions. Imagining a more congruent system isn’t hard — there are many in the literature, might I humbly suggest that now might be a good time to read Badros & Borning? — and even CSS could be repaired were we able to iterate quickly enough. Things have been moving faster lately, but fast enough to catch up with the yawning gap in platform capabilities? We’ll come back to the speed/scale of change later.

For now, consider that the debate (as captured by Nate Vaughn) is about the retrospective implications of the few things that have already gotten better for some set of developers in some situations. That this sort of meaningful progress (corners, gradients, animations, transitions, flexing boxes, etc.) is rare makes the debate all the more bone-chilling to me. We finally got a few of the long list of things we’ve been needing for a decade or more, and now because the future is unevenly distributed, we’re about to blow up the process that enabled even that modicum of progress? How is it we’re extrapolating causality about engine usage from this unevenness, anyhow? None of this is obvious to me. The credible possibility of losing prefixes as a way to launch-and-iterate is mind-exploding when you realize that the salient competition isn’t other browsers, it’s other platforms. Either the proponents of squatting other vendor’s prefixes haven’t thought this through very hard or they’re bad strategists on behalf of the web as a platform…not to mention their own products. The analysis that we’re being asked to accept rests on an entire series of poor arguments. Lets start with the…

Uncomfortable Assumptions

In an interview out yesterday with Eric Meyer, Tantek Çelik of Mozilla tried to present this debate as a question of barriers to the adoption of non-WebKit based browsers, specifically Firefox Mobile. Opera has made a similar case. What they ommit is that the only platforms where they can credibly ship such browsers are Android and S60 (a dead end). That’s a large (and growing) chunk of the world’s handsets — good news for me, as I now work on Chrome for Android here in London — but for whatever reason it appears that iOS users surf a whole lot more.

Let that sink in: on the devices that are the source of most mobile web traffic today, it’s not even possible to install a browser based on a different engine, at least not without a proxy architecture like the one used in the excellent Opera Mini or Amazon’s Silk. iOS and Windows Phone are both locked-down platforms that come with only one choice of engine (if not browser shell). When folks from the vendors who want to appropriate others’ prefixes talk about “not being able to compete”, remember that competition isn’t even an option for the most active users of mobile browsers. And it’s prefixes that are keeping us down? We must go deeper.

The tension comes into focus when we talk in terms of conversion, retention, and attrition. Conversions are users who, if able, switch to a new product from an old one. Retention is a measure of how many of those users continue to use the product after some period of time. Today (and since Windows first included a browser), the single largest factor in the conversion of users to new browsers is distribution with an OS. This is the entire rationale behind the EU’s browser choice UI, mandated on first start of new Windows installs. Attrition is the rate at which users stop choosing to use your product day after day, and for most desktop installed software, attrition is shockingly high after 3 to 6 months. The attrition rate is usually measured by cohorts over time; users who installed on the same day/week/month to measure what % of that group continue to use the product over increasingly long periods of time. The rate of decay falls, but the overall attrition invariably continues to rise. You might not get un-installed, but that doesn’t mean you’ll still be the go-to icon on the user’s home screen or desktop. Eventually every device is recycled, wiped, or left for history in a drafty warehouse and along with it, your software. A key factor in getting attrition under control for Chrome has been evangelism to improve site compatibility, e.g. “I’m not using your browser because my bank website doesn’t work with it”. That argument — that site compatibility is key to ensuring a competitive landscape for what otherwise are substitutes — puts the entire thing in some perspective. Attrition isn’t the same thing as conversion, and conversion is driven primarily by integrations and advertising. Implicit in the arguments by Tantek and others is a sub-argument of the form:

Our product would have measurably more users if sites were more compatible.

Thanks to what we know about what drives conversions, in the short run this is simply false. Long term, what invariably gives you more users is starting with more users. The set of things that are most effective at convincing users to swap browsers, even for a day, include: advertising, word-of-mouth, a superior product, distribution/bundling, and developer advocacy. Depressingly, only one of those involves actually being a better product, and the prerequisite for all of them is the ability to switch (thanks, Android team!). There’s a similar dynamic at work when doing advocacy to web developers: if you’re nowhere in their browser stats, they’re adding support for a standard or worse a second prefix in order to do service to a cause, not because it’s measurably good for them. Clearly, that’s going to be somewhat less effective. Where, then, is the multi-million dollar advertising campaign for Fennec? The carrier and OEM rev-share deals for bundling on new Android phones? Hmm. To hear Tantek et. al. tell it, non-WebKit-based browsers would be prevalent on mobile if only it weren’t for those damned browser prefixes causing users of other browsers to receive different experiences! Also, those kids and that damned dog.

Over the long haul compatibility can have a dramatic effect on the rate of attrition by changing the slope of the curve — which, remember, is a rate with decay and not a single % — but it begs the next uncomfortable question: what do we mean by “compatibility” here? What sorts of incompatibility cause attrition? Is it content that looks slightly worse but still essentially works (think grey PNG backgrounds on IE6) or does it simply turn you away, not allow you to play in any way, and generally fails (think the ActiveX dependencies of yore)?

Inaccessible or Ugly?

Eric was good enough to call out what I view as a key point in this debate: what sort of “broken” are we talking about? Tantek responded with a link to side-by-side screenshots of various properties rendered on Chrome for Android’s Beta and current Fennec. In some of these cases we may be looking at Fennec bugs. WordPress.com serves the same content to Fennec which seems to bork what float: left; means. That, or some media query is preventing the main blocks from being floated; it’s hard to tell which from a quick view-source:. For the versions of google.* included in the list, the front end is simply serving the desktop version to Fennec which makes the wonky button rendering even stranger. Is there room to improve what gets sent to Fennec? You bet, but that’s not what’s being argued in the main. Ask yourself this: is what you see on that page worth destroying the prefix system for? ‘Cause that’s what the advocates of prefix-squatting would have you believe. In effect, they’re suggesting that nothing will cause developers to test on non-pervasive engines, a deeply fascinating assertion. Even if we accept it, it doesn’t point clearly to a single tactic to resolve the tension. It certainly doesn’t argue most strongly for prefix-squatting.

An important point Eric failed to follow up on was Tantek’s assertion that Mozilla will be cloaking user-agent strings. Does he imagine that the only thing that might be cause someone to send different content is CSS support? API support for things like touch events differs, the performance characteristics of device classes and browsers vary wildly, and application developers are keen to deliver known-good, focused experiences. The endless saga of position: fixed; as plumbed by Google teams and others is a story of competing constraints: browser vendors optimize for content, content fights back. Repeat. What does Mozilla imagine is going to happen here? Maintained content will react to the browser usage of end-users (and as we’ve covered, compat != conversions). Unmaintained content, well, that’s what fallback is all about. And bad things deserve to lose. Assuming that your browser is 100% compatible with developer expectations and testing if you only switch the UA and implement a few prefixes is NPAPI-level crazy all over again, and it’s entirely avoidable. Tantek and Brendan, of all people, should be able to reason that far. I guess they’ll find out soon enough — although we will have all been made poorer for it.

Now, what about the related argument that Mozilla & Co. are only going to be doing this for properties which are “stable” (nevermind their actual standardization status)? The argument says that because something hasn’t changed in another vendor’s prefixed version in a while, it must be safe to squat on. Not only is this (again) incredibly short-sighted, it says that instead of forcing a standard over the line and clobbering both developers and other vendors with the dreaded label of “proprietary” (the usual and effective tactic in this situation), they’re instead willing to claim ownership and therefore blame for the spread of this soon-to-be-proprietary stuff, all the while punting on having an independent opinion about how the features should be structured and giving up on the standards process…and all for what?

Product vs. Platform

Perhaps there wasn’t space in Tantek’s interview with Eric, but both of them chose not to be introspective about the causes of WebKits use in so many mobile browsers, with Tantek merely flagging the use of a single engine by multiple products as “a warning sign.” But a warning of what, exactly? Eric didn’t challenge him on this point, but I sorely wish he had. Why did Safari, the Android Browser, Chrome, Silk, Black Berry, and many others all pick WebKit as the basis for their mobile browsers?

WebKit isn’t a browser. It’s just not. To make a browser based on WebKit, one might bring along at least the following bits of infrastructure which WebKit treats as bits to be plugged in:

• Networking
• Caches of some sort
• Graphics rendering
• A build system
• POSIX or other platform plumbing

And that’s a minimum. Competitive ports tend to include WebSQL, LocalStorage, and Indexed DB back-ends, video codecs, 3D infrastructure (deeply non-trivial), perhaps an alternative JavaScript engine (V8 or other), and alternative/additional image decoders (e.g., WebP). All of that is in addition to needing to create your own UI for bookmarking, navigation, etc. WebKit is an engine, not a fully-functioning vehicle. Therein may lay some of the difference in the relative success of the WebKit and Gecko on mobile to date. Want to build a Gecko-based browser? Great, first clone the entire Firefox codebase from Mercurial, then layer your stuff on top/around. Oh, wait, things might not cleanly be factored to allow you to plug in your own X, Y, or Z? Your builds take forever? Welcome to life in the Mozilla universe where your product is always second fiddle to Firefox. Now, that’s not by way of criticism, mind you. It is entirely reasonable for a product like Firefox not to pay coordination costs with other vendors/users of their code. God knows the overhead over here in WebKit land of trying to keep the menagerie of ports building against all changes is downright daunting. Mozilla (the organization) has made choices that prioritized the success of their product (Firefox) over their codebase (Gecko). WebKit was structured as platform only (no product), both forcing enormous costs onto every port while also freeing them from swimming upstream against a single product’s imperatives in the codebase.

What we’re witnessing isn’t open vs. closed, it’s differences in initial cost of adoption. In JS terms, it’s jQuery (focused core, plugins for everything else) vs. Sencha or Dojo (kitchen sink). Entirely different target users, and both will find their place. Nobody should be shocked to see smaller, more focused bits of code with good plugin characteristics spreading as the basis for new projects. The Mozilla Foundation wants to help prevent monoculture? In addition to making the Firefox product a success, there are concrete engineering things they can do to make Gecko more attractive to the next embedder, Firefox-branded or not. I haven’t heard of progress or prioritization along those lines, but I’m out of the loop. Perhaps such an effort is underway, if so, I applaud it. Whatever the future for Gecko, Product success isn’t related to platform success as a first approximation. Having a good, portable, pluggable core increases the odds of success in evolutionary terms, but it’s absolutely not determinant; see MSIE.

Speaking of IE…I respect those guys a lot, but the logical leap they’re asking us to swallow is that the reason people return Windows Mobile phones is that some CSS doesn’t work. That’s what attrition means on a platform where they’re the only native runtime. Data would change my mind, but it’s a hell of a lot to accept without proof.

The Time Component

Lets take a step back and consider Tantek’s claim that Mozilla has gotten very little traction in evangelizing multi-prefix or prefix-free development for the past year: Firefox for Android has been available since Oct. 2010 and stable for just 6 months. Opera Mobile on Android has been stable for just over a year. IE 9 (the only IE for mobile you could ever seriously consider not serving fallback experiences to) only appeared with Windows Phone 7.5 (aka “Mango”), shipped to users an entire 6 months ago.

And we expect webdevs to have updated all their (maintained) content? Never mind the tenuous correlation between the sorts of soft incompatibilities we’re seeing at the hands of CSS and user attrition; the argument that even this lesser form of harm hasn’t been blunted by evangelism appears suspect. Taking the incompatibilities seriously, I can quickly think of several other measures which are preferable to destroying the positive incentives towards standardization the prefix system creates (from least to most drastic):

• Continued evangelism to web developers with particular focus on major sites
• Political pressure on browser vendors to start dropping prefixes (i.e., we’d all be equally disadvantaged until users pick up the standard version)
• UA spoofing without prefix squatting
• Blacklists to trigger alternative identity (UA/prefixes) on a subset of sites

All of these are less blow-up-the-world than what MSFT, Mozilla, and Opera are proposing. It’s not even an exhaustive list. I’m sure you can think of more. Why these have either been not considered or dismissed remains a mystery.

It’s More Complicated Than That

In all of this, we’re faced with an existential question: what right do web developers have to shoot themselves in the foot? Is there a limit to that right? What sort of damage do we allow when some aspect of the system fails or falls out of kilter for some period of time? It’s a question with interesting parallels in other walks of life (for a flavor, substitute “web developers” with “banks”).

Can we show active harm to other browsers from the use of prefixes? The data is at best unclear. Arguing that any harm rises to a level that would justifies destroying the prefixes system entirely is rash. I argued many of the reasons for this in my last post, but lets assume in our mental model that developers respond to incentives in some measure. If, concurrently with achieving as-yet un-managed distribution, Mozilla et. al. implement others’ prefixes, what should we expect developers to do in response? After all, they will have reduced whatever tension might have been created by content that “looked wonky” and, where standards exist, will have reduced the incentive to switch to the standard version.

Now lets play the model one more turn of the wheel forward too: assume that Chrome or Safari (or both!) act in good faith and contemplate removing the -webkit-* prefix support for standardized features at a quick clip…and Mozilla doesn’t. You see how this quickly leads to a Mexican standoff: web developers won’t stop using prefixed versions because those are the way you get 100% support (thanks to Mozilla’s support for them); vendors won’t un-prefix things because others who squat their prefixes will then have a compatibility advantage; and nobody will be keen to add new things behind prefixes because they can no longer be assumed to be experiments that can change. Lose, lose, lose.

Some on the other side of the debate are keen to cite game theory as a support for their course of action, but the only conclusion I can draw is that their analysis must be predicated on a set of user and developer motivations that are entirely unlike the observable world we inhabit.

A Call To Reason, Not Arms

Based on a better understanding of the landscape, what should the various parties do to make the world better for themselves now and in the long run and for the web as a platform?

• Web Devs: first, do no harm; test in multiple runtimes, pointedly including a “fallback”. Then enhance with prefixes. Do not apologize for giving some (or even many) of your users a better experience. That, after all, is your job. But know this: prefixed properties are not supported, will go away, and when something you didn’t test the fallback for falls over, it’s your fault.
• Browser Vendors: invest in advocacy and distribution enhancing moves for your product before threatening to blow up effective standards policies. If you’re going to implement a prefixed version, please have a different opinion or push to ram a standard through to Recommendation ASAP. Incompatible right-hand-sides help developers understand that things are still evolving. DO NOT squat on prefixes. It’s both relative ineffective and will make developer’s lives harder when they want to legitimately move to standard or support your prefixes.
• Vendor CSS WG Reps: get it through your heads, you’re behind. It’s not quaint and it’s not excusable. The platform needs more powerful CSS features, and stat. It’s long past time to start stealing good ideas from the pre-processors. Appeals to a lack of manpower to implement must never block others and shouldn’t block standardization, so please stop making them. If you care about the platform’s success, let those who are able and willing to take risks do so.
• The CSS WG (as a whole): get the lead out. It’s not exclusively the W3C’s fault that things are slow, but the current MTTR (Mean Time To Recommendation) is still glacial. It is unreasonable to expect vendors to drop prefix support immediately upon standardization, but the W3C has a role here to advocate for quick sunsetting. Daniel Glazman is, as ever, right on most of this, but more can be done to streamline the process post CR.
• The WebKit Project: Add build flags to allow WebKit-based products to enable/disable vendor prefix support independently.
• Chrome/Safari/Other-prefix-supporting-browsers: Sunset prefixes as soon as is practicable post-standardization. Similarly, don’t ship prefixed features you’re not willing to be on the hook for via your reps to the CSS WG. Disabling them may be painful, but it’s the only good-faith thing to do.

Endnotes

I’ve left a lot out of this post, but it’s too long already. I do truly hope it’s the last I write on prefixes because, as I said up front, we have much bigger fish to fry. Stat. Prefixes do work, they’re essential to delivering a future platform that can compete, and yes, they should be torn down at the same pace they’re erected.

A few things that folks have asked about as tangents to this debate:

• It’s never a good thing for there to be homogeneity in the experimentation phase. The explicit goal of the prefixes system is to enable diversity of early opinion and fast coalescing around the best answer, thereby enabling the writing of a standard which is likely to need less revising and iteration. Diversity provides some value, the market tests the alternatives, and we deliver the most value we can over time through the standard version. It has always been thus, but prefixes make it less risky…assuming we don’t start stepping on everyone else’s toes.
• If the reasoning behind prefixes is to set up and tear down large-scale experiments, iterate, and collect feedback then Lea’s -prefix-free approach and PPK’s -alpha-*/-beta-* proposals are equally counter-productive and should be avoided at all costs. Making prefixes less painful to use reduces the natural incentives for migrating to a standard while blindly assuming the same right-hand for a future standard version as we have for some prefixed versions is plainly idiotic. What were they thinking?
• @-vendor-unlock is only slightly smarter, but in every possible way inferior to CSS Mixins. Would that the WG spent as much time on Mixins as they have on this prefix kerfuffle.
• Yes, I was in Paris when the CSS WG F2F was happening. No, I wasn’t at the meetings. Duty (Chrome for Android) called.
• If you’ve read this far, congrats. You may be the only one. I’ve been assured by CSS WG delegates that nobody cares what I think, which statistically seems to just be rounding down by a tiny bit. Fair enough.

Update: Michael Mullany of Sencha adds epicly good, epicly long context about what causes developers to target UAs and what the incentives that’ll change their minds about supporting a given browser really are.

Thanks to Frances Berriman, Jake Archibald, Tony Gentilcore, and Tab Atkins for reviewing earlier versions of this post. Errors of fact and form are all mine, however. Updated occasionally for clarity and to remove typos.

Which it is not.

Before I get to why, let me first say some nice, well-earned things about WebIDL: first, it has helped us out of the ad-hoc IDL sludge that used to be how APIs for JavaScript have been exposed in the past. It has shaved off many sharp edges and is giving spec authors a single dialect in which to write their API descriptions. From a browser perspective, this is a Very Good Thing (TM). Next, the draft in question contains some wonderful changes from the status quo, particularly the addition of a sane prototype to all WebIDL-specified objects.

That all sounds good, so what’s missing?

In a word, constructors.

Well, a lot more than that, but I’d settle for constructors. Functionally speaking, it boils down to the fact that WebIDL makes spec authors do extra work to make something like this sane:

new HTMLDivElement();

Why doesn’t this work today? Funny story…see, HTML defines HTMLDivElement as a regular WebIDL interface. WebIDL doesn’t really have the notion of concrete classes, just interfaces with and without constructors. Since the HTML spec is just doing what most specs will do — adding the smallest IDL you can get away with — the JS usability of this API is left in limbo; neither clearly HTML5′s responsibility nor WebIDL’s.

So what should a contentious version of HTML5 do? One answer is to specify a constructor, turning the IDL from this:

interface HTMLDivElement : HTMLElement {};

to this:

[Constructor]
interface HTMLDivElement : HTMLElement {};

Repeat ad-infinitum for each and every interface that should be constructable in every single spec that browser vendors ever implement. Don’t miss any! And please make sure that all your spec editors are on-board with good JS APIs as a goal! As it stands today, WebIDL doesn’t even force most spec authors to consider the question “do I need a constructor here?” — spoiler: yes — let alone the obvious follow-ups like “what arguments should one take?”.

The obvious better answer here is to flip the default on interfaces, causing them to generate constructors by default unless turned off with [NoConstructor] attributes or specified as partial interfaces (i.e., mixins or traits).

Cameron McCormack who is heading up the WebIDL effort tweeted in response to my exasperation that:

I think a “W3C Web API design guidelines” document would be a perfect place for such a recommendation.

For serious? Such a document might be useful (and I’m working on something that might pass as a first draft), but what’s the argument against flipping the default here? This isn’t a dissent on the facts of the situation: most WebIDL “interfaces” that are exposed to JS are things that could be easily new‘d up to useful ends. Most specs flub this in spectacular style. Most spec authors seem entirely ignorant of the problem and the design language of WebIDL continues to lead down a primrose copy-and-paste path that has little overlap with sanity. So why punt the decision? And why did it take and act of coordination with TC39 to get the prototype thing fixed?

And Why Are We Having This Discussion Anyway?

WebIDL, for all of its virtues, is deeply confused.

If you’re reading any of the stuff in the HTML5 spec that’s describing its API this way, it’s hard to see how it would have any sane relationship to JavaScript. Sure, you could argue that there might be other languages that matter, other languages for which you’d need to be able to generate some API, but none of them rise to anything like the importance of JavaScript. It is the programming language of the web, so if WebIDL has any animating force at all, it’s JS. Then there’s the “accident of history” aspect. Early DOM was specified as a form of IDL in part because there was some expectation that other languages would need to consume it and IDL was how C++ hackers (who still make up the entire set of people working on browser engines) are/were comfortable in describing their FFIs thanks to the legacy of COM/CORBA. Hilarious examples of multi-language-ism still persist in the WebIDL spec for no apparent reason whatsoever, warping the entire design around the altar of an ideal that is either quixotic or vestigial depending on which argument you give more weight.

Since the debate was re-kindled thanks to a debate at a TC39 meeting in July, I’ve been on the receiving end of more than one webdev’s rant about DOM’s JS incoherence, generally taking the form:

Why the *#!*?^@$ isn’t DOM just #@!*@ing specified in JavaScript?

To be honest, I have no answer aside from pointing to the IDL history, the fact that browser hackers don’t tend to write JS so don’t feel the pain, and noting that WebIDL is better in some important ways. Certainly these interfaces could be specified in a subset of JS with annotations for where native behavior is required. But their larger lament has merit too: seamlessness with JS is the bar WebIDL should be judged by. I.e. does it help spec authors do the right thing by JS devs? Or does it lead them down paths that make their generated APIs stick out like sore thumbs, full of anti-social/alien behavior such that you can’t think of them as “regular” JS?

Yes, constructors are only one minor step toward reaching this aspiration, but the fact that WebIDL has gotten to last-call without a reasonable solution to them speak volumes. If WebIDL isn’t animated by the need of JS developers, it would be good if that could be loudly called out somewhere so that the community can have the spirited debate that this point demands. If it is, can we please get on discussing how best to ensure that most “interfaces” generate constructors and stop punting?

Either way, WebIDL isn’t done yet.

Update: It occurred to me, as part of the discussion in the comments, that the provision against new with a class or type of any type is completely non-sensical in JS, as is the lack of call() and apply() methods on them. Idiomatic subclassing requires that the mixin-style be available, which uses ClassName.call(this). This is what you’d do with things that are “virtual” or “partial” interfaces if you’re describing them in actual JS. And there’s no real problem with folks new-ing them up. Doesn’t happen, doesn’t matter. Strong restrictions against it are, to quote Andrew DuPont, anti-social. Long story short: there is absolutely no reason whatsoever to disable construction on any WebIDL interface. It’s deeply non-sensical from the JavaScript perspective.

It’s good to see MSFT belatedly trying to put IE6 out to pasture, but what about IE 7? Or 8? Lets take stock of where we really are and where we’re likely to be in the next couple of years. First, remember that there’s no IE 9 for Windows XP — an OS that’s currently the most popular in the world — and no matter what happens with IE 6, IE 8 is the end of the upgrade road for XP. Unless you think half of the world’s computers will be replaced/upgraded in the next couple of years, it seems likely that IE 8 will be with us for the foreseeable future.

And what about the folks who do get IE 9? Well, so far, there’s nothing to make me believe that the uptake rate will be anything better than the IE 8 transition; a process which has taken 2 years to give ~30% of the market the latest version. If anything, we should expect that rate to be retarded somewhat by the XP hurdle.

MSFT’s browser replacement rates bear understanding because they’re the most popular and suffer from the longest half-lives. That is to say, the time it takes for an old version of IE to decay in the wild is much, much higher than for other browsers. Some part of this is surely due to sheer market share, but not all of it. The XP hurdle, for instance, is a form of structural drag on uptake rates — a flaw that browsers that aren’t tightly tied to OSes don’t suffer from. For web developers, I dare say that half-life of popular browsers matters much, much more than the current or trending market share since it’s predictive of our potential for browser improvement in the near future. It’s one thing to get the new shiny, but how long will it take you to install it? If the shiny is old and dingy by the time it’s in place, what good is that? It’s this lens that makes browser market share stats interesting to me; i.e., what percentage of the web’s users will get the new features soonest? ‘Cause those are the folks we can start building super compelling content for.

The average half-life of the majority of browsers in the wild also gates the rate of progress in standards. When the process is working well, bugs in browsers or pre-standards implementations of features aren’t a permanent features of the landscape. Instead, they’re the understandable and inevitable result of a process that prioritizes implementation experience and iteration over raw compliance with an academic spec that may or may not actually get it right on the first go ’round. But that iterative, feedback-rich process only works when browsers iterate quickly and web developers can target the future without thinking so hard about the past, else progress simply turns into something to resent and distrust. That’s good for no one, and a shorter half-life is the key to making progress more than just a spec-tease.

I’m personally hopeful that when IE 9 is finally RTM’d, that it includes some provisions for shortening its life expectancy in the ways that Chrome and Firefox have through aggressive auto-updating. Getting IE 9 out to the world will be a good thing, but only if it happens quickly and if IE 10 can follow it even faster.

There’s obviously a lot more to talk about at the browser panel — Chrome 10 just launched with Crankshaft, for instance — but the fact that nearly every Chrome user will have those improvements this week and that if you’re building a Chrome Web Store app, you’ll get to target those improvements nearly instantly seems like the biggest, most interesting change from where we were just a couple of years ago.

How bad is it? The next time you read some tech journalist write about how some new browser version is just around the corner and how it’ll make everything better, remember that:

• 50+% of Windows users are on XP a year after Windows 7 shipped and 3.5 years since Vista shipped
• Windows XP will be supported until 2014, giving organizations on XP extra breathing room to limp along on IE 6-8
• There will be no IE 9 for Windows XP
• After something like 4 years of MSFT urging customers in the strongest possible language to get off of IE 6, it still has 16% of the market, and it’s not falling nearly fast enough. At the current 0.75%/month dropoff, we’re looking at 20+ more months of IE 6. At this rate, kids born today will be walking and maybe even talking by the time we can write IE 6 into the history books.

The goal here isn’t to drive you to drink, but to call out the dichotomy between when users get features and when developers can address features. For users, things tend to get better as soon as you pick up a new browser. Developers have to wait until every user makes that sort of choice. Said differently, users can benefit from the bleeding edge but developers are beholden to the late adopters.

We should cut tech journalists and users a little slack, though; in a world without adoption friction, the launch of a new feature would translate directly into the sort of “now you can build sites and apps with awesome thing X”. The instinct to believe that’s how it should be is spot on. But that’s observably not how the world works. Don’t believe the hype. New browsers alone haven’t fixed the problem so far, so what makes us think they will in future? No vendor wants to talk about their last version, but for web developers stuck in the trenches, that all there is to talk about. That’s where the pain is, after all, and counting on browser upgrades to fix the problem quickly isn’t working well enough. Chrome’s aggressive auto-update feature is changing the way things will work in future, but for now we’re still stuck in the slow-upgrade dynamic.

We need a Plan B.

As far as I know, it was a total surprise to the Dojo community (myself included). I can’t wait to get started writing apps for this thing and see what device APIs Palm has surfaced.

In the past couple of years the shift from desktop-centric to a more web-centric OSCON has continued to make the conference useful and engaging, and great work on topics like JavaScript/Ajax performance, Dojo, Comet, and many of the emerging back-end bits of infrastructure that make it all go have made my yearly trip to Portland worthwhile.

Great talks have gotten lost from the JavaScript/web track in years past because they’ve missed the submission deadline, so if you’re hacking on something fascinating, now’s the time to get that proposal in. Make sure you flag it with the right track when you submit (javascript, ajax, web, or whatever they’re calling it this year), and don’t hesitate to ping me if you’re unsure about whether or not your talk got slotted correctly for the review process.

Maybe.

These things matter only in regards to how well they represent end-user workloads and how trustworthy their findings are. The first is much harder than the second, and end-to-end benchmarking is pretty much the only way to get there. As a result, sites like Tom’s Hardware focus on application-level benchmarks while still publishing “low level” numbers. Venerable test suites like SPECint have even moved toward running “full stack” style benchmarks which may emphasize a particular workload but are broad enough to capture the wider system effects which matter in the real world.

Marketing departments also like small, easily digestible, whole numbers. Saying something like “200% Faster!” sure sounds a lot better than “on a particular test which is part of a larger suite of tests, our system ran in X time vs. Y time for the competitor”. Both may be true, but the second statement gives you some context. Preferably even that statement would occur above an actual table of numbers or graphs. Numbers without context are lies waiting to be repeated.

With all of this said, James Ward’s Census benchmark makes a valiant stab at a full-stack test of data loading and rendering performance for RIA technologies. Last month Jared dug further into the numbers and found the methodology wanting, but given some IP issues couldn’t patch the sources himself. Since I wasn’t encumbered in the same way I thought I might as well try my hand at it, but after hours of attempting to get the sources to build, I finally gave up and decided to re-write the tests. The result is Census 2.

There are several goals of this re-write:

• Fairness. Tests need to be run multiple times for them to be representative in any way. Likewise, systems not being directly tested need to be factored out as much as possible. C2 does this by reducing the number of dependencies and running tests (at least) 5 times and discarding outliers before reporting an average. I’ve also worked to make sure that the tests put the best foot forward for all of the tested technologies.
• Hackability. Benchmarks like Census serve first as a way for decision makers to understand options but second as a way for developers to know how they’re doing. Making it trivial to add tests helps both audiences.
• Portability. The test suite should run nearly everywhere with a minimum of setup and fuss. This ensures that the largest numbers of people can benefit from the fairness and hackability of the tests.

The results so far have been instructive. On smaller data sets HTML wins hands-down for time-to-render, even despite its disadvantage in over-the-wire size. For massive data sets, pagination saves even the most feature-packed of RIA Grids, allowing the Dojo Grid to best even XSLT and a more compact JSON syntax. Of similar interest is the delta between page cycle times on modern browsers vs their predecessors. Flex can have a relatively even performance curve over host browsers, but the difference between browsers today is simply stunning.

Given the lack of an out-of-the-box paginating data store for Flex, RIAs built on that stack seem beholden to either Adobe’s LCDS licensing or are left to build ad-hoc pagination into apps by hand to get reasonable performance for data-rich business applications. James Ward has already exchanged some mail with me on this topic and it’s my hope that we can show how to do pagination in Flex without needing LCDS in the near future.

The tests aren’t complete. There’s still work to do to get some of the SOAP and AMF tests working again. If you have ideas about how to get this done w/o introducing a gigantic harball of a Java toolchain, I’m all ears. Also on the TODO list is an AppEngine app for recording and analyzing test runs so that we can say something interesting about performance on various browsers.

Census 2 is very much an open source project and so if you’d like to get your library or technology tested, please don’t hesitate to send me mail or, better yet, attach patches to the Bug Tracker.

Update: I failed to mention earlier that one of the largest changes in C2 vs. Census is that we report full page cycle times. Instead of reporting just the “internal” timings of an RIA which has been fully boostrapped, the full page times report the full time from page loading to when the output is responsive to user action. This keeps JavaScript frameworks (or even Flex) from omitting from the reports the price that users pay to download their (often sizable) infrastructure. There’s more work to do in reporting overall sizes and times (“bandwidth” numbers don’t report gzipped sizes, e.g.), but if you want the skinny on real performance, scroll down to the red bars. That’s where the action is.

The truly dispiriting thing, though, is how badly cmd.exe still sucks. I fully admit that my personal programming proclivities are not normal, but to be reasonably productive I need a Unix-like shell, a terminal that works (can be resized, has reasonable VT100 emulation, etc.), and the ability to fix the “Caps Lock” key to do the right thing – namely, have it fire the “Ctrl” key instead. This is all relatively straightforward to do on Linux and OS X. Here’s how I got it done with Windows:

Do the MSFT Patch Dance

Make coffee?

Install Cygwin

We’ve all done it a thousand times. This’ll make 1001. It’s kind of comforting that the Cygwin home page hasn’t changed perceptibly in nearly a decade.

Get SharpKeys

Instead of fugly registry hacks, SharpKeys allows you to map the dreaded and useless “Caps Lock” key to something actually useful. If your key-mapping preferences swing some other way, SharpKeys can likely handle that too. Not sure why it’s not built into Windows, frankly.

Set Up Puttycyg

Having cygwin is nice, but having a terrible shell with Cygwin? Not so nice. Enter Puttycyg, a small hack on the venerable Putty SSH client for windows that provides an option to launch a local Cygwin session in lieu of connecting to another system.

Once I extracted it and ensured the Puttycyg directory was in my windows PATH, I created a desktop shortcut to the putty.exe included in the distribution and configured the shortcut (right-click) to read:

"C:\Documents and Settings\slightlyoff\Desktop\puttycyg\putty.exe" -cygterm -

And then set the “Shortcut key:” to be:

Ctrl + Alt + T

Now, whenever I want a fully functional shell from my desktop, I just hit that key combination and it All Works (TM).

Today, Dojo’s delegate() function is a straightforward implementation of the Boodman/Crockford delegation pattern which Doug calls “beget” and which ES 3.1 will refer to as Object.create:

1
2
3
4
5
6
7
8
9
10
11
12

dojo.delegate = (function(){
    // boodman/crockford delegation w/ cornford optimization
    function TMP(){};
    return function(obj, props){
        TMP.prototype = obj;
        var tmp = new TMP();
        if(props){
            dojo._mixin(tmp, props);
        }
        return tmp; // Object
    }
})();

This function returns a new object which looks to the old object for things it does not itself have. Imagine an object foo which contains pithy truisms:

var foo = {
  science: "rocks!",
  learning: "is how you know you're alive"
};

We now want to promigulate our opinions, so we can delegate the responsibility of forming them:

var bar = dojo.delegate(foo, {
  testify: function(){
    console.debug("science ", this.science, "and learning", this.learning);
  }
);

Now, our bar object can change its mind independently of foo, but until it does, it’ll behave as though foo‘s views are its own:

bar.testify(); // outputs: "science rocks and learning is how you know you're alive"
 
// bar refines its opinion
bar.science = "is a process"; 
bar.learning = "requires humility";
foo.science == "rocks!"; // still true
 
bar.testify(); // outputs: "science is a process and learning requires humility"

But what about when the chain gets deeper? The fact that bar can’t “see” foo‘s values via this isn’t much of a problem when the hierarchy isn’t very long, but if you’re specializing a behavior or complex interaction, making it possible to get at the parent’s values for properties and methods becomes more pressing.

Neil has previously written about lightweight subclassing, but for as good as it its, it doesn’t get us all the way there either. In regular OO-style languages, the inheritance system gives you an out via a “super” keyword or convention. This type of property shadowing-with-exceptions is a huge boon to composition in class-based languages, but it’s not the whole story. Indeed, the Traits paper was all about the shortcomings of this special-purpose mechanism. What we want for both long delegation chains and long inheritance hierarchies is a more general system; in essence a way to say “I want to control how things are shadowed and which ones an item points at in each level of the hierarchy”.

What if we could make delegate() savvy of this type of indirection? Here’s my quick prototype:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

delegate = (function(){
    var tobj = {};
    var TMP = function(){};
    return function(obj, props){
        TMP.prototype = obj;
        var tmp = new TMP();
        if(props){
            var remaps = props["->"];
            if(remaps){
                delete props["->"];
                for(var x in remaps){
                    if(tobj[x] === undefined || tobj[x] != remaps[x]){
                        if(remaps[x] == null){
                            // support hiding via null assignment
                            tmp[x] = null;
                        }else{
                            // alias the local version away
                            tmp[remaps[x]] = obj[x];
                        }
                    }
                }
            }
            dojo.mixin(tmp, props);
        }
        return tmp; // Object
    }
})();

This new version of delegate() accepts a specially named “->” property in the list of items to add to the destination object. Items in this list can either “shadow null” (hide entirely) the parent’s property or can provide a new name for it, assuming of course that the new object will also have a property of that name. Here’s a quick example of “->” at work with our previous example. This time, foo also has a “testify” method that we’d like bar to be able to control without having to copy the implementation:

var foo = {
    science: "rocks!",
    learning: "is how you know you're alive",
    testify: function(){
        console.debug("science ", this.science, "and learning", this.learning);
    }
};
 
var bar = delegate(foo, {
    "->": {
        "testify": "grampsSays" // maps foo's "testify" to bar's "grampsSays"
    },
    testify: function(){
        if(this.science && this.learning){
            this.grampsSays(); // call the re-named "testify"
        }else{
            console.debug("this object is strikingly ignorant");
        }
    },
});
 
bar.testify(); // outputs: "science rocks and learning is how you know you're alive"
bar.science = false;
bar.testify(); // outputs: "this object is strikingly ignorant"

That New Object Smell

The last missing piece of the hierarchy pie here is that there’s no initializer for the objects which come from a delegation. A simple addition of some property detection code to look for an initializer can easily handle that:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

delegate = (function(){
    var tobj = {};
    var TMP = function(){};
    return function(obj, props){
        // boodman/crockford delegation w/ cornford optimization. 
 
        TMP.prototype = obj;
        var tmp = new TMP();
        if(props){
            var remaps = props["->"];
            if(remaps){
                delete props["->"];
                // like dojo.mixin(), except w/o key/key mapping
                for(var x in remaps){
                    // "safe" copy properties
                    if(tobj[x] === undefined || tobj[x] != remaps[x]){
                        if(remaps[x] == null){
                            // support hiding via null assignment
                            tmp[x] = null;
                        }else{
                            // alias the local version away
                            tmp[remaps[x]] = obj[x];
                        }
                    }
                }
            }
            dojo.mixin(tmp, props);
        }
 
        // support for "constructor" functions. The name "init" is arbitrary.
        if(typeof tmp["init"] == "function"){
            tmp.init.call(tmp);
        }
 
        return tmp; // Object
    }
})();

And there we have it. A style of delegation that easily supports both Trait-like name aliasing (and null shadowing) as well as internal initializers. Since our upgraded delegate can handle nulling out a parent’s value for a property, we also have a straightforward way to prevent parent initializers from being called (or being called/chained – at our discretion – by a new name):

var foo = {
    science: "rocks!",
    learning: "is how you know you're alive",
    testify: function(){
        console.debug("science ", this.science, "and learning", this.learning);
    }
};
 
var bar = delegate(foo, {
    init: function(){ this.testify(); }
});
// outputs: "science rocks and learning is how you know you're alive"
 
var baz = delegate(bar, {
    // map away the parent's constructor
    "->": {
        "init": "superInit"
    },
    // provide our own constructor
    init: function(){
        console.debug("howdy!");
        this.superInit(); // call the super-object ctor
    }
});
// outputs: "howdy", "science rocks and learning is how you know you're alive"
 
var thud = delegate(baz, {
    "->": { "init": null } // hide the parent ctor
});
// outputs: nothing

This form of delegate is likely to appear in Dojo 1.3 along with similar improvements to dojo.declare() to help alleviate the composition problems associated with using complex sets of mixins.

Update: corrected the null-out branch and updated the text with Doug’s note that beget/delegate will be called Object.create() in 3.1.

While in Boston, Gavin and Jill joined a gaggle of Dojo hackers at a dinner ostensibly to mourn my birthday (thanks to Dylan and Pete for organizing!) and in the course of conversation Jill asked something along the lines of:

So why do people get so excited about closures?

Which prompted a several of us to flail and flop gasping the salt flats of analogy like fish out of polite-conversation water. After about 10 minutes of this, Jill succinctly summed it all up in the form of a question:

Oh, so it’s like “action-oriented programming”?

This is perhaps the most insightful and succinct description I have ever heard of what JavaScript is all about.

Update: Jennifer just played this for me and it gets right to the heart of this post: the important part of doing what we do with computers, and more importantly, with the web, is to give the power of Computer Science to real people…and it starts with insights like Jill’s that build a shared way of thinking and talking about the world. It makes me sad that many programmers miss that, but when non-programmers can share in the beauty and power of code, it does a lot to make it all seem worthwhile.

At the risk of talking too much out of school, I want to set the record straight in some ways. First, let me set some facts out:

• ES4, as outlined last fall, was not ActionScript 3. Major changes to the semantics of Adobe’s initial contribution ensured that any truly compliant ES4 implementation from Adobe would have required many concessions, including the inclusion of a client-side compiler. ES4 without eval() (ala AS3) would never have cut it as a “real” ES4.
• Adobe had not donated “ActionScript 3″ to Mozilla nor Open Sourced ActionScript 3. Adobe had donated a high-performance byte-code VM which a separate front-end compiler targeted. It is absolutely possible that the Tamarin VM can and will run whatever syntax for the next version of JavaScript is ratified. The design of the VM is, of course, predicated upon the language semantics but lets not confuse the front-end compiler and the language that it consumes with the VM and byte-code format which it executes. The front-end compiler was not donated to Mozilla (although something similar exists in the OSS’d Flex SDK) nor does it ship inside of the Flash player today. Adobe can continue to evolve their developer languages and byte-code VMs independently so long as they never ship an eval() feature. Even if they do, there are other (harder to swallow) options available, but no road is cut off to Adobe which was not already long-since abandoned in other ways by the ES4 process.
• Like Adobe, Microsoft has jumped ahead in the evolution of JavaScript via the seemingly forgotten JScript.NET. Like Adobe, Microsoft has had to come back from that position to meet the web where it really is. Microsoft now ships 3 – yes, that’s right, three – JavaScript-ish languages which are capable of running in a browser:
  • JScript (via Windows Scripting Host)
  • JScript.NET
  • JScript for the DLR (via Silverlight)

  Make no mistake about it, these are all separate implementations which likely share little if any code. The DLR variant is even new in the last several years. Creating a new JavaScript compiler and runtime is not an overly onerous task for MSFT and clearly not one that will take a long time should the occasion arise. What’s confusing and can likely be tied to strategic wrangling is the puzzling lack of progress in the WSH version of JScript (the one everyone uses day-to-day). Any strategic discussion of JScript as a platform needs to start from this perspective.

• Much in ES4 was new. The gradual typing system (which I’m a big supporter of) was something of a large-scale experiment coupled with as much rigor as I’ve ever seen in a standards body in proving things out in an RI. The Class system (which I wasn’t a big supporter of) needed to marry class-based inheritance to prototypal inheritance in ways which were new. ES4 contained many good features and syntactic forms which were borrowed from other places, but as Brendan Eich has said, it was a process of synthesis plus invention. In my opinion, much of the failure of ES4 as an initiative can be traced to significant failings of ActionScript 3 as a language. AS3 is a Java-like language with a Java-like execution model. JavaScript is a dynamic language with a dynamic execution model. The gulf in expectations between those camps turns up in every corner of the language design. For instance: what does it mean to load new code? Are classes forever “closed” or can they be re-opened? Do you really need the static and final keywords? Solving these issues in the context of the old is easy. In the context of the new, it’s hard. ES4 suffered because it had to do it in a new world which no one was yet writing code for.

So, lets pop up and talk about strategy for a minute. Fundamentally, very little has changed in terms of available strategic options for any of the players:

• MSFT can still hold the web hostage to their ailing WSH VM by continuing to ignore its performance, regardless of bug fixes and syntactic updates. Doesn’t matter if it’s amputation or debilitating arthritis, crippled is crippled. For what it’s worth, my interactions with the MSFT reps on TC39 give me no reason to believe that they won’t be improving their VM.
• Adobe can still chose to implement a language which implements an ECMA spec. They can do this any time they damn-well please. It may not align so cleanly with their current technology roadmap, but it’s absolutely feasible and when Mozilla ships a “regular JavaScript” front-end to the Tamarin back-end, it’ll be even easier. Note that this plan can be done in parallel with AS3 evolution since Tamarin (as a VM) need not be aware of any language semantics on its own.
• Browser vendors can still wring large speedups out of ES3 and 3.1

What died here wasn’t Adobe’s attempts to “own” a spec. If there were such hopes in play, they had been quietly put down one rational, backwards compatible decision after another in the year preceding the Oslo meeting. What died was an assumption that the web can evolve without implementations being out in front of the spec. AS3 was one implementation of a JavaScript-like language that might have been a contender for crown of “the next one”, but so was JScript.NET. That neither of them “won” simply because they had been built in good faith is as true a sign as I can find that the process is working. Adobe gets it. Lets end the silly meme that “Adobe lost” or that “Microsoft won”. The game has hardly begun and it won’t be settled in a standards body anyway. What matters – and what we all need to keep our eyes keenly trained on – is what the big implementations do in the way of compatibility, performance, and feature set once ES3.1 arrives.

I’m incredibly excited about the SitePen Dojo Toolbox AIR app that just launched. I’ve been using early versions for a couple of weeks now, and in that time it has earned a privileged place on my desktop. Being able to make local builds without delving to the command line is something that I’ve wanted to be able to show new Dojo users for years and this tool finally makes it easy. There’s more work to do around configuring the profiles themselves, but the new tool demystifies many of the build configuration options significantly. Having a searchable API viewer available is also a godsend. Huge props to the team that put it together!

Edit: I forgot to note one of my favorite parts of the project, namely that because the Dojo build system is written in JavaScript (thanks to James Burke‘s awesome work), the build system in the Toolbox doesn’t require Java. Instead, SitePen was able to port the few Rhino dependencies in the build tool to work with the native JavaScript engine in AIR. It’s a minor thing, but it speaks to the excellent engineering behind Dojo’s tooling and the power of having actual web-native technologies inside a desktop app.

Fundamentally the Dojo and Zend teams really “get” each other. Both are deep packages that give you an opinion about how best to do something but also all of the tools you’ll need to make it work at scale. The “use at will” term that the Zend folks use made immediate sense to us. Like Dojo, Zend doesn’t saddle you with more than you’re really going to need up-front, but at the same time, when you need something awesome which is well tested and integrated, it’s right there. No digging around on google to find a “plug in” that will get you where you want to go…both Zend and Dojo give you a full stack of great components to work with out of the box.

There have been some questions on IRC today about why Dojo and not something else, and we know that the Zend folks are committed to continuing to allow Zend to work with other frameworks as well and we fully support them in that. There seem to be lots of choices of Ajax frameworks which Zend could have integrated, but when you look at the requirements of serious products which need to ship tested solutions to really hard problems, the field whittles down very fast. In response to the needs of our users, both Zend and Dojo take seriously our responsibilities to provide integrated, high-quality, unambiguously licensed products that will let user scale both up and down. Key to this is understanding the full spectrum of use-cases, informed by past experience, and striking a balance between enterprise-ready features and close-to-the-metal primitives. The Zend Framework has a larger view of the server-side than Dojo can, and as a result there are new opportunities to optimize and improve the user experience for all classes of users through this integration. This isn’t just about including some scripts on a page, this integration is about improving user and developer experiences, and both Dojo and Zend bring a lot to the table which can compliment the other. Dojo’s strengths in progressive enhancement, packaging, localization, and accessibility all have obvious allegories in the ZF world where a complete integration can more value based on what developers are already doing. All of those features of Dojo will work better when the server-side knows how to “hint” things for the client and work with, not against, the client to deliver better experiences.

I’m perhaps most excited about the data-driven opportunities in the Zend/Dojo integration. Dojo’s data infrastructure is second-to-none, and the design of the dojo.data and dojo.rpc layers provide Zend Framework integration the ability to take advantage of systems like the incredible Dojo Grid and the client-side charting package. There’s more to look forward to, and figuring it out together with the folks at Zend is a great opportunity for the Dojo community.

Pete Higgins and I will be participating in next Tuesday’s discussion with a broader chunk of the Zend Framework world, but until then (and long after) we’ll be hanging out in the #dojo and #zftalk channels on irc.freenode.net. We’re looking forward to working more with the ZF community to build great experiences, and are hugely excited about the direction things are already taking!

As they started talking about the platform and what’s part of it (and what’s not), I couldn’t escape escape the feeling that they’d gotten it right. It is absolutely the case that for most apps, scaling requires some amount of re-architecting. Systems like Rails are built with such a dependence (intentional or not) on the features of relational data stores that they quickly hit bottlenecks because frameworks aren’t keeping developers out of the gutter. This is nearly the same lesson that the security community collectively came to when it started to beat the average developer about the head regarding the awesome power of defaults. What systems do and don’t do for you “cheaply” defines their character, and in many systems those choices aren’t made consciously, or if they are, they don’t have the benefit of a different perspective which might de-emphasize certain traits. Call it libertarian paternalism, choice architecture, or pure condescension, but whatever it is, systems and platforms today are in the explicit business of making some things easier than others.

As the presenters showed how to make an app quickly, I knew I was looking at something I hadn’t seen the likes of since Jot. We all know that Big Table is a column-oriented data-store, but since most people are still stuck on the likes of MySQL, it’s hard to appreciate how liberating it is not having to think about how adding properties to models will affect a schema. The way App Engine is constructed lets the data store layer provide something called expando models. These models give us the kind of flexibility that I’ve only ever enjoyed before inside of Jot. Want a property? Just add it. No migration, no schema version…just data finding a happy home, and as your app’s skeleton “solidifies” and you figure out which properties really do need to be faster, you can migrate that data into fixed properties with indexes and types and all that jazz. It’s like a gradual type system, only for data stores. It’s agility nirvana for application development.

Speaking of application development nirvana, they also had the good sense to start with a great language (Python) and a great webapp framework (Django) as the basis for the new system. For all the Rubyists and Java heads out there who are surely crying bloody murder, I suggest that they just try it. Seriously. Django’s template system is freaking sweet, and Python (despite it’s lambda-related warts) is close enough to being executable pseudo-code as to still hold the second place in my toolbox of languages.

There’s a lot which I’m sure others will (and have) covered about how App Engine is going to change the game for startups and players like Amazon, but I think that if someone else had launched this system it would still survive on its merits alone. The only wrinkle in the whole thing will be seeing what’s done about pricing over the long haul. It really shouldn’t be hard for Google to beat S3 on price, and I’m sure there will still be a market for EC2 for non-traditional tasks, but fundamentally I think App Engine has all the makings of something really, truly better than the current (assumed) stack.

After more than a year of mourning the effective loss of Jot as a platform, writing apps on the server just got interesting for me again, and that may be the highest praise I can offer and framework or platform.