Which it is not.

Before I get to why, let me first say some nice, well-earned things about WebIDL: first, it has helped us out of the ad-hoc IDL sludge that used to be how APIs for JavaScript have been exposed in the past. It has shaved off many sharp edges and is giving spec authors a single dialect in which to write their API descriptions. From a browser perspective, this is a Very Good Thing (TM). Next, the draft in question contains some wonderful changes from the status quo, particularly the addition of a sane prototype to all WebIDL-specified objects.

That all sounds good, so what’s missing?

In a word, constructors.

Well, a lot more than that, but I’d settle for constructors. Functionally speaking, it boils down to the fact that WebIDL makes spec authors do extra work to make something like this sane:

new HTMLDivElement();

Why doesn’t this work today? Funny story…see, HTML defines HTMLDivElement as a regular WebIDL interface. WebIDL doesn’t really have the notion of concrete classes, just interfaces with and without constructors. Since the HTML spec is just doing what most specs will do — adding the smallest IDL you can get away with — the JS usability of this API is left in limbo; neither clearly HTML5′s responsibility nor WebIDL’s.

So what should a contentious version of HTML5 do? One answer is to specify a constructor, turning the IDL from this:

interface HTMLDivElement : HTMLElement {};

to this:

[Constructor]
interface HTMLDivElement : HTMLElement {};

Repeat ad-infinitum for each and every interface that should be constructable in every single spec that browser vendors ever implement. Don’t miss any! And please make sure that all your spec editors are on-board with good JS APIs as a goal! As it stands today, WebIDL doesn’t even force most spec authors to consider the question “do I need a constructor here?” — spoiler: yes — let alone the obvious follow-ups like “what arguments should one take?”.

The obvious better answer here is to flip the default on interfaces, causing them to generate constructors by default unless turned off with [NoConstructor] attributes or specified as partial interfaces (i.e., mixins or traits).

Cameron McCormack who is heading up the WebIDL effort tweeted in response to my exasperation that:

I think a “W3C Web API design guidelines” document would be a perfect place for such a recommendation.

For serious? Such a document might be useful (and I’m working on something that might pass as a first draft), but what’s the argument against flipping the default here? This isn’t a dissent on the facts of the situation: most WebIDL “interfaces” that are exposed to JS are things that could be easily new‘d up to useful ends. Most specs flub this in spectacular style. Most spec authors seem entirely ignorant of the problem and the design language of WebIDL continues to lead down a primrose copy-and-paste path that has little overlap with sanity. So why punt the decision? And why did it take and act of coordination with TC39 to get the prototype thing fixed?

And Why Are We Having This Discussion Anyway?

WebIDL, for all of its virtues, is deeply confused.

If you’re reading any of the stuff in the HTML5 spec that’s describing its API this way, it’s hard to see how it would have any sane relationship to JavaScript. Sure, you could argue that there might be other languages that matter, other languages for which you’d need to be able to generate some API, but none of them rise to anything like the importance of JavaScript. It is the programming language of the web, so if WebIDL has any animating force at all, it’s JS. Then there’s the “accident of history” aspect. Early DOM was specified as a form of IDL in part because there was some expectation that other languages would need to consume it and IDL was how C++ hackers (who still make up the entire set of people working on browser engines) are/were comfortable in describing their FFIs thanks to the legacy of COM/CORBA. Hilarious examples of multi-language-ism still persist in the WebIDL spec for no apparent reason whatsoever, warping the entire design around the altar of an ideal that is either quixotic or vestigial depending on which argument you give more weight.

Since the debate was re-kindled thanks to a debate at a TC39 meeting in July, I’ve been on the receiving end of more than one webdev’s rant about DOM’s JS incoherence, generally taking the form:

Why the *#!*?^@$ isn’t DOM just #@!*@ing specified in JavaScript?

To be honest, I have no answer aside from pointing to the IDL history, the fact that browser hackers don’t tend to write JS so don’t feel the pain, and noting that WebIDL is better in some important ways. Certainly these interfaces could be specified in a subset of JS with annotations for where native behavior is required. But their larger lament has merit too: seamlessness with JS is the bar WebIDL should be judged by. I.e. does it help spec authors do the right thing by JS devs? Or does it lead them down paths that make their generated APIs stick out like sore thumbs, full of anti-social/alien behavior such that you can’t think of them as “regular” JS?

Yes, constructors are only one minor step toward reaching this aspiration, but the fact that WebIDL has gotten to last-call without a reasonable solution to them speak volumes. If WebIDL isn’t animated by the need of JS developers, it would be good if that could be loudly called out somewhere so that the community can have the spirited debate that this point demands. If it is, can we please get on discussing how best to ensure that most “interfaces” generate constructors and stop punting?

Either way, WebIDL isn’t done yet.

Update: It occurred to me, as part of the discussion in the comments, that the provision against new with a class or type of any type is completely non-sensical in JS, as is the lack of call() and apply() methods on them. Idiomatic subclassing requires that the mixin-style be available, which uses ClassName.call(this). This is what you’d do with things that are “virtual” or “partial” interfaces if you’re describing them in actual JS. And there’s no real problem with folks new-ing them up. Doesn’t happen, doesn’t matter. Strong restrictions against it are, to quote Andrew DuPont, anti-social. Long story short: there is absolutely no reason whatsoever to disable construction on any WebIDL interface. It’s deeply non-sensical from the JavaScript perspective.

It’s good to see MSFT belatedly trying to put IE6 out to pasture, but what about IE 7? Or 8? Lets take stock of where we really are and where we’re likely to be in the next couple of years. First, remember that there’s no IE 9 for Windows XP — an OS that’s currently the most popular in the world — and no matter what happens with IE 6, IE 8 is the end of the upgrade road for XP. Unless you think half of the world’s computers will be replaced/upgraded in the next couple of years, it seems likely that IE 8 will be with us for the foreseeable future.

And what about the folks who do get IE 9? Well, so far, there’s nothing to make me believe that the uptake rate will be anything better than the IE 8 transition; a process which has taken 2 years to give ~30% of the market the latest version. If anything, we should expect that rate to be retarded somewhat by the XP hurdle.

MSFT’s browser replacement rates bear understanding because they’re the most popular and suffer from the longest half-lives. That is to say, the time it takes for an old version of IE to decay in the wild is much, much higher than for other browsers. Some part of this is surely due to sheer market share, but not all of it. The XP hurdle, for instance, is a form of structural drag on uptake rates — a flaw that browsers that aren’t tightly tied to OSes don’t suffer from. For web developers, I dare say that half-life of popular browsers matters much, much more than the current or trending market share since it’s predictive of our potential for browser improvement in the near future. It’s one thing to get the new shiny, but how long will it take you to install it? If the shiny is old and dingy by the time it’s in place, what good is that? It’s this lens that makes browser market share stats interesting to me; i.e., what percentage of the web’s users will get the new features soonest? ‘Cause those are the folks we can start building super compelling content for.

The average half-life of the majority of browsers in the wild also gates the rate of progress in standards. When the process is working well, bugs in browsers or pre-standards implementations of features aren’t a permanent features of the landscape. Instead, they’re the understandable and inevitable result of a process that prioritizes implementation experience and iteration over raw compliance with an academic spec that may or may not actually get it right on the first go ’round. But that iterative, feedback-rich process only works when browsers iterate quickly and web developers can target the future without thinking so hard about the past, else progress simply turns into something to resent and distrust. That’s good for no one, and a shorter half-life is the key to making progress more than just a spec-tease.

I’m personally hopeful that when IE 9 is finally RTM’d, that it includes some provisions for shortening its life expectancy in the ways that Chrome and Firefox have through aggressive auto-updating. Getting IE 9 out to the world will be a good thing, but only if it happens quickly and if IE 10 can follow it even faster.

There’s obviously a lot more to talk about at the browser panel — Chrome 10 just launched with Crankshaft, for instance — but the fact that nearly every Chrome user will have those improvements this week and that if you’re building a Chrome Web Store app, you’ll get to target those improvements nearly instantly seems like the biggest, most interesting change from where we were just a couple of years ago.

How bad is it? The next time you read some tech journalist write about how some new browser version is just around the corner and how it’ll make everything better, remember that:

• 50+% of Windows users are on XP a year after Windows 7 shipped and 3.5 years since Vista shipped
• Windows XP will be supported until 2014, giving organizations on XP extra breathing room to limp along on IE 6-8
• There will be no IE 9 for Windows XP
• After something like 4 years of MSFT urging customers in the strongest possible language to get off of IE 6, it still has 16% of the market, and it’s not falling nearly fast enough. At the current 0.75%/month dropoff, we’re looking at 20+ more months of IE 6. At this rate, kids born today will be walking and maybe even talking by the time we can write IE 6 into the history books.

The goal here isn’t to drive you to drink, but to call out the dichotomy between when users get features and when developers can address features. For users, things tend to get better as soon as you pick up a new browser. Developers have to wait until every user makes that sort of choice. Said differently, users can benefit from the bleeding edge but developers are beholden to the late adopters.

We should cut tech journalists and users a little slack, though; in a world without adoption friction, the launch of a new feature would translate directly into the sort of “now you can build sites and apps with awesome thing X”. The instinct to believe that’s how it should be is spot on. But that’s observably not how the world works. Don’t believe the hype. New browsers alone haven’t fixed the problem so far, so what makes us think they will in future? No vendor wants to talk about their last version, but for web developers stuck in the trenches, that all there is to talk about. That’s where the pain is, after all, and counting on browser upgrades to fix the problem quickly isn’t working well enough. Chrome’s aggressive auto-update feature is changing the way things will work in future, but for now we’re still stuck in the slow-upgrade dynamic.

We need a Plan B.

As far as I know, it was a total surprise to the Dojo community (myself included). I can’t wait to get started writing apps for this thing and see what device APIs Palm has surfaced.

In the past couple of years the shift from desktop-centric to a more web-centric OSCON has continued to make the conference useful and engaging, and great work on topics like JavaScript/Ajax performance, Dojo, Comet, and many of the emerging back-end bits of infrastructure that make it all go have made my yearly trip to Portland worthwhile.

Great talks have gotten lost from the JavaScript/web track in years past because they’ve missed the submission deadline, so if you’re hacking on something fascinating, now’s the time to get that proposal in. Make sure you flag it with the right track when you submit (javascript, ajax, web, or whatever they’re calling it this year), and don’t hesitate to ping me if you’re unsure about whether or not your talk got slotted correctly for the review process.

Maybe.

These things matter only in regards to how well they represent end-user workloads and how trustworthy their findings are. The first is much harder than the second, and end-to-end benchmarking is pretty much the only way to get there. As a result, sites like Tom’s Hardware focus on application-level benchmarks while still publishing “low level” numbers. Venerable test suites like SPECint have even moved toward running “full stack” style benchmarks which may emphasize a particular workload but are broad enough to capture the wider system effects which matter in the real world.

Marketing departments also like small, easily digestible, whole numbers. Saying something like “200% Faster!” sure sounds a lot better than “on a particular test which is part of a larger suite of tests, our system ran in X time vs. Y time for the competitor”. Both may be true, but the second statement gives you some context. Preferably even that statement would occur above an actual table of numbers or graphs. Numbers without context are lies waiting to be repeated.

With all of this said, James Ward’s Census benchmark makes a valiant stab at a full-stack test of data loading and rendering performance for RIA technologies. Last month Jared dug further into the numbers and found the methodology wanting, but given some IP issues couldn’t patch the sources himself. Since I wasn’t encumbered in the same way I thought I might as well try my hand at it, but after hours of attempting to get the sources to build, I finally gave up and decided to re-write the tests. The result is Census 2.

There are several goals of this re-write:

• Fairness. Tests need to be run multiple times for them to be representative in any way. Likewise, systems not being directly tested need to be factored out as much as possible. C2 does this by reducing the number of dependencies and running tests (at least) 5 times and discarding outliers before reporting an average. I’ve also worked to make sure that the tests put the best foot forward for all of the tested technologies.
• Hackability. Benchmarks like Census serve first as a way for decision makers to understand options but second as a way for developers to know how they’re doing. Making it trivial to add tests helps both audiences.
• Portability. The test suite should run nearly everywhere with a minimum of setup and fuss. This ensures that the largest numbers of people can benefit from the fairness and hackability of the tests.

The results so far have been instructive. On smaller data sets HTML wins hands-down for time-to-render, even despite its disadvantage in over-the-wire size. For massive data sets, pagination saves even the most feature-packed of RIA Grids, allowing the Dojo Grid to best even XSLT and a more compact JSON syntax. Of similar interest is the delta between page cycle times on modern browsers vs their predecessors. Flex can have a relatively even performance curve over host browsers, but the difference between browsers today is simply stunning.

Given the lack of an out-of-the-box paginating data store for Flex, RIAs built on that stack seem beholden to either Adobe’s LCDS licensing or are left to build ad-hoc pagination into apps by hand to get reasonable performance for data-rich business applications. James Ward has already exchanged some mail with me on this topic and it’s my hope that we can show how to do pagination in Flex without needing LCDS in the near future.

The tests aren’t complete. There’s still work to do to get some of the SOAP and AMF tests working again. If you have ideas about how to get this done w/o introducing a gigantic harball of a Java toolchain, I’m all ears. Also on the TODO list is an AppEngine app for recording and analyzing test runs so that we can say something interesting about performance on various browsers.

Census 2 is very much an open source project and so if you’d like to get your library or technology tested, please don’t hesitate to send me mail or, better yet, attach patches to the Bug Tracker.

Update: I failed to mention earlier that one of the largest changes in C2 vs. Census is that we report full page cycle times. Instead of reporting just the “internal” timings of an RIA which has been fully boostrapped, the full page times report the full time from page loading to when the output is responsive to user action. This keeps JavaScript frameworks (or even Flex) from omitting from the reports the price that users pay to download their (often sizable) infrastructure. There’s more work to do in reporting overall sizes and times (“bandwidth” numbers don’t report gzipped sizes, e.g.), but if you want the skinny on real performance, scroll down to the red bars. That’s where the action is.

The truly dispiriting thing, though, is how badly cmd.exe still sucks. I fully admit that my personal programming proclivities are not normal, but to be reasonably productive I need a Unix-like shell, a terminal that works (can be resized, has reasonable VT100 emulation, etc.), and the ability to fix the “Caps Lock” key to do the right thing – namely, have it fire the “Ctrl” key instead. This is all relatively straightforward to do on Linux and OS X. Here’s how I got it done with Windows:

Do the MSFT Patch Dance

Make coffee?

Install Cygwin

We’ve all done it a thousand times. This’ll make 1001. It’s kind of comforting that the Cygwin home page hasn’t changed perceptibly in nearly a decade.

Get SharpKeys

Instead of fugly registry hacks, SharpKeys allows you to map the dreaded and useless “Caps Lock” key to something actually useful. If your key-mapping preferences swing some other way, SharpKeys can likely handle that too. Not sure why it’s not built into Windows, frankly.

Set Up Puttycyg

Having cygwin is nice, but having a terrible shell with Cygwin? Not so nice. Enter Puttycyg, a small hack on the venerable Putty SSH client for windows that provides an option to launch a local Cygwin session in lieu of connecting to another system.

Once I extracted it and ensured the Puttycyg directory was in my windows PATH, I created a desktop shortcut to the putty.exe included in the distribution and configured the shortcut (right-click) to read:

"C:\Documents and Settings\slightlyoff\Desktop\puttycyg\putty.exe" -cygterm -

And then set the “Shortcut key:” to be:

Ctrl + Alt + T

Now, whenever I want a fully functional shell from my desktop, I just hit that key combination and it All Works (TM).

Today, Dojo’s delegate() function is a straightforward implementation of the Boodman/Crockford delegation pattern which Doug calls “beget” and which ES 3.1 will refer to as Object.create:

1
2
3
4
5
6
7
8
9
10
11
12

dojo.delegate = (function(){
    // boodman/crockford delegation w/ cornford optimization
    function TMP(){};
    return function(obj, props){
        TMP.prototype = obj;
        var tmp = new TMP();
        if(props){
            dojo._mixin(tmp, props);
        }
        return tmp; // Object
    }
})();

This function returns a new object which looks to the old object for things it does not itself have. Imagine an object foo which contains pithy truisms:

var foo = {
  science: "rocks!",
  learning: "is how you know you're alive"
};

We now want to promigulate our opinions, so we can delegate the responsibility of forming them:

var bar = dojo.delegate(foo, {
  testify: function(){
    console.debug("science ", this.science, "and learning", this.learning);
  }
);

Now, our bar object can change its mind independently of foo, but until it does, it’ll behave as though foo‘s views are its own:

bar.testify(); // outputs: "science rocks and learning is how you know you're alive"
 
// bar refines its opinion
bar.science = "is a process"; 
bar.learning = "requires humility";
foo.science == "rocks!"; // still true
 
bar.testify(); // outputs: "science is a process and learning requires humility"

But what about when the chain gets deeper? The fact that bar can’t “see” foo‘s values via this isn’t much of a problem when the hierarchy isn’t very long, but if you’re specializing a behavior or complex interaction, making it possible to get at the parent’s values for properties and methods becomes more pressing.

Neil has previously written about lightweight subclassing, but for as good as it its, it doesn’t get us all the way there either. In regular OO-style languages, the inheritance system gives you an out via a “super” keyword or convention. This type of property shadowing-with-exceptions is a huge boon to composition in class-based languages, but it’s not the whole story. Indeed, the Traits paper was all about the shortcomings of this special-purpose mechanism. What we want for both long delegation chains and long inheritance hierarchies is a more general system; in essence a way to say “I want to control how things are shadowed and which ones an item points at in each level of the hierarchy”.

What if we could make delegate() savvy of this type of indirection? Here’s my quick prototype:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

delegate = (function(){
    var tobj = {};
    var TMP = function(){};
    return function(obj, props){
        TMP.prototype = obj;
        var tmp = new TMP();
        if(props){
            var remaps = props["->"];
            if(remaps){
                delete props["->"];
                for(var x in remaps){
                    if(tobj[x] === undefined || tobj[x] != remaps[x]){
                        if(remaps[x] == null){
                            // support hiding via null assignment
                            tmp[x] = null;
                        }else{
                            // alias the local version away
                            tmp[remaps[x]] = obj[x];
                        }
                    }
                }
            }
            dojo.mixin(tmp, props);
        }
        return tmp; // Object
    }
})();

This new version of delegate() accepts a specially named “->” property in the list of items to add to the destination object. Items in this list can either “shadow null” (hide entirely) the parent’s property or can provide a new name for it, assuming of course that the new object will also have a property of that name. Here’s a quick example of “->” at work with our previous example. This time, foo also has a “testify” method that we’d like bar to be able to control without having to copy the implementation:

var foo = {
    science: "rocks!",
    learning: "is how you know you're alive",
    testify: function(){
        console.debug("science ", this.science, "and learning", this.learning);
    }
};
 
var bar = delegate(foo, {
    "->": {
        "testify": "grampsSays" // maps foo's "testify" to bar's "grampsSays"
    },
    testify: function(){
        if(this.science && this.learning){
            this.grampsSays(); // call the re-named "testify"
        }else{
            console.debug("this object is strikingly ignorant");
        }
    },
});
 
bar.testify(); // outputs: "science rocks and learning is how you know you're alive"
bar.science = false;
bar.testify(); // outputs: "this object is strikingly ignorant"

That New Object Smell

The last missing piece of the hierarchy pie here is that there’s no initializer for the objects which come from a delegation. A simple addition of some property detection code to look for an initializer can easily handle that:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

delegate = (function(){
    var tobj = {};
    var TMP = function(){};
    return function(obj, props){
        // boodman/crockford delegation w/ cornford optimization. 
 
        TMP.prototype = obj;
        var tmp = new TMP();
        if(props){
            var remaps = props["->"];
            if(remaps){
                delete props["->"];
                // like dojo.mixin(), except w/o key/key mapping
                for(var x in remaps){
                    // "safe" copy properties
                    if(tobj[x] === undefined || tobj[x] != remaps[x]){
                        if(remaps[x] == null){
                            // support hiding via null assignment
                            tmp[x] = null;
                        }else{
                            // alias the local version away
                            tmp[remaps[x]] = obj[x];
                        }
                    }
                }
            }
            dojo.mixin(tmp, props);
        }
 
        // support for "constructor" functions. The name "init" is arbitrary.
        if(typeof tmp["init"] == "function"){
            tmp.init.call(tmp);
        }
 
        return tmp; // Object
    }
})();

And there we have it. A style of delegation that easily supports both Trait-like name aliasing (and null shadowing) as well as internal initializers. Since our upgraded delegate can handle nulling out a parent’s value for a property, we also have a straightforward way to prevent parent initializers from being called (or being called/chained – at our discretion – by a new name):

var foo = {
    science: "rocks!",
    learning: "is how you know you're alive",
    testify: function(){
        console.debug("science ", this.science, "and learning", this.learning);
    }
};
 
var bar = delegate(foo, {
    init: function(){ this.testify(); }
});
// outputs: "science rocks and learning is how you know you're alive"
 
var baz = delegate(bar, {
    // map away the parent's constructor
    "->": {
        "init": "superInit"
    },
    // provide our own constructor
    init: function(){
        console.debug("howdy!");
        this.superInit(); // call the super-object ctor
    }
});
// outputs: "howdy", "science rocks and learning is how you know you're alive"
 
var thud = delegate(baz, {
    "->": { "init": null } // hide the parent ctor
});
// outputs: nothing

This form of delegate is likely to appear in Dojo 1.3 along with similar improvements to dojo.declare() to help alleviate the composition problems associated with using complex sets of mixins.

Update: corrected the null-out branch and updated the text with Doug’s note that beget/delegate will be called Object.create() in 3.1.

While in Boston, Gavin and Jill joined a gaggle of Dojo hackers at a dinner ostensibly to mourn my birthday (thanks to Dylan and Pete for organizing!) and in the course of conversation Jill asked something along the lines of:

So why do people get so excited about closures?

Which prompted a several of us to flail and flop gasping the salt flats of analogy like fish out of polite-conversation water. After about 10 minutes of this, Jill succinctly summed it all up in the form of a question:

Oh, so it’s like “action-oriented programming”?

This is perhaps the most insightful and succinct description I have ever heard of what JavaScript is all about.

Update: Jennifer just played this for me and it gets right to the heart of this post: the important part of doing what we do with computers, and more importantly, with the web, is to give the power of Computer Science to real people…and it starts with insights like Jill’s that build a shared way of thinking and talking about the world. It makes me sad that many programmers miss that, but when non-programmers can share in the beauty and power of code, it does a lot to make it all seem worthwhile.

At the risk of talking too much out of school, I want to set the record straight in some ways. First, let me set some facts out:

• ES4, as outlined last fall, was not ActionScript 3. Major changes to the semantics of Adobe’s initial contribution ensured that any truly compliant ES4 implementation from Adobe would have required many concessions, including the inclusion of a client-side compiler. ES4 without eval() (ala AS3) would never have cut it as a “real” ES4.
• Adobe had not donated “ActionScript 3″ to Mozilla nor Open Sourced ActionScript 3. Adobe had donated a high-performance byte-code VM which a separate front-end compiler targeted. It is absolutely possible that the Tamarin VM can and will run whatever syntax for the next version of JavaScript is ratified. The design of the VM is, of course, predicated upon the language semantics but lets not confuse the front-end compiler and the language that it consumes with the VM and byte-code format which it executes. The front-end compiler was not donated to Mozilla (although something similar exists in the OSS’d Flex SDK) nor does it ship inside of the Flash player today. Adobe can continue to evolve their developer languages and byte-code VMs independently so long as they never ship an eval() feature. Even if they do, there are other (harder to swallow) options available, but no road is cut off to Adobe which was not already long-since abandoned in other ways by the ES4 process.
• Like Adobe, Microsoft has jumped ahead in the evolution of JavaScript via the seemingly forgotten JScript.NET. Like Adobe, Microsoft has had to come back from that position to meet the web where it really is. Microsoft now ships 3 – yes, that’s right, three – JavaScript-ish languages which are capable of running in a browser:
  • JScript (via Windows Scripting Host)
  • JScript.NET
  • JScript for the DLR (via Silverlight)

  Make no mistake about it, these are all separate implementations which likely share little if any code. The DLR variant is even new in the last several years. Creating a new JavaScript compiler and runtime is not an overly onerous task for MSFT and clearly not one that will take a long time should the occasion arise. What’s confusing and can likely be tied to strategic wrangling is the puzzling lack of progress in the WSH version of JScript (the one everyone uses day-to-day). Any strategic discussion of JScript as a platform needs to start from this perspective.

• Much in ES4 was new. The gradual typing system (which I’m a big supporter of) was something of a large-scale experiment coupled with as much rigor as I’ve ever seen in a standards body in proving things out in an RI. The Class system (which I wasn’t a big supporter of) needed to marry class-based inheritance to prototypal inheritance in ways which were new. ES4 contained many good features and syntactic forms which were borrowed from other places, but as Brendan Eich has said, it was a process of synthesis plus invention. In my opinion, much of the failure of ES4 as an initiative can be traced to significant failings of ActionScript 3 as a language. AS3 is a Java-like language with a Java-like execution model. JavaScript is a dynamic language with a dynamic execution model. The gulf in expectations between those camps turns up in every corner of the language design. For instance: what does it mean to load new code? Are classes forever “closed” or can they be re-opened? Do you really need the static and final keywords? Solving these issues in the context of the old is easy. In the context of the new, it’s hard. ES4 suffered because it had to do it in a new world which no one was yet writing code for.

So, lets pop up and talk about strategy for a minute. Fundamentally, very little has changed in terms of available strategic options for any of the players:

• MSFT can still hold the web hostage to their ailing WSH VM by continuing to ignore its performance, regardless of bug fixes and syntactic updates. Doesn’t matter if it’s amputation or debilitating arthritis, crippled is crippled. For what it’s worth, my interactions with the MSFT reps on TC39 give me no reason to believe that they won’t be improving their VM.
• Adobe can still chose to implement a language which implements an ECMA spec. They can do this any time they damn-well please. It may not align so cleanly with their current technology roadmap, but it’s absolutely feasible and when Mozilla ships a “regular JavaScript” front-end to the Tamarin back-end, it’ll be even easier. Note that this plan can be done in parallel with AS3 evolution since Tamarin (as a VM) need not be aware of any language semantics on its own.
• Browser vendors can still wring large speedups out of ES3 and 3.1

What died here wasn’t Adobe’s attempts to “own” a spec. If there were such hopes in play, they had been quietly put down one rational, backwards compatible decision after another in the year preceding the Oslo meeting. What died was an assumption that the web can evolve without implementations being out in front of the spec. AS3 was one implementation of a JavaScript-like language that might have been a contender for crown of “the next one”, but so was JScript.NET. That neither of them “won” simply because they had been built in good faith is as true a sign as I can find that the process is working. Adobe gets it. Lets end the silly meme that “Adobe lost” or that “Microsoft won”. The game has hardly begun and it won’t be settled in a standards body anyway. What matters – and what we all need to keep our eyes keenly trained on – is what the big implementations do in the way of compatibility, performance, and feature set once ES3.1 arrives.

I’m incredibly excited about the SitePen Dojo Toolbox AIR app that just launched. I’ve been using early versions for a couple of weeks now, and in that time it has earned a privileged place on my desktop. Being able to make local builds without delving to the command line is something that I’ve wanted to be able to show new Dojo users for years and this tool finally makes it easy. There’s more work to do around configuring the profiles themselves, but the new tool demystifies many of the build configuration options significantly. Having a searchable API viewer available is also a godsend. Huge props to the team that put it together!

Edit: I forgot to note one of my favorite parts of the project, namely that because the Dojo build system is written in JavaScript (thanks to James Burke‘s awesome work), the build system in the Toolbox doesn’t require Java. Instead, SitePen was able to port the few Rhino dependencies in the build tool to work with the native JavaScript engine in AIR. It’s a minor thing, but it speaks to the excellent engineering behind Dojo’s tooling and the power of having actual web-native technologies inside a desktop app.

Fundamentally the Dojo and Zend teams really “get” each other. Both are deep packages that give you an opinion about how best to do something but also all of the tools you’ll need to make it work at scale. The “use at will” term that the Zend folks use made immediate sense to us. Like Dojo, Zend doesn’t saddle you with more than you’re really going to need up-front, but at the same time, when you need something awesome which is well tested and integrated, it’s right there. No digging around on google to find a “plug in” that will get you where you want to go…both Zend and Dojo give you a full stack of great components to work with out of the box.

There have been some questions on IRC today about why Dojo and not something else, and we know that the Zend folks are committed to continuing to allow Zend to work with other frameworks as well and we fully support them in that. There seem to be lots of choices of Ajax frameworks which Zend could have integrated, but when you look at the requirements of serious products which need to ship tested solutions to really hard problems, the field whittles down very fast. In response to the needs of our users, both Zend and Dojo take seriously our responsibilities to provide integrated, high-quality, unambiguously licensed products that will let user scale both up and down. Key to this is understanding the full spectrum of use-cases, informed by past experience, and striking a balance between enterprise-ready features and close-to-the-metal primitives. The Zend Framework has a larger view of the server-side than Dojo can, and as a result there are new opportunities to optimize and improve the user experience for all classes of users through this integration. This isn’t just about including some scripts on a page, this integration is about improving user and developer experiences, and both Dojo and Zend bring a lot to the table which can compliment the other. Dojo’s strengths in progressive enhancement, packaging, localization, and accessibility all have obvious allegories in the ZF world where a complete integration can more value based on what developers are already doing. All of those features of Dojo will work better when the server-side knows how to “hint” things for the client and work with, not against, the client to deliver better experiences.

I’m perhaps most excited about the data-driven opportunities in the Zend/Dojo integration. Dojo’s data infrastructure is second-to-none, and the design of the dojo.data and dojo.rpc layers provide Zend Framework integration the ability to take advantage of systems like the incredible Dojo Grid and the client-side charting package. There’s more to look forward to, and figuring it out together with the folks at Zend is a great opportunity for the Dojo community.

Pete Higgins and I will be participating in next Tuesday’s discussion with a broader chunk of the Zend Framework world, but until then (and long after) we’ll be hanging out in the #dojo and #zftalk channels on irc.freenode.net. We’re looking forward to working more with the ZF community to build great experiences, and are hugely excited about the direction things are already taking!

As they started talking about the platform and what’s part of it (and what’s not), I couldn’t escape escape the feeling that they’d gotten it right. It is absolutely the case that for most apps, scaling requires some amount of re-architecting. Systems like Rails are built with such a dependence (intentional or not) on the features of relational data stores that they quickly hit bottlenecks because frameworks aren’t keeping developers out of the gutter. This is nearly the same lesson that the security community collectively came to when it started to beat the average developer about the head regarding the awesome power of defaults. What systems do and don’t do for you “cheaply” defines their character, and in many systems those choices aren’t made consciously, or if they are, they don’t have the benefit of a different perspective which might de-emphasize certain traits. Call it libertarian paternalism, choice architecture, or pure condescension, but whatever it is, systems and platforms today are in the explicit business of making some things easier than others.

As the presenters showed how to make an app quickly, I knew I was looking at something I hadn’t seen the likes of since Jot. We all know that Big Table is a column-oriented data-store, but since most people are still stuck on the likes of MySQL, it’s hard to appreciate how liberating it is not having to think about how adding properties to models will affect a schema. The way App Engine is constructed lets the data store layer provide something called expando models. These models give us the kind of flexibility that I’ve only ever enjoyed before inside of Jot. Want a property? Just add it. No migration, no schema version…just data finding a happy home, and as your app’s skeleton “solidifies” and you figure out which properties really do need to be faster, you can migrate that data into fixed properties with indexes and types and all that jazz. It’s like a gradual type system, only for data stores. It’s agility nirvana for application development.

Speaking of application development nirvana, they also had the good sense to start with a great language (Python) and a great webapp framework (Django) as the basis for the new system. For all the Rubyists and Java heads out there who are surely crying bloody murder, I suggest that they just try it. Seriously. Django’s template system is freaking sweet, and Python (despite it’s lambda-related warts) is close enough to being executable pseudo-code as to still hold the second place in my toolbox of languages.

There’s a lot which I’m sure others will (and have) covered about how App Engine is going to change the game for startups and players like Amazon, but I think that if someone else had launched this system it would still survive on its merits alone. The only wrinkle in the whole thing will be seeing what’s done about pricing over the long haul. It really shouldn’t be hard for Google to beat S3 on price, and I’m sure there will still be a market for EC2 for non-traditional tasks, but fundamentally I think App Engine has all the makings of something really, truly better than the current (assumed) stack.

After more than a year of mourning the effective loss of Jot as a platform, writing apps on the server just got interesting for me again, and that may be the highest praise I can offer and framework or platform.

1. Array’s Can’t Be Usefully Subclassed (test case)

   At first blush, this seems wrong. You can use the Array base-class as the prototypal delegate for any user-defined class you wish. Methods are correctly delegated to and hash-style indexes work fine. Almost everything works right…except when you try to use the built-in array manipulation methods like push, pop, and shift. They dutifully change the internal contents of the subclass instance’s indexed attributes, but they don’t manipulate the length property. This means that while you can use for(var x in list){ ... style iteration, you can’t do anything *aside from key iteration* to know how many items are in the array. Obviously, one could try to wrap the intrinsic functions and detect how they manipulate the length property, but then you’ve ruined their [DontEnum] status and now they end up in the iterable surface area of instances. Ugg.

   Arrays without a working length property are nearly useless, and JScript mangles the design of toolkits as a result.

   So how do we get dojo.NodeList to be a “real” array with extra methods then?

   As you might expect, it’s a giant hack. When you use the “new” keyword with the dojo.NodeList function, you expect that the system will create a new instance and do it’s normal “stamp with a constructor” business. Instead, we resort to creating (and populating) a regular Array instance and “NodeList-ifying” it by copying named attributes from the class prototype into the instance as member properties. The “constructor” function then explicitly return a new object, bypassing the “new” keyword’s create/stamp machinery, at which point the return of the new operator becomes our explicit return and not the object which it would have otherwise implicitly returned.

   In Dojo 0.9 we had used an even more aggressively hackish workaround for IE which involved creating a sub-document and mapping its interpreter’s intrinsic Array class into the parent document at a different name. Both are slow for different reasons but we eventually switched to the create-and-mix-in style because some popup blockers were interfering with the old method.

   Lest you think that Dojo is a dirty, dirty toolkit for doing this kind of thing, consider the janktastic “it’s not really an array” thing that JQuery resorts to instead. By giving up all “[]” index operations, JQuery manually maintains it’s internal length property by re-implementing all of push, pop, etc. functions. This has the benefit of allowing prototypal delegation to work of pre-existing instances when new features are added to the base prototype, but at the expense of no longer being able to think about a dense list of things as an array. Dojo’s approach is painful, but so are all the alternatives today.

   I think it’s safe to say that both toolkits would subclass Array directly to save code, were it a reasonable thing to do.

2. Where Art Thou Getters/Setters?

   As JavaScript toolkits get pushed out of their current workhorse tasks of plastering over JavaScript and DOM implementation gaffes by positive browser progress and Moore’s Law, they increasingly take on application construction tasks (e.g., dojo.data). As the toolkits have approached these tasks, we’ve collectively started to hit some very serious usability limitations due, in large part, to JScript’s lack of progress.

   Toolkits like Dojo have widget systems because HTML just hasn’t kept up. That means that these toolkits have a responsibility to keep as many of the positive aspects of the “native” web as they can. From CSS customization to accessibility all the way through implementing declarative creation and DOM-ish JavaScript APIs, the better a job a toolkit can do in making the abstraction feel more solid, the better the toolkit is. Widgets are essentially a way to “subclass HTML elements”.

   In many places, DOM allows you to affect the behavior of the visible document using “setter”-style syntax. For example:

   
   document.getElementById("foo").style.border = "5px dotted black";
   

   Custom widget classes can have the same behavior on every browser except IE.

   This means, of course, that JavaScript toolkits can’t really implement the behavior, backing JavaScript programmers up against a wall when they design their tools. Instead of providing the natural property-oriented behavior, it forces class authors to write getSomeProperty/setSomeProperty method pairs on their classes should they want to do anything when values are gotten or set. The resulting code feels a lot more like Java than JavaScript, which is usually a sign that something is horribly wrong in a browser.

   As bad as this problem is for visual widgets, it’s worse for data binding systems. API’s like dojo.data would be designed in fundamentally different ways if getters and setters were available everywhere. Instead of the rigamarole of making users fetch an item reference and then fetch attribute values using the opaque item handle and the property name, we’d just set up getters and setters on the properties themselves and defer the implementation of fetching those values down to the data store. Further, assigning a linkage between a dojo.data query or store and a widget which represents it could be as simple as assigning a property to the widget object.

   So are workarounds to this possible? I think they are, and I’m testing some of them out for use in Dojo 1.1 right now. I’ll post more about them should they pan out. Every avenue which looks potentially workable right now involves gigantic hacks which also deeply constrain API designs. Fundamentally, this problem can’t be solved without good language-level support.

   It’s perhaps folly to assume that this will be addressed in IE 8, but given the enormous back-pressure of nearly every JavaScript toolkit author demanding this feature and the embarrassment of every other browser beating them to the punch, I have some hope that we could see getters and setters for JScript in the near future. It won’t matter much, though, unless the JScript team ships their new engine to all IE versions when they release IE 8. Not bloody likely.

3. Performance

   Kudos are in order to the JScript team for fixing their long-b0rken GC heuristic and pushing it out to everyone…but it’s the tip of the iceberg.

   Performance is one of those areas where differences in implementations can tightly circumscribe what’s possible despite exacting spec conformance. On this front, JScript’s raw VM-level execution time leaves a lot to be desired, but the true travesties really show up when you hit the DOM for computed style information or try to do anything reasonably complicated that involves string operations.

   Most non-trivial blocks of JS code today rely on innerHTML to bootstrap some new chunk of DOM in response to user action due in large part to the cross-browser speed and size advantages of innerHTML vs. raw DOM methods for equivalent DOM structures. This reality pushes IE’s string performance woes to the fore as more and more client-side systems push far enough to hit the new “wall”.

   Similarly, getting computed box model calculations out of IE is not for the faint of optimization foo. When we profiled Dojo’s widgets to plan our attack for 0.9 and 1.0, we noted very quickly that getting box-model data out of the browser for any element is hugely costly on every browser, but on IE the cost was not just big… it was enormous. Our best guess right now is that the properties on the currentStyle property are re-calculated when they’re requested from script and not cached in the bound object when layout happens. The resulting performance penalty requires that developers nearly never manage layout in code, severely constraining the layouts which are attempted by toolkits like Dojo.

   Across the board, from DOM performance to raw JScript execution speed, IE is a dog, and the odds are good that whatever toolkit you’re using spends a lot of time working around that reality.

4. Doctype Switching

   Doctype switching to toggle box-model behavior is perhaps the single most limiting implementation error in IE. Saner browsers allow you to use a CSS property to affect the layout model in use in a particular section of a document. This makes tons of sense in a templated world where most of the markup your system generates starts and ends in the “news hole”. Today, that covers nearly everyone. A quick line count in any HTML document shows that doctypes are a scarce resource whose scarcity is made problematic when it’s semantics are overloading. I’ve been on product teams where the idea of changing the doctype would require months to recover from. That kind of cost related to what should be simply a markup dialect change (not a formatting policy change) implies strongly that the doctype is a terribly brittle way to control several independent concerns.

   Instead of giving devs fine-grained layout system control, IE makes it all-or-nothing. The global flag approach backs toolkit developers into doing script-based layout calculations or “just throw it in another div” solutions where we’d really rather not. Both are slow and both may be required since it’s completely impractical to dictate to users which doctype they’ll be using. While any app may be able to be disciplined enough to not care, toolkit developers must work everywhere. Hilarity ensues.

   I fear this is going to get even worse with IE8 as the IE team looks to implement some of HTML 5 and hopefully many of CSS 2.1′s clarifications. The sooner they abandon the global switch, the better…but I’ll wager it’s pain they just don’t feel. Building a browser is a very different pursuit from building portable apps to run inside it.

5. HTC’s Can’t Be Inlined (Even With Hacks)

   Modern browsers have built-in widget systems. On IE, it’s HTCs + Viewlink and on Firefox it’s XBL. Even a cursory reading through the docks for both is enough to illuminate the gigantic overlap. Alas, no one is yelling at them to standardize and the result is a terrible mess in which both sub-optimal formats limp along with nearly zero Open Web usage.

   So why do I single out IE for whipping here when XBL is just as lame and similarly b0rken with regards to single-file embedding? Well, on Mozilla, you have a lot more “outs”. I strongly suspect that you can use “data:” urls to generate and evaluate component definitions for FF, which would enable compiling down from a single (more sane) format in the running page environment. IE prevents any such useful code-loading approaches, meaning you have to generate files on disk in their b0rken-ass format in order to be able to use them. Given how far we’ve gotten with non-builtin widget systems, it’s pretty clear that toolkit authors aren’t going to contort themselves into requiring a build step that splats files all over disk just so that we can give IE and Mozilla different views of the same component description. Instead, we all limp along on our own class hierarchies without any of the benefits of element subclassing, getter/setter handling, and inline (scoped) method description that these browser-provided systems would allow. It’s kind of pitiful, really.

   IE8 may include strong “data:” URL support given that it’s needed to pass Acid2, but I’m not holding my breath. I strongly suspect that HTC’s are a dark, unloved corner of the Trident codebase that none of the current engineering team are really fired up about fixing (which they could have done by just allowing Data Islands to contain HTC definitions…but I digress). The takeaway here is that we probably shouldn’t even need JS toolkits to build widget systems, but it’s too late now. We’re an abstraction short and a decade late and now the Flex frankenstein is beating HTML at it’s own game.

In a vacuum of feature data or builds to work with, it’s prudent to assume that IE 8′s DOM and JavaScript implementations will continue to warp attempts at building useful, idiomatic JavaScript libraries to ease the problems that HTML + CSS aren’t effectively solving. So next time you wonder why your toolkit of choice is built the way it is or why it’s even necessary, just remember that in many cases they are protecting you from a decade or more of bad decision making.

From that perspective, they’re worth every penny.

To be honest, SitePen is usually hiring, so why the blog post? Because I’m hiring for a SitePen R&D Associate. This isn’t your average programming job. Not only will all the work from this new position be released as Open Source software,it’s a self-directed research job. Combined with SitePen’s completely-virtual structure, you can work on what you feel is important and live wherever you want.

Read on for the full description:

Position

R&D Associate (title negotiable)

Description

SitePen is a fast-growing services company with deep roots in Open Source web software and an ongoing commitment to give back to the Open Web. The problems we solve for clients sit at the intersection of Computer Science and Interaction Design while cutting across a wide variety of problem domains. As an R&D Associate at SitePen you’ll be responsible for investigating and developing solutions to the thorniest problems in real-world web application development and since everything you do will be Open Source, your impact will be both meaningful and lasting.

Some topics currently of interest to SitePen include:

• web framework scalability
• parametric CSS
• ES3/ES4 implementations across VMs
• publish/subscribe messaging and scalability
• web-based interaction design/analysis tools
• Open Data portability and metadata normalization
• browser-based data visualization

But those are just the problems that keep us awake at night. What we’re really interested in is hearing about how you’d like to make a difference in the evolution of the web. At SitePen, you’ll have the freedom to pursue your interests and a mandate to work with broader communities to make your ideas reality.

Required Qualifications

• contributor to at least one Open Source project (active committers preferred)
• fluent in one functional or scripting language (C experience a plus)
• must be able to author technical papers
• must be comfortable presenting and defending work to groups of various sizes
• permanant legal right to work in the United States

How To Apply

Send a plain-text email to “alex@sitepen.com” with the subject line “R&D Associate Application”. In the body of the email, please explain why you think you’d be good for the job, what research interests you would like to pursue, and if you have one, a link to your website/blog. Please include links to your Open Source involvement and note major contributions (planning, design, features implemented, research contributed, UI/UX/IxD, etc.). Also, either include a link to an online resume or attach one in plain-text or PDF format.

There is no deadline for application, but the earlier you apply the better your odds.

Benefits

• Senior Engineer grade pay
• Comprehensive health insurance
• All work product Open Source

Additional Information

• Location: anywhere you damn well please. SitePen is an entirely virtual organization.
• Travel:10-25% travel, depending on personal choices regarding conferences and symposiums. Presenting papers and speaking on your work is encouraged.
• Job Type: Full Time
• Reports To: Director of R&D (Alex Russell)

Uniquely for an R&D job, there are no minimum education requirements. We only care how effectively you can advance the state of the art on the Open Web. If that sounds like a calling you can get behind, I’d love to hear from you.