Which it is not.

Before I get to why, let me first say some nice, well-earned things about WebIDL: first, it has helped us out of the ad-hoc IDL sludge that used to be how APIs for JavaScript have been exposed in the past. It has shaved off many sharp edges and is giving spec authors a single dialect in which to write their API descriptions. From a browser perspective, this is a Very Good Thing (TM). Next, the draft in question contains some wonderful changes from the status quo, particularly the addition of a sane prototype to all WebIDL-specified objects.

That all sounds good, so what’s missing?

In a word, constructors.

Well, a lot more than that, but I’d settle for constructors. Functionally speaking, it boils down to the fact that WebIDL makes spec authors do extra work to make something like this sane:

new HTMLDivElement();

Why doesn’t this work today? Funny story…see, HTML defines HTMLDivElement as a regular WebIDL interface. WebIDL doesn’t really have the notion of concrete classes, just interfaces with and without constructors. Since the HTML spec is just doing what most specs will do — adding the smallest IDL you can get away with — the JS usability of this API is left in limbo; neither clearly HTML5′s responsibility nor WebIDL’s.

So what should a contentious version of HTML5 do? One answer is to specify a constructor, turning the IDL from this:

interface HTMLDivElement : HTMLElement {};

to this:

[Constructor]
interface HTMLDivElement : HTMLElement {};

Repeat ad-infinitum for each and every interface that should be constructable in every single spec that browser vendors ever implement. Don’t miss any! And please make sure that all your spec editors are on-board with good JS APIs as a goal! As it stands today, WebIDL doesn’t even force most spec authors to consider the question “do I need a constructor here?” — spoiler: yes — let alone the obvious follow-ups like “what arguments should one take?”.

The obvious better answer here is to flip the default on interfaces, causing them to generate constructors by default unless turned off with [NoConstructor] attributes or specified as partial interfaces (i.e., mixins or traits).

Cameron McCormack who is heading up the WebIDL effort tweeted in response to my exasperation that:

I think a “W3C Web API design guidelines” document would be a perfect place for such a recommendation.

For serious? Such a document might be useful (and I’m working on something that might pass as a first draft), but what’s the argument against flipping the default here? This isn’t a dissent on the facts of the situation: most WebIDL “interfaces” that are exposed to JS are things that could be easily new‘d up to useful ends. Most specs flub this in spectacular style. Most spec authors seem entirely ignorant of the problem and the design language of WebIDL continues to lead down a primrose copy-and-paste path that has little overlap with sanity. So why punt the decision? And why did it take and act of coordination with TC39 to get the prototype thing fixed?

And Why Are We Having This Discussion Anyway?

WebIDL, for all of its virtues, is deeply confused.

If you’re reading any of the stuff in the HTML5 spec that’s describing its API this way, it’s hard to see how it would have any sane relationship to JavaScript. Sure, you could argue that there might be other languages that matter, other languages for which you’d need to be able to generate some API, but none of them rise to anything like the importance of JavaScript. It is the programming language of the web, so if WebIDL has any animating force at all, it’s JS. Then there’s the “accident of history” aspect. Early DOM was specified as a form of IDL in part because there was some expectation that other languages would need to consume it and IDL was how C++ hackers (who still make up the entire set of people working on browser engines) are/were comfortable in describing their FFIs thanks to the legacy of COM/CORBA. Hilarious examples of multi-language-ism still persist in the WebIDL spec for no apparent reason whatsoever, warping the entire design around the altar of an ideal that is either quixotic or vestigial depending on which argument you give more weight.

Since the debate was re-kindled thanks to a debate at a TC39 meeting in July, I’ve been on the receiving end of more than one webdev’s rant about DOM’s JS incoherence, generally taking the form:

Why the *#!*?^@$ isn’t DOM just #@!*@ing specified in JavaScript?

To be honest, I have no answer aside from pointing to the IDL history, the fact that browser hackers don’t tend to write JS so don’t feel the pain, and noting that WebIDL is better in some important ways. Certainly these interfaces could be specified in a subset of JS with annotations for where native behavior is required. But their larger lament has merit too: seamlessness with JS is the bar WebIDL should be judged by. I.e. does it help spec authors do the right thing by JS devs? Or does it lead them down paths that make their generated APIs stick out like sore thumbs, full of anti-social/alien behavior such that you can’t think of them as “regular” JS?

Yes, constructors are only one minor step toward reaching this aspiration, but the fact that WebIDL has gotten to last-call without a reasonable solution to them speak volumes. If WebIDL isn’t animated by the need of JS developers, it would be good if that could be loudly called out somewhere so that the community can have the spirited debate that this point demands. If it is, can we please get on discussing how best to ensure that most “interfaces” generate constructors and stop punting?

Either way, WebIDL isn’t done yet.

Update: It occurred to me, as part of the discussion in the comments, that the provision against new with a class or type of any type is completely non-sensical in JS, as is the lack of call() and apply() methods on them. Idiomatic subclassing requires that the mixin-style be available, which uses ClassName.call(this). This is what you’d do with things that are “virtual” or “partial” interfaces if you’re describing them in actual JS. And there’s no real problem with folks new-ing them up. Doesn’t happen, doesn’t matter. Strong restrictions against it are, to quote Andrew DuPont, anti-social. Long story short: there is absolutely no reason whatsoever to disable construction on any WebIDL interface. It’s deeply non-sensical from the JavaScript perspective.

Wait, what? Why does Alex Russell hate feature testing, kittens, and cute fuzzy ducklings?

I don’t. Paul warned me that my approach isn’t going to be popular at first glance, but hear me out. My assumptions are as follows:

• Working is better than busted
• Fast is better than slow
• No browser vendor changes the web-facing features in a given version. Evar. Does not happen

If you buy those, then I think we can all get some satisfaction by retracing our steps and asking, seriously, what is the point of feature testing?

Ok, I’ll go first: feature testing is motivated by a desire not to be busted, particularly in the face of new versions of UA’s which will (hopefully) improve standards support and reduce the need for hacks in the first place. Sensible enough. Why should users wait for a new version of your library just ’cause a new browser was released or because you didn’t test on some version of something.

Extra bonus: if you don’t mind running them every time, you can write just the feature test and your work is done now and in the future! Awesome! Except some of us do mind. Yes, things are now resilient in the face of new UA’s and new versions of old ones, but only on the back of testing for everything you need ever time you load a library on a page. Slowly. Veeerrrrry slooowly.

Paul suggested that some library could use something like Local Storage to cache the results of these tests locally, but this hardly seems like an answer. First, what if the user upgrades their browser? Guess you have to cache and check against the UA string anyway. And what about the cost of going to storage? Paul reported that these tests can be wicked expensive to run at all, on the order of 30ms for the full suite (which you hopefully won’t hit…but sheesh). Reported worst-case for has.js is even worse. But apparently going to Local Storage is also expensive. And we’re still running all these tests in the fast path the first time anyway. If we think that they’re so expensive that we want to cache the results, why don’t we think they’re so expensive that we don’t want to run them in the common case?

Now for a modest proposal: feature tests should only ever be run when you don’t know what UA you’re running in.

Feature testing libraries should contain pre-built caches — the kind that come with the library, not the kind that get built on the client — but they should only be consulted for UA versions that you know you know. If we assume that behavior for UA/version combination never changes, we’ve got ourselves a get-out-of-jail free card. Libraries can have O(1) behavior in the common case and in the situations where feature testing would keep you from being busted, you’re still not busted.

So what’s the cost to this? Frankly, given the size of some of the feature tests I’ve seen, it’s going to be pretty minimal vs. the bloat the feature tests add. All performance work is always a tradeoff, but if your library thinks it’s important not to break and to be fast, then I don’t see many alternatives. New versions of libraries can continue to update the caches and tests as necessary, keeping the majority of users fast, while at the same time keeping things working in hostile or unknown environments.

Anyhow, if you’re a library author or maintainer, please please please consider the costs of feature tests, particularly the sort that mangle DOM and or read-back computed layout values. Going slow hurts users, hurts the web, and hurts the culture of performance that’s so critical to keeping the platform a viable contender for the next generation of apps. We owe it to users to go faster.

A quick aside: I hesitated writing this for the same reasons that Paul cautioned me about how unpopular this was going to be: there’s a whole lot of know-nothing advocacy that’s still happening in the JS/webdev/design world these days, and it annoys me to no end. I’m not sure how our community got so religious and fact-disoriented, but it has got to stop. If you read this and your takeaway was “Alex Russell is against feature testing”, then you’re part of the problem. Think of it like a feature test for bogosity. Did you pass? If so, congrats, and thanks for being part of the bits of the JavaScript universe that I like.

Less experienced JS programmers often feel as though they’re waltzing in a minefield, though. At many steps along the path to JS enlightenment everything feels like it’s breaking down around you. The lack of block lexical scope sends you on pointless errands, the various OO patterns give you fits as you try to do anything but what’s in the examples, and before you know it even the trusty “dot” operator starts looking suspect. What do you mean that this doesn’t point to the object I got the function from?

Repairs for some of the others are on the way in ES6 so I want to focus on the badly botched situation regarding “promiscuous this“, in particular how ES5 has done us few favors and why we’re slated to continue the cavalcade of failure should parts of the language sprout auto-binding.

Here’s the problem in 5 lines:

1
2
3
4
5

var obj = {
  _counter: 0,
  inc: function() { return this._counter++; },
};
node.addEventListener("click", obj.inc);

See the issue? obj.inc results in a reference to the inc method without any handle or reference to its original context (obj). This is asymmetric with the behavior we see when we directly call methods since in that case the dot operator populates the ThisBinding scope. We can see it clearly when we assign to intermediate variables:

var _counter = 0; // global "_counter", we'll see why later
var inc = obj.inc;
obj.inc(); // 1
obj.inc(); // 2
inc(); // 1

Reams have been written on the topic, and ES5′s belated and weak answer is to directly transcribe what JS libraries have been doing by providing a bind() method that returns a new function object that carries the correct ThisBinding. Notably, you can’t un-bind a bound function object, nor can you treat a bound function as equal to its unbound ancestor. This, then, is just an API formalism around the pattern of using closures to carry the ThisBinding object around:

var bind = function(obj, name) {
  return function() {
     return obj[name].apply(obj, arguments);
  };
};
// Event handling now looks like:
//   node.addEventListener("click", bind(obj, "inc"));
 
var inc = bind(obj, "inc");
obj.inc(); // 1
obj.inc(); // 2
inc(); // 3
 
inc === obj.inc; // false

ES5′s syntax is little better but it is built-in and can potentially perform much better:

var inc = obj.inc.bind(obj);
// In a handler:
node.addEventListener("click", obj.inc.bind(obj));

Syntax aside, we didn’t actually solve the big problems since unbound functions can still exist, meaning we still have to explain to developers that they need to think of the dot operator doing different things based on what charachters happen to come after the thing on the right-hand side of the dot. Worse, when you get a function it can either be strongly-bound (i.e., it breaks the .call(otherThis, ...) convention) or unbound — potentially executing in the “wrong” ThisBinding. And there’s no way to tell which is which.

So what would be better?

It occurs to me that what we need isn’t automatic binding for some methods, syntax for easier binding, or even automatic binding for all methods. No, what we really want is weak binding; the ability to retrieve a function object through the dot operator and have it do the right thing until you say otherwise.

We can think of weak binding as adding an annotation about the source object to a reference. Each de-reference via [[Get]] creates a new weak binding which is then used when a function is called. This has the side effect of describing current [[Get]] behavior when calling methods (since the de-reference would carry the binding and execution can be described separately). As a bonus, this gives us the re-bindability that JS seems to imply should be possible thanks to the .call(otherThis) contract:

var o = {
  log: function(){
    console.log(this.msg);
  },
  msg: "hello, world!",
};
 
var o2 = {
  msg: "howdy, pardner!",
};
 
o.log(); // "hello, world!"
o2.log = o.log;
// calling log through o2 replaces weak binding
o2.log(); // "howdy, pardner!"

But won’t this break the entire interwebs!?!?

Maybe not. Hear me out.

We’ve already seen our pathological case in earlier examples. Here’s the node listener use-case again, this time showing us exactly what context is being used for unbound methods:

document.body.addEventListener("click", function(evt) {
  console.log(this == document.body); // true in Chrome and FF today
}, true);

We can think of dispatch of the event calling the anonymous function with explicit ThisBinding, using something like listener.call(document.body, evt); as the call signature for each registered handler in the capture phase. Now, it’s pretty clear that this is whack. DOM dispatch changing the ThisBinding of passed listeners is an incredibly strange side-effect and means that even if we add weak binding, this context doesn’t change. At this point though we can clearly talk about the DOM API bug in the context of sane, consistent language behavior. The fact that event listeners won’t preserve weak binding and will continue to require something like this is an issue that can be wrestled down in one working group:

node.addEventListener("click", 
       (function(evt) { ... }).bind(otherThis),
       true);

The only case I can think of when weak bindings will change program semantics is when unbound method calls in the global object do work on this in a way that is intentional. We have this contrived example from before too, but as you can see, it sure looks like a bug, no?

1
2
3
4
5
6
7
8
9
10
11
12

var _counter = 0; // a.k.a.: "this._counter", a.k.a.: "window._counter"
var obj = {
  _counter: 0,
  inc: function() { return this._counter++; },
};
var inc = obj.inc;
obj.inc(); // 1
obj.inc(); // 2
console.log(obj._counter, this._counter); // 2, 0
inc(); // 1
inc(); // 2
console.log(obj._counter, this._counter); // 2, 2

If this turns out to be a problem in real code, we can just hide weak bindings behind some use directive.

Weak binding now gives us a middle ground: functions that are passed to non-pathological callback systems “do the right thing”, most functions that would otherwise need to have been bound explicitly can Just Work (and can be rebound to boot), and the wonky [[Get]] vs. [[Call]] behavior of the dot operator is resolved in a tidy way. One less bit of unexploded ordinance removed.

So the question now is: why won’t this work? TC39 members, what’s to keep us from doing this in ES6?

Update: Mark Miller flags what looks to be a critical flaw:

var obj = {
  callbacks: [],
  register: function(func) {
    for (var i = 0, i < this.callbacks.length; i++) {
      this.callbacks[i]();
    }
  },
};
obj.register(foo.bar); // Does the wrong thing!

The problem here is our call into each of the callback functions which still execute in the scope of the wrong object. This means that legacy code still does what it always did, but that’s just as broken as it was. We’d still need new syntax to make things safe. Ugg. ]]> http://infrequently.org/2010/09/javascript-uxo-removal/feed/ 4 http://infrequently.org/2009/04/ending-the-gajs-wait/ http://infrequently.org/2009/04/ending-the-gajs-wait/#comments Sun, 12 Apr 2009 04:10:16 +0000 alex http://alex.dojotoolkit.org/?p=908 Google Analytics is ubiquitous, not least of all because it’s better at what it does than most of the alternatives. Also, it doesn’t require any install or maintenance. And it’s free. What’s not to like?

Frankly, not much, but if I had to nit pick, I’d note that the worst part of Google Analytics is the ga.js script that does the actual tracking of content. It’s not bad, in and of itself, but it tends to load slowly. There are several reasons for this:

• Another DNS lookup to resolve http://www.google-analytics.com/. This DNS entry is likely to be “warm” given how frequently ga.js is used on the web, but as Jim Roskind explained on the Chromium blog, it’s the outliers that kill you.
• It’s kinda big. At 9K on the wire (22K unzipped), ga.js is kinda chunky for what it does most of the time, namely tracking a single page load.
• The default instructions are bone-headed. They direct you to do a document.write() which is a blocking, synchronous operation WRT page loading. This is tres dumb. Reasonable people should just include ga.js with a <script> tag, but nearly nobody does. Turns out that sane defaults still matter.
• Load times seem totally random. As with DNS lookup, ga.js‘s latency varies wildly. This isn’t backed up by anything empirical, but many pages feel blocked by ga.js for a near eternity.

So how to fix this? Dojo 1.3′s dojox.analytics.Urchin to the rescue! Given that I was going to be including Dojo on the page to do other things anyway, I can use 1.3′s new Urchin system to help amortize the cost of using Google Analytics. The code running on this blog now includes the following code (more or less):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

<script type="text/javascript"
  src="http://ajax.googleapis.com/ajax/libs/dojo/1.3/dojo/dojo.xd.js">
</script>
<script type="text/javascript">
  dojo.addOnLoad(function(){
    setTimeout(function(){
      dojo.require("dojox.analytics.Urchin");
      dojo.addOnLoad(function(){
        var tracker = new dojox.analytics.Urchin({
          acct: "UA-XXXXXX-X" // your tracking # here
        });
      });
    }, 100);
  });
</script>

You can see the real thing by looking at the bottom of this page which pulls in custom.js which includes this logic. Pete Higgins blogged about how the module works when he first wrote it, and the strategy is to load Dojo from a CDN (since the page wanted it for other things) and wait until after the page has loaded to include ga.js. This delayed loading ensures that the page is responsive before we start doing anything related to tracking. The nested calls to dojo.addOnLoad show how we can wait for one set of modules to finish before kicking off another group, and in this case we also wait until after the page is responsive to load dojox.analytics.Urchin. This module further delays loading of ga.js, meaning that it doesn’t matter how long it takes for things like DNS to resolve and for Google to serve ga.ja since it’s not ever blocking the user.

Looking at the “before” and “after” shots in firebug gives you some idea of how this technique can really make a difference:

onload waits for ga.js

using dojox.analytics.Urchin, we can get a responsive page and *then* track usage

We get page tracking and the user gets an interactive page faster. Rad.

Maybe.

These things matter only in regards to how well they represent end-user workloads and how trustworthy their findings are. The first is much harder than the second, and end-to-end benchmarking is pretty much the only way to get there. As a result, sites like Tom’s Hardware focus on application-level benchmarks while still publishing “low level” numbers. Venerable test suites like SPECint have even moved toward running “full stack” style benchmarks which may emphasize a particular workload but are broad enough to capture the wider system effects which matter in the real world.

Marketing departments also like small, easily digestible, whole numbers. Saying something like “200% Faster!” sure sounds a lot better than “on a particular test which is part of a larger suite of tests, our system ran in X time vs. Y time for the competitor”. Both may be true, but the second statement gives you some context. Preferably even that statement would occur above an actual table of numbers or graphs. Numbers without context are lies waiting to be repeated.

With all of this said, James Ward’s Census benchmark makes a valiant stab at a full-stack test of data loading and rendering performance for RIA technologies. Last month Jared dug further into the numbers and found the methodology wanting, but given some IP issues couldn’t patch the sources himself. Since I wasn’t encumbered in the same way I thought I might as well try my hand at it, but after hours of attempting to get the sources to build, I finally gave up and decided to re-write the tests. The result is Census 2.

There are several goals of this re-write:

• Fairness. Tests need to be run multiple times for them to be representative in any way. Likewise, systems not being directly tested need to be factored out as much as possible. C2 does this by reducing the number of dependencies and running tests (at least) 5 times and discarding outliers before reporting an average. I’ve also worked to make sure that the tests put the best foot forward for all of the tested technologies.
• Hackability. Benchmarks like Census serve first as a way for decision makers to understand options but second as a way for developers to know how they’re doing. Making it trivial to add tests helps both audiences.
• Portability. The test suite should run nearly everywhere with a minimum of setup and fuss. This ensures that the largest numbers of people can benefit from the fairness and hackability of the tests.

The results so far have been instructive. On smaller data sets HTML wins hands-down for time-to-render, even despite its disadvantage in over-the-wire size. For massive data sets, pagination saves even the most feature-packed of RIA Grids, allowing the Dojo Grid to best even XSLT and a more compact JSON syntax. Of similar interest is the delta between page cycle times on modern browsers vs their predecessors. Flex can have a relatively even performance curve over host browsers, but the difference between browsers today is simply stunning.

Given the lack of an out-of-the-box paginating data store for Flex, RIAs built on that stack seem beholden to either Adobe’s LCDS licensing or are left to build ad-hoc pagination into apps by hand to get reasonable performance for data-rich business applications. James Ward has already exchanged some mail with me on this topic and it’s my hope that we can show how to do pagination in Flex without needing LCDS in the near future.

The tests aren’t complete. There’s still work to do to get some of the SOAP and AMF tests working again. If you have ideas about how to get this done w/o introducing a gigantic harball of a Java toolchain, I’m all ears. Also on the TODO list is an AppEngine app for recording and analyzing test runs so that we can say something interesting about performance on various browsers.

Census 2 is very much an open source project and so if you’d like to get your library or technology tested, please don’t hesitate to send me mail or, better yet, attach patches to the Bug Tracker.

Update: I failed to mention earlier that one of the largest changes in C2 vs. Census is that we report full page cycle times. Instead of reporting just the “internal” timings of an RIA which has been fully boostrapped, the full page times report the full time from page loading to when the output is responsive to user action. This keeps JavaScript frameworks (or even Flex) from omitting from the reports the price that users pay to download their (often sizable) infrastructure. There’s more work to do in reporting overall sizes and times (“bandwidth” numbers don’t report gzipped sizes, e.g.), but if you want the skinny on real performance, scroll down to the red bars. That’s where the action is.

Here we go again.

Every four years we go into the partisan spasms of a heat-not-light [dis]enfranchisement “debate”. What’s perpetually lost is any coherent discussion of the causes of the debate. As with gerrymandering, the causes are structural and neither party feels it would serve them to effectively deal with the situation at it’s roots. Not because solving the problem wouldn’t be better for everyone (in both cases, it objectively would), but because it would change the game, and our 2-party system is nothing if not over-specialized to the current rules of the game. In this sense, both redistricting reform (killing gerrymandering dead) and automatic voter registration systems cause the old rules to lose their hold, instantly changing the calculus of holding on to power. Automatic registration via so-called “motor-voter” laws, registration with passport applications, automatic updates with change-of-address on US mail, and other sensible policies are opposed by partisans of both parties because they have the potential to allow discontent to cause large swings in electorate behavior. Today, between the systematic disenfranchisement caused by requiring citizens to register to vote and the effective gerrymandering-based dampening of popular will, enormous changes in likely voter feelings about the direction of the country (or, less often, a locality/district) are required to break an incumbent’s grip on power. As an example, the “right-track/wrong-track” polling numbers in 2006 said that nearly 70% of Americans thought the country was on the wrong track. That feeling swept the Democrats into a razor-thin congressional majority (which the Republicans have been filibustering to death ever since), but in a 2:1 year, wouldn’t you expect a similar landslide in Congress? Perhaps not in the Senate where fewer seats are up for a vote as a structural hedge against change, but certainly we should have collectively turned enough of the bums out to make the remaining ones fear for their jobs. As it happened, in a race with 33 seats up for discussion (15 held by Republicans), only 5 of the bums lost their seats. Now compare that to the most recent pre-Bush period of comparable “right-track/wrong-track” numbers: the Republican revolution” of ’94. On that basis it becomes evident that a shift in power half-again as large – based on the same “market fundamentals” – was at least justified. 8 seats changed hands in ’94 vs. 5 in ’06. The difference? A census and therefore a chance for many states to gerry….er…redistrict. The calcifying power of…um…power worked its magic 2 years ago and the Senate is deadlocked as a result.

All of these effects cause fewer parts of the country to be competitive. If you support competition in markets (as I do), then there’s no higher calling than supporting a competitive marketplace for ideas, and both parties are doing their best to stifle the pricing function of our public policy market. No wonder then that Republicans think they can steal (or at least dispute) an election by challenging voters in Ohio, despite many discredited attempts in the past to find large-scale vote fraud. Eight years of intense Republican scrutiny into the non-issue has utterly failed to turn up any massive conspiracy or even credible small-scale fraud. Remember, pressuring US AG’s to prosecute thin or non-existent voter fraud cases – then firing them when they couldn’t find any and/or wouldn’t make them up – is what got Alberto Gonzalez hauled before congress for months on end, eventually leading to his disgraced resignation and continuing questions as to the impartiality of the DOJ. This isn’t a winner of an issue for the GOP, but it sure is an effective way to try to keep the under-informed from coming to the polls and having their votes counted. And they know it. Scare-tactics and fear-mongering as [keep|get]-out-the-vote strategy. Democrats, naturally, feel that their on-the-ground vote registration in targeted areas can be an effective weapon in distorting the outcome.

Either way, it’s shameful.

The Democratic approach is perhaps less reprehensible since the folks who are registered can vote however they please, but there’s no denying the attempt at demographic distortion. But why should citizens need to register as a separate process from establishing residency, getting mail, or a driver’s license anyway? And why isn’t same-day registration the law of the land? If Wyoming can do it, why not New York and California?

Fundamentally, all the objections to broad enfranchisement end in a large-scale IT problem…but not a particularly hard one. What I’m not hearing from nearly anyone is a recognition that this shouldn’t be a debate that happens in a functioning democracy. That we have it on a quadrennial basis is simply an embarrassing acknowledgment that our democracy is not functioning.

If McCain really wants to put “country first” and be a maverick, he’d be bringing this up as an urgent topic of national debate. It would certainly be better than the sleazy robo-call effort his campaign is now investing in. And if Obama wanted to build real credibility with Americans about his willingness to effect real change, shouldn’t he start here? Certainly it’s a better use of the national discourse than helping McCain draw unwarranted (and likely un-wanted) attention to a plumber. And where the hell is the 4th Estate when we need them?

Oh, right. Camped out on some plumber’s lawn.

Luckily, I happen to live in California where the Governor (who I disagree with more than not) has put redistricting reform on the ballot in the form of Prop. 11. I’ll be voting “yes” on 11, if only to make it that less likely that 4 years from now we’ll have to have this “discussion” again.

While in Boston, Gavin and Jill joined a gaggle of Dojo hackers at a dinner ostensibly to mourn my birthday (thanks to Dylan and Pete for organizing!) and in the course of conversation Jill asked something along the lines of:

So why do people get so excited about closures?

Which prompted a several of us to flail and flop gasping the salt flats of analogy like fish out of polite-conversation water. After about 10 minutes of this, Jill succinctly summed it all up in the form of a question:

Oh, so it’s like “action-oriented programming”?

This is perhaps the most insightful and succinct description I have ever heard of what JavaScript is all about.

Update: Jennifer just played this for me and it gets right to the heart of this post: the important part of doing what we do with computers, and more importantly, with the web, is to give the power of Computer Science to real people…and it starts with insights like Jill’s that build a shared way of thinking and talking about the world. It makes me sad that many programmers miss that, but when non-programmers can share in the beauty and power of code, it does a lot to make it all seem worthwhile.

So what does Chrome mean for those of us who aren’t breaking out the champagne? Well, first, it’s the second sign (after Gears and YBP (har!)) that the content authors are taking back the web from the “browser guys”. I’ve been fascinated for the last 6 months or so by the strategic mis-alignment which results when both the browsing and authoring experience in the hands of organizations only care about one but not the other. Mozilla gets paid by search-box revenue and users download it because it’s better for browsing, therefore Mozilla is incented to build new ways to browse, but their investments in content are somewhat mis-aligned (and, frankly, it shows). Google and Yahoo, on the other hand, are critically dependent on the content getting better, so they produce plugins to augment HTML in un-intrusive ways. Chrome crosses over into the browser business from the perspective of content, and it also shows, albeit in a good-ish way. I guess we’ll need to wait and see how browsing-oriented Chrome gets (e.g., will it sprout an extensions platform – ala Firefox – or will the propsect of an ad-blocking plugin for the Google browser make that proposal D.O.A.?).

Regardless of how Chrome evolves as a product, the important question now is: how will it be distributed? The obviously non-evil thing to do is to say “look, it’s great, it’s free” and hope that the world discovers it on its own thanks to word-of-mouth and/or leverage of the Google brand. Given that Chrome delivers new awesome things which are end-user-visible (some “end-user-awesome”, if you will), there’s some real chance that Chrome can get to roughly Firefox level market-share without breaking too much of a sweat. Not that Firefox’s market share is anything to really covet, given that MoFo/MoCo have been toiling at it for a decade now. To get real, honest-to-god leverage out of this process, Chrome is going to need something like 60+% market share, and that means changing ingrained user habits. I put the probability of that happening without distribution channel love at roughly bupkis.

Microsoft killed Netscape by bundling the browser with the OS. Apple is making inroads by bundling. Firefox is even getting aggressive. So where does this leave “don’t be evil”? Given the toolbar promotional deals which Google has cut in the past, I think there’s some organizational capacity inside the Goog to use the distribution channels they’ve already created as a way of getting to critical mass. What I don’t see, though, is a view of how to bring the mission of Gears into alignment with Chrome (or vice versa). They’re both important, but Chrome is a long-term bet while Gears is the near-future solution. They are not in opposition, but their strategies for gaining leverage over the problems facing content authors are very different.

We need what Gears can offer to every browser right now while Chrome dukes it out for market share on the browsing experience merits. Hopefully, if nothing else, the Chrome installer will add Gears to other browsers on the system that users install Chrome to. Even if they don’t pick the googly experience for browsing day-to-day, perhaps Chrome can still serve to give new tools to the content-author side of the house. Other browser vendors won’t do such a thing since they win or loose on an exclusive “I must replace the other guy” basis. Here, Google (and by “Google”, I mean “the open web”) wins either way. Hopefully Google’s interest in making the content experience better trumps the “we’re all browser guys now” instinct in this case.

We’ll find out tomorrow, I guess. Here’s to hoping.

At the risk of talking too much out of school, I want to set the record straight in some ways. First, let me set some facts out:

• ES4, as outlined last fall, was not ActionScript 3. Major changes to the semantics of Adobe’s initial contribution ensured that any truly compliant ES4 implementation from Adobe would have required many concessions, including the inclusion of a client-side compiler. ES4 without eval() (ala AS3) would never have cut it as a “real” ES4.
• Adobe had not donated “ActionScript 3″ to Mozilla nor Open Sourced ActionScript 3. Adobe had donated a high-performance byte-code VM which a separate front-end compiler targeted. It is absolutely possible that the Tamarin VM can and will run whatever syntax for the next version of JavaScript is ratified. The design of the VM is, of course, predicated upon the language semantics but lets not confuse the front-end compiler and the language that it consumes with the VM and byte-code format which it executes. The front-end compiler was not donated to Mozilla (although something similar exists in the OSS’d Flex SDK) nor does it ship inside of the Flash player today. Adobe can continue to evolve their developer languages and byte-code VMs independently so long as they never ship an eval() feature. Even if they do, there are other (harder to swallow) options available, but no road is cut off to Adobe which was not already long-since abandoned in other ways by the ES4 process.
• Like Adobe, Microsoft has jumped ahead in the evolution of JavaScript via the seemingly forgotten JScript.NET. Like Adobe, Microsoft has had to come back from that position to meet the web where it really is. Microsoft now ships 3 – yes, that’s right, three – JavaScript-ish languages which are capable of running in a browser:
  • JScript (via Windows Scripting Host)
  • JScript.NET
  • JScript for the DLR (via Silverlight)

  Make no mistake about it, these are all separate implementations which likely share little if any code. The DLR variant is even new in the last several years. Creating a new JavaScript compiler and runtime is not an overly onerous task for MSFT and clearly not one that will take a long time should the occasion arise. What’s confusing and can likely be tied to strategic wrangling is the puzzling lack of progress in the WSH version of JScript (the one everyone uses day-to-day). Any strategic discussion of JScript as a platform needs to start from this perspective.

• Much in ES4 was new. The gradual typing system (which I’m a big supporter of) was something of a large-scale experiment coupled with as much rigor as I’ve ever seen in a standards body in proving things out in an RI. The Class system (which I wasn’t a big supporter of) needed to marry class-based inheritance to prototypal inheritance in ways which were new. ES4 contained many good features and syntactic forms which were borrowed from other places, but as Brendan Eich has said, it was a process of synthesis plus invention. In my opinion, much of the failure of ES4 as an initiative can be traced to significant failings of ActionScript 3 as a language. AS3 is a Java-like language with a Java-like execution model. JavaScript is a dynamic language with a dynamic execution model. The gulf in expectations between those camps turns up in every corner of the language design. For instance: what does it mean to load new code? Are classes forever “closed” or can they be re-opened? Do you really need the static and final keywords? Solving these issues in the context of the old is easy. In the context of the new, it’s hard. ES4 suffered because it had to do it in a new world which no one was yet writing code for.

So, lets pop up and talk about strategy for a minute. Fundamentally, very little has changed in terms of available strategic options for any of the players:

• MSFT can still hold the web hostage to their ailing WSH VM by continuing to ignore its performance, regardless of bug fixes and syntactic updates. Doesn’t matter if it’s amputation or debilitating arthritis, crippled is crippled. For what it’s worth, my interactions with the MSFT reps on TC39 give me no reason to believe that they won’t be improving their VM.
• Adobe can still chose to implement a language which implements an ECMA spec. They can do this any time they damn-well please. It may not align so cleanly with their current technology roadmap, but it’s absolutely feasible and when Mozilla ships a “regular JavaScript” front-end to the Tamarin back-end, it’ll be even easier. Note that this plan can be done in parallel with AS3 evolution since Tamarin (as a VM) need not be aware of any language semantics on its own.
• Browser vendors can still wring large speedups out of ES3 and 3.1

What died here wasn’t Adobe’s attempts to “own” a spec. If there were such hopes in play, they had been quietly put down one rational, backwards compatible decision after another in the year preceding the Oslo meeting. What died was an assumption that the web can evolve without implementations being out in front of the spec. AS3 was one implementation of a JavaScript-like language that might have been a contender for crown of “the next one”, but so was JScript.NET. That neither of them “won” simply because they had been built in good faith is as true a sign as I can find that the process is working. Adobe gets it. Lets end the silly meme that “Adobe lost” or that “Microsoft won”. The game has hardly begun and it won’t be settled in a standards body anyway. What matters – and what we all need to keep our eyes keenly trained on – is what the big implementations do in the way of compatibility, performance, and feature set once ES3.1 arrives.

I’ve had the pleasure of working with Peter both in the Dojo project and at SitePen and his energy and enthusiasm for making Dojo better and helping designers and developers work better together is infectious.

For anyone not familiar with Peter’s work, he’s been instrumental in the creation of the amazing Dojo Campus website (along with the Uxebu chaps) as well as being primary author of many DojoX components. His work in shepherding new contributors through the contributor to committer process is nearly legendary inside the project, and Peter has been a one-man outreach and support machine via the Dojo blog and his endless patience on the forums and IRC (#dojo on irc.freenode.net). I couldn’t be happier about where Dojo is headed under his direction.

There have already been some recurring questions about this transition amongst the committers and on today’s Open Web podcast, so let me quickly recap them here:

Q: Will you still be involved in Dojo?

Absolutely! I’m excited that Pete is taking on the figurehead and “vision thing” duties which is a role that he’s naturally suited to. Part of this transition is about me wanting more time to focus on experimental and edgy stuff that can make a huge difference in how we work with the web and I have no doubt that Peter is the right guy to help us grow the truly open Dojo community even further. He absolutely gets the importance of a truly open community, the need to be conservative about where IP comes from and meet our promises of backwards-compatibility, and how Dojo can make big changes in the lives of application developers and designers. I’m grateful to be have the opportunity to continue working with him on Dojo and will continue to do so in whatever capacity Peter deems appropriate.

Q: Does this change your role at the Foundation?

Nope. I’m still serving as President of the Dojo Foundation. This transition will allow me to also focus more time on ensuring that the Foundation is running well, that a new Board is elected soon, and that the Foundation’s other projects succeed on their own terms. The Foundation has always been about more than just giving Dojo a home, and we’re now looking to expand the umbrella of the Foundation to help nurture other JavaScript and Open Web projects more than ever.

Q: Will you still be doing talks on Dojo?

Yep, I’ll still be out there advocating the Dojo case, but you can expect to see Peter doing more of that over time as well. If you’re planning a conference and are looking for a cogent person to talk about Dojo, Peter is now your go-to guy. I’ve enjoyed having the opportunity to think and talk about where the Open Web is headed, so I’ll also be doing a lot more of that. There are lots of meta-issues that this transition will let me work harder on, so expect more from me there.

My hat’s off to the Dojo community and Peter in particular. The work that has gone into 1.2 and will land in 1.3 and beyond under his direction really is changing the way we view what the web can and should be used for.

Speaking as a member of the OSCON program committee, I’m very happy about the quality of the talks in the web-ish tracks this year. There’s even a Dojo talk – even though for the first time in a long while, I won’t be giving any talks. The inimitable Matthew Russell, author of ORA’s Dojo: TDG will be giving an awesome talk on 2D drawing with Dojo’s GFX system. I know he’s got some awesome demos worked up, so I can’t wait to see the talk. Gavin Doughtie, occasional contributor to the GFX system, is also giving several talks that you’ll find me in. Should be a lot of fun.

On a more macro scale, though, I’ve started to become concerned that “Open Source” as a brand has lost its way. Those who would speak for Open Source have focused narrowly on licensing and have largely ignored the other social processes and artifacts that define what it means to contribute to OSS projects and how those artifacts lead to success or failure of projects, and therefore, of the movement such as it is. There’s a huge disconnect between what the letter of the Open Source law dictates (the licenses) and the social and process constraints that are required to build high-quality, trustable communities that ensure 100 point OSS products, and many businesses have struck on these differences as a way to use the Open Source brand to imply or insinuate that users should trust their products more than is warranted. OSI’s failure to address this brand erosion has had some troubling effects in the small JavaScript corner of the OSS world of late, and I know we’re not alone. OSI has also proven completely impotent in preventing license proliferation, further eroding the Open Source brand. There are, of course, lots of folks who are also concerned about these thing, and so I’m excited to see David Recordon (of OpenID, etc. fame) giving a talk which looks to talk about some of the community aspects. I tend to blow off “community” talks at conferences, but given David’s use of the phrase “Open Web” and his unique perspective, I’ll be interested to see what he says. I’ll also be curious to see if and how any of this is discussed at the FLOSSCON meeting of OSS Foundation leaders tomorrow and Sunday.

If you’ll be in Portland next week, don’t hesitate to join us for the dojo.dinner() on Wed. I’m looking forward to seeing everyone again and talking though the issues. Should be a great time.

Congrats guys!

Catch this fun gem from FF3b2:

>>> typeof []
"object"
>>> var a = [];
>>> var b = new Array();
>>> a.constructor == b.constructor
false
>>> c = [];
[]
>>> c.constructor == a.constructor
true
>>> d = new Array();
[]
>>> b.constructor == d.constructor;
true
>>> b.constructor == a.constructor;
false

WTF?

Now we need to know a couple more details to see if it’s going to have real legs:

• What is the precedence for resolution of conflicting rules? If the compatible rule is provided as an HTTP header and as a meta tag, which wins? If multiple tags occur, is it first or last wins?
• If a rule is provided that ignores IE (assuming other renderers follow suit), and a subsequent meta tag shows up which specifies a rule for IE, will IE handle it correctly?

  For instance, if the following occurs in a document:

  
<meta http-equiv="X-UA-Compatible" content="FF=3" />
...
<meta http-equiv="X-UA-Compatible" content="IE=8" />


  What policy will IE use?

• Will there be any way to specify the box-model behavior on a sub-page basis?
• Will the IE 8 rendering engine now be distributed to users of the IE 7 chrome, invoking itself only when the right meta flags are thrown? And what is Microsoft’s policy toward distributing renderers now that they have logically cut the chrome/renderer chord?
• Where will you be standardizing this convention? When?

These questions need to be answered, and soon. If the IE team has just replaced one scarce resource for another, we’re not much better off over the long haul. It’s great news that the IE team is really implementing the “renderer selection switch” which many of us have dreamed of for so long…but having it continue to be global to the page or in other ways encourage contention on yet another single element in the document wouldn’t be stepping forward enough.

1. Array’s Can’t Be Usefully Subclassed (test case)

   At first blush, this seems wrong. You can use the Array base-class as the prototypal delegate for any user-defined class you wish. Methods are correctly delegated to and hash-style indexes work fine. Almost everything works right…except when you try to use the built-in array manipulation methods like push, pop, and shift. They dutifully change the internal contents of the subclass instance’s indexed attributes, but they don’t manipulate the length property. This means that while you can use for(var x in list){ ... style iteration, you can’t do anything *aside from key iteration* to know how many items are in the array. Obviously, one could try to wrap the intrinsic functions and detect how they manipulate the length property, but then you’ve ruined their [DontEnum] status and now they end up in the iterable surface area of instances. Ugg.

   Arrays without a working length property are nearly useless, and JScript mangles the design of toolkits as a result.

   So how do we get dojo.NodeList to be a “real” array with extra methods then?

   As you might expect, it’s a giant hack. When you use the “new” keyword with the dojo.NodeList function, you expect that the system will create a new instance and do it’s normal “stamp with a constructor” business. Instead, we resort to creating (and populating) a regular Array instance and “NodeList-ifying” it by copying named attributes from the class prototype into the instance as member properties. The “constructor” function then explicitly return a new object, bypassing the “new” keyword’s create/stamp machinery, at which point the return of the new operator becomes our explicit return and not the object which it would have otherwise implicitly returned.

   In Dojo 0.9 we had used an even more aggressively hackish workaround for IE which involved creating a sub-document and mapping its interpreter’s intrinsic Array class into the parent document at a different name. Both are slow for different reasons but we eventually switched to the create-and-mix-in style because some popup blockers were interfering with the old method.

   Lest you think that Dojo is a dirty, dirty toolkit for doing this kind of thing, consider the janktastic “it’s not really an array” thing that JQuery resorts to instead. By giving up all “[]” index operations, JQuery manually maintains it’s internal length property by re-implementing all of push, pop, etc. functions. This has the benefit of allowing prototypal delegation to work of pre-existing instances when new features are added to the base prototype, but at the expense of no longer being able to think about a dense list of things as an array. Dojo’s approach is painful, but so are all the alternatives today.

   I think it’s safe to say that both toolkits would subclass Array directly to save code, were it a reasonable thing to do.

2. Where Art Thou Getters/Setters?

   As JavaScript toolkits get pushed out of their current workhorse tasks of plastering over JavaScript and DOM implementation gaffes by positive browser progress and Moore’s Law, they increasingly take on application construction tasks (e.g., dojo.data). As the toolkits have approached these tasks, we’ve collectively started to hit some very serious usability limitations due, in large part, to JScript’s lack of progress.

   Toolkits like Dojo have widget systems because HTML just hasn’t kept up. That means that these toolkits have a responsibility to keep as many of the positive aspects of the “native” web as they can. From CSS customization to accessibility all the way through implementing declarative creation and DOM-ish JavaScript APIs, the better a job a toolkit can do in making the abstraction feel more solid, the better the toolkit is. Widgets are essentially a way to “subclass HTML elements”.

   In many places, DOM allows you to affect the behavior of the visible document using “setter”-style syntax. For example:

   
   document.getElementById("foo").style.border = "5px dotted black";
   

   Custom widget classes can have the same behavior on every browser except IE.

   This means, of course, that JavaScript toolkits can’t really implement the behavior, backing JavaScript programmers up against a wall when they design their tools. Instead of providing the natural property-oriented behavior, it forces class authors to write getSomeProperty/setSomeProperty method pairs on their classes should they want to do anything when values are gotten or set. The resulting code feels a lot more like Java than JavaScript, which is usually a sign that something is horribly wrong in a browser.

   As bad as this problem is for visual widgets, it’s worse for data binding systems. API’s like dojo.data would be designed in fundamentally different ways if getters and setters were available everywhere. Instead of the rigamarole of making users fetch an item reference and then fetch attribute values using the opaque item handle and the property name, we’d just set up getters and setters on the properties themselves and defer the implementation of fetching those values down to the data store. Further, assigning a linkage between a dojo.data query or store and a widget which represents it could be as simple as assigning a property to the widget object.

   So are workarounds to this possible? I think they are, and I’m testing some of them out for use in Dojo 1.1 right now. I’ll post more about them should they pan out. Every avenue which looks potentially workable right now involves gigantic hacks which also deeply constrain API designs. Fundamentally, this problem can’t be solved without good language-level support.

   It’s perhaps folly to assume that this will be addressed in IE 8, but given the enormous back-pressure of nearly every JavaScript toolkit author demanding this feature and the embarrassment of every other browser beating them to the punch, I have some hope that we could see getters and setters for JScript in the near future. It won’t matter much, though, unless the JScript team ships their new engine to all IE versions when they release IE 8. Not bloody likely.

3. Performance

   Kudos are in order to the JScript team for fixing their long-b0rken GC heuristic and pushing it out to everyone…but it’s the tip of the iceberg.

   Performance is one of those areas where differences in implementations can tightly circumscribe what’s possible despite exacting spec conformance. On this front, JScript’s raw VM-level execution time leaves a lot to be desired, but the true travesties really show up when you hit the DOM for computed style information or try to do anything reasonably complicated that involves string operations.

   Most non-trivial blocks of JS code today rely on innerHTML to bootstrap some new chunk of DOM in response to user action due in large part to the cross-browser speed and size advantages of innerHTML vs. raw DOM methods for equivalent DOM structures. This reality pushes IE’s string performance woes to the fore as more and more client-side systems push far enough to hit the new “wall”.

   Similarly, getting computed box model calculations out of IE is not for the faint of optimization foo. When we profiled Dojo’s widgets to plan our attack for 0.9 and 1.0, we noted very quickly that getting box-model data out of the browser for any element is hugely costly on every browser, but on IE the cost was not just big… it was enormous. Our best guess right now is that the properties on the currentStyle property are re-calculated when they’re requested from script and not cached in the bound object when layout happens. The resulting performance penalty requires that developers nearly never manage layout in code, severely constraining the layouts which are attempted by toolkits like Dojo.

   Across the board, from DOM performance to raw JScript execution speed, IE is a dog, and the odds are good that whatever toolkit you’re using spends a lot of time working around that reality.

4. Doctype Switching

   Doctype switching to toggle box-model behavior is perhaps the single most limiting implementation error in IE. Saner browsers allow you to use a CSS property to affect the layout model in use in a particular section of a document. This makes tons of sense in a templated world where most of the markup your system generates starts and ends in the “news hole”. Today, that covers nearly everyone. A quick line count in any HTML document shows that doctypes are a scarce resource whose scarcity is made problematic when it’s semantics are overloading. I’ve been on product teams where the idea of changing the doctype would require months to recover from. That kind of cost related to what should be simply a markup dialect change (not a formatting policy change) implies strongly that the doctype is a terribly brittle way to control several independent concerns.

   Instead of giving devs fine-grained layout system control, IE makes it all-or-nothing. The global flag approach backs toolkit developers into doing script-based layout calculations or “just throw it in another div” solutions where we’d really rather not. Both are slow and both may be required since it’s completely impractical to dictate to users which doctype they’ll be using. While any app may be able to be disciplined enough to not care, toolkit developers must work everywhere. Hilarity ensues.

   I fear this is going to get even worse with IE8 as the IE team looks to implement some of HTML 5 and hopefully many of CSS 2.1′s clarifications. The sooner they abandon the global switch, the better…but I’ll wager it’s pain they just don’t feel. Building a browser is a very different pursuit from building portable apps to run inside it.

5. HTC’s Can’t Be Inlined (Even With Hacks)

   Modern browsers have built-in widget systems. On IE, it’s HTCs + Viewlink and on Firefox it’s XBL. Even a cursory reading through the docks for both is enough to illuminate the gigantic overlap. Alas, no one is yelling at them to standardize and the result is a terrible mess in which both sub-optimal formats limp along with nearly zero Open Web usage.

   So why do I single out IE for whipping here when XBL is just as lame and similarly b0rken with regards to single-file embedding? Well, on Mozilla, you have a lot more “outs”. I strongly suspect that you can use “data:” urls to generate and evaluate component definitions for FF, which would enable compiling down from a single (more sane) format in the running page environment. IE prevents any such useful code-loading approaches, meaning you have to generate files on disk in their b0rken-ass format in order to be able to use them. Given how far we’ve gotten with non-builtin widget systems, it’s pretty clear that toolkit authors aren’t going to contort themselves into requiring a build step that splats files all over disk just so that we can give IE and Mozilla different views of the same component description. Instead, we all limp along on our own class hierarchies without any of the benefits of element subclassing, getter/setter handling, and inline (scoped) method description that these browser-provided systems would allow. It’s kind of pitiful, really.

   IE8 may include strong “data:” URL support given that it’s needed to pass Acid2, but I’m not holding my breath. I strongly suspect that HTC’s are a dark, unloved corner of the Trident codebase that none of the current engineering team are really fired up about fixing (which they could have done by just allowing Data Islands to contain HTC definitions…but I digress). The takeaway here is that we probably shouldn’t even need JS toolkits to build widget systems, but it’s too late now. We’re an abstraction short and a decade late and now the Flex frankenstein is beating HTML at it’s own game.

In a vacuum of feature data or builds to work with, it’s prudent to assume that IE 8′s DOM and JavaScript implementations will continue to warp attempts at building useful, idiomatic JavaScript libraries to ease the problems that HTML + CSS aren’t effectively solving. So next time you wonder why your toolkit of choice is built the way it is or why it’s even necessary, just remember that in many cases they are protecting you from a decade or more of bad decision making.

From that perspective, they’re worth every penny.