Real Constructors & WebIDL Last Call

For those who haven’t been following the progress of WebIDL — and really, how could you not? An IDL? For the web? I’d like to subscribe to your newsletter… — the standard is now in last call, which is W3C for “alllllllllllmost done”.

Which it is not.

Before I get to why, let me first say some nice, well-earned things about WebIDL: first, it has helped us out of the ad-hoc IDL sludge that used to be how APIs for JavaScript have been exposed in the past. It has shaved off many sharp edges and is giving spec authors a single dialect in which to write their API descriptions. From a browser perspective, this is a Very Good Thing (TM). Next, the draft in question contains some wonderful changes from the status quo, particularly the addition of a sane prototype to all WebIDL-specified objects.

That all sounds good, so what’s missing?

In a word, constructors.

Well, a lot more than that, but I’d settle for constructors. Functionally speaking, it boils down to the fact that WebIDL makes spec authors do extra work to make something like this sane:

new HTMLDivElement();

Why doesn’t this work today? Funny story…see, HTML defines HTMLDivElement as a regular WebIDL interface. WebIDL doesn’t really have the notion of concrete classes, just interfaces with and without constructors. Since the HTML spec is just doing what most specs will do — adding the smallest IDL you can get away with — the JS usability of this API is left in limbo; neither clearly HTML5’s responsibility nor WebIDL’s.

So what should a contentious version of HTML5 do? One answer is to specify a constructor, turning the IDL from this:

interface HTMLDivElement : HTMLElement {};

to this:

[Constructor]
interface HTMLDivElement : HTMLElement {};

Repeat ad-infinitum for each and every interface that should be constructable in every single spec that browser vendors ever implement. Don’t miss any! And please make sure that all your spec editors are on-board with good JS APIs as a goal! As it stands today, WebIDL doesn’t even force most spec authors to consider the question “do I need a constructor here?” — spoiler: yes — let alone the obvious follow-ups like “what arguments should one take?”.

The obvious better answer here is to flip the default on interfaces, causing them to generate constructors by default unless turned off with [NoConstructor] attributes or specified as partial interfaces (i.e., mixins or traits).

Cameron McCormack who is heading up the WebIDL effort tweeted in response to my exasperation that:

I think a “W3C Web API design guidelines” document would be a perfect place for such a recommendation.

For serious? Such a document might be useful (and I’m working on something that might pass as a first draft), but what’s the argument against flipping the default here? This isn’t a dissent on the facts of the situation: most WebIDL “interfaces” that are exposed to JS are things that could be easily new‘d up to useful ends. Most specs flub this in spectacular style. Most spec authors seem entirely ignorant of the problem and the design language of WebIDL continues to lead down a primrose copy-and-paste path that has little overlap with sanity. So why punt the decision? And why did it take and act of coordination with TC39 to get the prototype thing fixed?

And Why Are We Having This Discussion Anyway?

WebIDL, for all of its virtues, is deeply confused.

If you’re reading any of the stuff in the HTML5 spec that’s describing its API this way, it’s hard to see how it would have any sane relationship to JavaScript. Sure, you could argue that there might be other languages that matter, other languages for which you’d need to be able to generate some API, but none of them rise to anything like the importance of JavaScript. It is the programming language of the web, so if WebIDL has any animating force at all, it’s JS. Then there’s the “accident of history” aspect. Early DOM was specified as a form of IDL in part because there was some expectation that other languages would need to consume it and IDL was how C++ hackers (who still make up the entire set of people working on browser engines) are/were comfortable in describing their FFIs thanks to the legacy of COM/CORBA. Hilarious examples of multi-language-ism still persist in the WebIDL spec for no apparent reason whatsoever, warping the entire design around the altar of an ideal that is either quixotic or vestigial depending on which argument you give more weight.

Since the debate was re-kindled thanks to a debate at a TC39 meeting in July, I’ve been on the receiving end of more than one webdev’s rant about DOM’s JS incoherence, generally taking the form:

Why the *#!*?^@$ isn’t DOM just #@!*@ing specified in JavaScript?

To be honest, I have no answer aside from pointing to the IDL history, the fact that browser hackers don’t tend to write JS so don’t feel the pain, and noting that WebIDL is better in some important ways. Certainly these interfaces could be specified in a subset of JS with annotations for where native behavior is required. But their larger lament has merit too: seamlessness with JS is the bar WebIDL should be judged by. I.e. does it help spec authors do the right thing by JS devs? Or does it lead them down paths that make their generated APIs stick out like sore thumbs, full of anti-social/alien behavior such that you can’t think of them as “regular” JS?

Yes, constructors are only one minor step toward reaching this aspiration, but the fact that WebIDL has gotten to last-call without a reasonable solution to them speak volumes. If WebIDL isn’t animated by the need of JS developers, it would be good if that could be loudly called out somewhere so that the community can have the spirited debate that this point demands. If it is, can we please get on discussing how best to ensure that most “interfaces” generate constructors and stop punting?

Either way, WebIDL isn’t done yet.

Update: It occurred to me, as part of the discussion in the comments, that the provision against new with a class or type of any type is completely non-sensical in JS, as is the lack of call() and apply() methods on them. Idiomatic subclassing requires that the mixin-style be available, which uses ClassName.call(this). This is what you’d do with things that are “virtual” or “partial” interfaces if you’re describing them in actual JS. And there’s no real problem with folks new-ing them up. Doesn’t happen, doesn’t matter. Strong restrictions against it are, to quote Andrew DuPont, anti-social. Long story short: there is absolutely no reason whatsoever to disable construction on any WebIDL interface. It’s deeply non-sensical from the JavaScript perspective.

18 Comments

  1. Posted October 3, 2011 at 7:28 am | Permalink

    As several people explained to you earlier, HTML would not just have to add [Constructor], but would also need to define which Document object the newly constructed node is associated with (its ownerDocument).

    Flipping some switch in IDL will do nothing to make that work.

  2. Posted October 3, 2011 at 7:30 am | Permalink

    Anne:

    And as the debate progressed, it was clearly explained that new Image() works just fine. The ambient document of the global in question is a workable default. Need something from a different document? Grab it’s scripting context or use the factories.

    Can we get past debunked arguments already? We have this working in a patch for WebKit without issues. It’s not a problem.

    Regards

  3. Posted October 3, 2011 at 7:36 am | Permalink

    Yes, new Image() works precisely because HTML defines those details.

  4. Posted October 3, 2011 at 7:41 am | Permalink

    Anne:

    There’s a meta point here, which is why do these memory ownership details need anything but defaults specified for the generated constructors? Maybe we’re just arguing a small point which seems large, but saying “you need to define it” seems to be to be the follow-on from the argument that we should FORCE the issue by causing things to have constructors by default.

    Note that I’m not saying that flipping the default is all that needs to happen, only that it’s a necessary first step. Perhaps you don’t agree? If not, I haven’t seen you propose a different mechanism by which to start getting these mis-designs fixed in a systemic way. WebIDL has leverage to force the issue.

    Regards

  5. Posted October 3, 2011 at 7:42 am | Permalink

    Note that I am not opposed to having this feature, I am just saying that flipping a switch in IDL will not make it work.

    You would also get lots of undefined behavior. new Element(). What is its local name? new Node() or new CharacterData(). What are their types? How do they interact with appendChild()? Etc.

  6. Posted October 3, 2011 at 7:44 am | Permalink

    new Element() can have a writable tagName property or allow it to be passed in. That behavior needs specification, but nothing is forcing the issue today. I think you’re making my point for me = )

  7. Posted October 3, 2011 at 7:44 am | Permalink

    I do not think you need to force the issue. I think most people agree on what we want here (e.g. earlier this year I introduced new Event() and company and got them implemented by WebKit and Opera more recently). As I suggested earlier you should file bugs on the specifications that do not have constructors where you think it would be sensible to have them.

  8. Posted October 3, 2011 at 7:49 am | Permalink

    The problem is that what happens with new Element() and e.g. appendChild() if you do not set its local name. And even if you could give a local name its namespace would likely not be what you want and its interface would likely not be what you want either.

    Adding meaningful constructors for existing interfaces is a non-trivial exercise that should not be forced, but rather carefully thought out.

  9. Posted October 3, 2011 at 7:50 am | Permalink

    I think we disagree on the larger point: it’s nearly always sane to have a constructor. Saying we should go fight this on a spec-by-spec basis seems incongruous. Sure, we could, but why? WebIDL specifies what should happen in lots of other cases for JS APIs by default (conversions, etc.) which have over-rides.

    We don’t go fight those battles door-to-door, why should we do so here?

    In any case, it’s not just “meaningful” constructors. DOM is hostile to JS because even default ctors are missing. Yes, we can do better by taking arguments, but the default is nearly always meaningful, so just flipping the bit does have tons of value.

  10. Posted October 3, 2011 at 8:02 am | Permalink

    Namespace? It’s HTML. appendChild() can reject what it needs to in any case. And if Element really needs to be a pure virtual, it can take the new [NoConstructor] or whatever. It really is a special case (as is Node)…which means it should be special-case’d ;-)

  11. Posted October 3, 2011 at 8:35 am | Permalink

    I think certainly think of quite a few cases where it does not make sense to have a constructor (XMLHttpRequestUpload, XMLHttpRequestEventTarget, Element, Node, CharacterData, EventTarget, EventListener, Window, WindowProxy, Navigator), but my point is that a specification needs to be precise. You cannot just say it will “do the logical thing”. Those details need to be defined.

    That the node document of new HTMLDivElement() is the Document object associated with the global object on which the constructor was invoked is such a detail. new XMLHttpRequest() has something similar. new Event() needs to set the initialized flag.

    Before these details are defined they need to be worked out. If you flip a switch in IDL that will not result in them being worked out. [NoConstructor] will just be added all over the place, until it is figured out what should happen.

  12. Posted October 3, 2011 at 5:12 pm | Permalink

    You guys should stop pretending that you represent real developers. That stopped being true some time ago. Don’t get me wrong, WebIDL is a great idea and I’m glad that it has been specified. But the pace that this spec (and countless others) has been developed, leaves us normal folk gasping for breath. I’m sure that everything is open and above-board but unless you are directly involved it is impossible to keep up.

  13. Posted October 4, 2011 at 3:26 am | Permalink

    Hey Dean,

    I’m not sure anyone is pretending anything. I’m saying that it’s pretty tough medicine to swallow that WebIDL is keeping some APIs less idiomatic than they could be. It’s the sort of thing I hope webdevs will appreciate now but I’m not kidding myself. This is pretty inside-baseball.

    Anyway, I’m in London. Let me know if you want to get a pint and talk it over.

    Regards

  14. Posted October 4, 2011 at 8:29 am | Permalink

    I think that IDL’s purpose is noble, and it should continue to keep that. Having Dash come along shows now, more than ever, we should experiment, and keeping in IDL would do that.

    HOWEVER, working every day with the DOM and the fact that it leaves so much to interpretation, there should be another working group, which specifies how JSDOM works. Because as it goes right now, JS is THE language of the web, however keeping it open for more languages to interact with DOM elements is a good cause.

    I think keeping IDL is great, and allowing for a committee to say, “Hey, this is how JS should work with HTML” is worth it. That belongs neither in the HTML spec nor in the ECMAScript spec, and putting it in a middle one would be the best solution.

  15. Posted October 4, 2011 at 8:30 am | Permalink

    Alex, sorry, I guess my comment is a bit whiny. :)

    I’d love to meet for a pint. My phone number is still the same.

  16. Posted October 5, 2011 at 8:52 am | Permalink

    I’m back in London on the 14th. Perhaps we can catch up then?

  17. Posted October 5, 2011 at 11:29 am | Permalink

    Sounds good. I’ll take you out for some real London ale.

  18. haraken
    Posted October 10, 2011 at 9:23 pm | Permalink

    Hi, Alex and Anne

    Recently I have been implementing constructors for DOM objects (of course after defining IDL specs:-). Sooner or later I am planning to implement [NamedConstructor] IDL, and then propose specs and implement constructors for HTMLElement, although I have not yet considered the specs in detail. Making HTMLElement constructable is essential to the Component Model (http://wiki.whatwg.org/wiki/Component_Model).