Infrequently Noted

Alex Russell on browsers, standards, and the process of progress.

File uploading with Dojo

Professional's need professional tools, which is one of the main reasons we started the Dojo project. Dojo's I/O system is a prime example. From built-in caching (and cache busting) to making requests bookmarkable, the abstraction makes doing Ajax easier than with any other client-side library. As I promised earlier I'm going to cover the memory leak prevention and file uploading support in

As with DOM event handlers, IE gets easily confused about what it can and cannot clean up when you make XMLHTTP requests. The naive way of listening for the "finished" state in an asynchronous request is to attach a function to the XMLHTTP object's onreadystatechange slot and determine inside the handler if the desired state has been reached. The problem though is the same as with DOM event handlers; since the XMLHTTP object in IE is an ActiveX control, IEs reference counting mechanism goes walkabout when presented with an event handler that is assigned from the JavaScript context. The contexts (closures) referenced by the event handler get leaked. Oops.

After much testing, we figured out a pattern that prevents this kind of leakage. Instead of attaching a method to be called back to, Dojo's IO system starts a timer to watch for state changes on in-flight network requests. When a change is detected, we dispatch the correct event handler from outside the XMLHTTP object's event system. Since we never set a closure reference between the ActiveX object and the JavaScript context, we never leak it. Of course, once there are no more operations "in the air", we cancel the timer that watches for state changes to prevent further resource usage.

But wait! there's more!

Dojo's IO system not only handles hand-rolled requests, it will also submit entire forms in the background for you. Just pass in a reference to the form's DOM Node in the formNode parameter and it gets automatically serialized into a request of the specified variety (either POST or GET). There's no reason that adding Ajax to legacy apps should be harder than doing it in new development.

The big caveat-emptor, however, has always been uploading files. If you pass a form node to the default XMLHTTP back end for, it will refuse to process the request since there's no portable way to get the contents of a file on disk from inside the browser environment. But forms are different. Browsers let users explicitly specify which file to upload in a form and submitting these forms carries the encoded form content along for the ride. It'd be really handy if we could treat these forms the same way in an Ajax application.

As alluded to earlier, supports "pluggable" back-ends. When a request is passed into bind, it asks each of the registered transport systems if they will accepted the request in turn. The first one to say "yes" gets passed the request to handle. In most cases, this is the XMLHTTP transport, but not for forms with file uploads. For these occasions, Dojo provides an "IframeIO" module. Since we can "target" a form to be submitted in whatever frame we please, the IframeIO class lets the browser process the request normally, but still in the background. Using it is as simple as adding the following line to your scripts:


Now, passing the formNode parameter will work for both upload and non-upload cases. The XMLHTTP transport will still get used for most requests but when presented with a file upload, your code will now handle it gracefully, albeit with that click sound in the background when the form is submitted. A small price to pay for being able to add file-upload capability to single-page applications!

Next time: uploading files...without the file?

Why I expect so little of IE 7...and nothing beyond that

For the better part of a year, I've been saying to anyone that will listen that IE 7 is either a ruse or a bad joke. Getting into the details of why takes a lot of doing, and so I usually skip it. Dare Obasanjo has captured it perfectly.

Now where'd I put that reference?

IE's abysmal memory management is the bane of any professional JS developer's life. Without rehashing all of the patterns, you can tell your app is running afoul of IE's crappy reference counting when users report that it "feels slow" but that flushing the cache and/or restarting the browser "fixes" it. At this point you can either dig into the MSDN and Usenet articles on the topic and try to develop a workaround for your app, or you can use dojo.event.connect().

We didn't cover it in the event system article,but Dojo transparently prevents leakage when you use dojo.event.connect(). Dojo isn't the first system to do this, of course. The Signals and Slots system from netWindows baked in leakage prevention and others predate even it. As the problem has recently become better appreciated, other modern tools are starting to deal with leaks.

For those using Dojo, the solution to leakage is amazingly straightforward. In almost every case, you can just replace your node.onclick = function(){ ... }; statements with dojo.event.connect() equivalents and the leakage disappears. Under the covers, the node and the connected-to event handlers are placed in a "registry" for cleanup when the page starts to dissapear. A small map of property names is stored on the nodes so that the system never tries to strip more properties than necessary (which can be a serious performance problem).

The only exception to the transparency of this system is when DOM nodes are destroyed before the page unloaded. In these cases, DOM nodes may go out of scope and become "unreachable" by the event system for cleanup when the page unloads. To prevent leakage in these situations, Dojo provides dojo.event.browser.clean(nodeRef) to let you make sure that you're "OK". If you are going to remove nodes from a page, you can use this idiom to make sure that your app doesn't leak:

dojo.event.browser.clean( parent.removeChild( child ) );

Of course, this leak prevention is also tied into the guts of the Dojo widget system. Event handlers in widget templates are automatically registered for cleaning and when a widget is destroyed, the system takes care to clean up after you.

While many developers will inevitably roll their own and spend painful weeks trying to manually track down leakage, users of Dojo may never have to know that it was a problem. Which is how it should be.

Next time: how lets you upload files and beat memory leaks, all before lunch.

another reason to use Dojo: dojo.string.Builder

I work at a pretty magical place. Not only is working with Jot like programming for the "giant data store in the sky", I get to work on Dojo and do so with an amazing group of folks. Take, for example, the case of dojo.string.Builder.

The basic idea is that instead of doing naive "+=" concatenation of strings, the string builder class should more efficiently manage the creation of huge string variables. In most scripting languages, the way the creation of huge, immutable intermediate variables is by creating an array or list and adding each new fragment to it, eventually join()-ing it to produce the end string. Until this week, dojo.string.Builder did exactly this.

David smelled a rat, though. His experience didn't match the conventional wisdom. We quickly had a set of tests for comparing the speed of various string creation methods. On JScript (Internet Explorer and WSH), it seems that stringArray.join("") is fastest by a wide margin. Testing showed it to be 10x faster than the "+=" operator in most cases. Interestingly, on Spidermonkey (Mozilla and Firefox) and Rhino the "+=" operator beat array construction/join by 3x. As a result, dojo.string.Builder now picks the fastest variant available for the interpreter your code is running on.

Taken alone, it's a small thing, but incidents like this make me feel amazingly lucky to be working with the crew that's contributing to Dojo.

Semantically right, practially wrong

About a year ago, I tried to invite PPK to the Dojo project. He declined at the time, stating that he doesn't find libraries useful, which seemed a reasonable thing to say. If it doesn't fit his tastes, so be it.

Today, however, he went a step further (via ajaxian), stating that neither client side nor server side programmers are qualified to write the coming generation of responsive webapps. From a semantic standpoint, I think I might agree: if someone considers themselves just one or the other, they won't be able to do it. But I don't think that's his point, and on that point I think he's dead wrong. I think Mark Wubben and Bob Ippolito's comments on the ajaxian post are dead on.

Sure, JavaScript isn't a "fun" language unless you magically wind up using it in a non-browser environment (in which case you should probably be using Python anyway), but to do "real engineering" in a browser, you need a "real engineer" on the client side. Someone who gets HTTP, JS, DOM, whatever your server-side language is, and can shoot the shit about TCP/IP stack config paramaters on whatever your server-side OS happens to be.

Anyone who can credibly claim to have written a non-trivial amount of idiomatic JS (can you explain a closure? how does prototype inheritance differ from class-based inheritance?) probably fits this bill, and if not, they already show the ability to pick up difficult concepts and will be able to fill in where they aren't already experts. The question isn't "server side" or "client side", but do you come at it from an engineer's perspective, regardless of what side you're on?

As a library author, I find his remarks about them interesting but still naive. I know PPK doesn't like libraries, but they tend to be written first and foremost for the benefit of the people who wrote them: e.g., just the kinds of engineers he dismisses on both sides of the HTTP stack.

Frankly, good engineer's don't re-invent the wheel for kicks, and so a good engineer dropped from a server environment to a client-side environment will instinctively look for good tools. Implying that one has to suffer to get things done on the client is just plain wrong. Suffering doesn't make one a better client-side engineer. Engineering things does.

Older Posts

Newer Posts