Dojo: Twice As Fast When It Matters Most

Some folks have noticed a new landing page for dojotoolkit.org, one that includes hard numbers about the performance of Dojo vs. jQuery. Every library makes tradeoffs for speed in order to provide better APIs, but JavaScript toolkit performance shootouts obscure that reality more often than not. After all, there would hardly be a need for toolkits if the built in APIs were livable. Our new site isn’t arguing that Dojo gives you the fastest possible way to do each of the tasks in the benchmark, all we argue is that we provide the fastest implementation that you’ll love using.


Smaller is better.

I gathered the numbers and stand behind them, so let me quickly outline where they come from, why they’re fair, and why they matter to your app.

I took the average of three separate runs of the TaskSpeed benchmark in comparing the latest versions of both Dojo and jQuery. The numbers were collected on isolated VM’s on a system doing little else. You may not be able to reproduce the exact numbers, but across a similar set of runs, the relative timings should be representative.

So why is TaskSpeed a fair measuring stick? First, it does representative tasks and the runtime harness is calibrated to ensure statistically significant results. Secondly, the versions of the code for each library are written by the library authors themselves. The Dojo team contributed the Dojo versions of the baseline tasks and the jQuery team contributed theirs. If any library wants to take issue with the tests or the results, they only need to send Pete a patch. Lastly, the tests run in relative isolation in iframes. This isn’t bulletproof — GC interactions can do strange things and I’ve argued for longer runs — but it’s pretty good as these things go. I took averages of multiple runs in part to hedge against these problems.

The comparison to jQuery is fair on the basis of syntax and market share. If you compare the syntax used for Dojo’s tests with the jQuery versions, you’ll see that they’re similarly terse and provide analogous conveniences for DOM manipulation, but the Dojo versions lose the brevity race in a few places. That’s the price of speed, and TaskSpeed makes those design decisions clear. As for market share, I’ll let John do the talking. It would be foolish of me to suggest that we should be comparing Dojo to some other library without simultaneously suggesting that his market share numbers are wrong; and I doubt they are.

Given all of that, do the TaskSpeed numbers actually matter for application performance? I argue that they do for two reasons. First, TaskSpeed is explicitly designed to capture common-case web development tasks. You might argue that the weightings should be different (a discussion I’d like to see happen more openly), but it’s much harder to argue that the tests do things that real applications don’t. Because the toolkit teams contributed the test implementations, they provide a view to how developers should approach a task using a particular library. It’s also reasonable to suspect that they demonstrate the fastest way in each library to accomplish each task. It’s a benchmark, after all. This dynamic makes plain the tradeoffs between speed and convenience in API design, leaving you to make informed decisions based on the costs and benefits of convenience. The APIs, after all, are the mast your application will be lashed to.

I encourage you to go run the numbers for yourself, investigate each library’s contributed tests to get a sense for the syntax that each encourages, and get involved in making the benchmarks and the libraries better. That’s the approach that the Dojo team has taken, and one that continues to pay off for Dojo’s users in the form of deliberately designed APIs and tremendous performance.

17 Comments

  1. Posted January 28, 2010 at 4:02 pm | Permalink

    That seems like a tricky game to play – especially putting those results up on the Dojo homepage. Will you keep them up even when Dojo’s results are slower? Because as of jQuery 1.4.2, jQuery is faster than Dojo:
    http://www.flickr.com/photos/jeresig/4311864455/

    I definitely disagree that Taskspeed is a good representation of what’s commonly tackled in web pages. It’s certainly more comprehensive than Slickspeed (just selectors) but that’s a pretty low rope to walk over.

    There’s a reason why the jQuery 1.4 release notes didn’t look at the performance of jQuery relative to other frameworks and only compared against itself: At this point in time improvements are coming so quickly across all frameworks that it’s futile to try and represent them in a single snapshot – and it would be incredibly foolhardy to assume that the current performance levels will always stay that way. A morning’s worth of hacking on my end more than doubled the “performance” reported by Taskspeed – but I wouldn’t go so far as to change the jQuery homepage to report that fact – since I know that the next release of Dojo (and the other frameworks) will work to try and improve those numbers again.

    Either way, I’m looking forward to seeing the updated Dojo homepage.

  2. Posted January 28, 2010 at 4:18 pm | Permalink

    So reporting performance comparisons helped spur some good changes to a widely used library, even if it’s not our own….you’ll forgive me if I don’t see that as a slippery slope. It’s competition and it’s good for users. I continue to be for it.

    As for updating the site, a major overhaul of the site has been in the works for some time and will land soon. This interim site and the numbers it reports are accurate as far as we know based on released versions.

    Regards

  3. Posted January 28, 2010 at 9:32 pm | Permalink

    @alex: I agree either ways the competition is going to bring out the best of the libs :)

    @John: I personally use jQuery, but in business need to force hard in competition.

    Having said that, @alex, Hope free-open-source doesn’t strangle its own siblings!

  4. Posted January 29, 2010 at 6:45 am | Permalink

    Rutuaj,

    There’s no strangling going on here. As you can see, the simple act of putting this out there has spurred another open source project to better itself. And no doubt Dojo will respond with a similar action.

    This is friendly competition, in which everyone wins. And it’s great to see.

    Thanks

    Shane

  5. Les
    Posted January 29, 2010 at 7:05 am | Permalink

    Alex, you work for Google. Is there a reason why Google Closure is not included in TaskSpeed?

  6. Jake McGraw
    Posted January 29, 2010 at 7:38 am | Permalink

    http://twitpic.com/10bcfa/full

    jQuery has reached a saturation point now where it is often the default choice for client side application development. The momentum behind it and the number of developers who enjoy using it far outweighs any marginal performance differences.

    John has spoken out against in-browser speed tests for almost two years now. The fact that he could knock out jQuery 1.4.2pre in less than 24 hours and invalidate your test results makes this post and the Dojo home page changes appear completely pointless and needlessly confrontational.

  7. Tim
    Posted January 29, 2010 at 10:10 am | Permalink

    Looks like QOOXDOO kicks all other frameworks butt for pure speed.

  8. Tim
    Posted January 29, 2010 at 10:12 am | Permalink

    @Alex, why are you using a waaaaaaay old version of QOOXDOO? In TaskSpeed, it only has QOOXDOO version 0.8.

    Version 1.0.1 is the current release.

    http://news.qooxdoo.org/qooxdoo-1-0-1-released

  9. Tim
    Posted January 29, 2010 at 11:38 am | Permalink

    @John

    When can we expect 1.4.2 of JQuery to be released.

    I always welcome performance improvements.

  10. Posted January 29, 2010 at 12:16 pm | Permalink

    Tim: you probably should ping Pete Higgins about that, or better yet, send him a patch.

  11. Posted January 29, 2010 at 12:17 pm | Permalink

    Les: ’cause nobody has sent Pete a patch?

  12. Posted January 30, 2010 at 10:50 am | Permalink

    My question is – matters most for what? it has been my experience for most tasks on modern computers it is very rare to experience a performance issue nowadays. Aside from obvious pitfalls such as extreme looping or working on many nodes simultaneously, there is no performance concern to speak of.

    In my opinion the choice of framework should depend on the API, extendability, support, community and documentation. Performance is not really a concern anymore.

  13. Tony Issakov
    Posted February 1, 2010 at 12:09 am | Permalink

    Dojo and jQuery are both amazing tools yet neither satisfies everything all the time. And why should they.. They’re targeting different markets if you get back to their roots.

    Watching the community in general though is like watching an argument about .Net vs Java. My apple is a much better apple compared to your orange…. uhh?

    I really felt like the JS community was growing up when I heard both Alex and John on the same podcast telling the world we can do more and do it better.

    I can understand why Alex wants the world to pay attention to dojo. If I hear one more JS user (or more recently rails developer) tell me “Dojo is slow and bloated…oh but I’ve never seen it or used it”, I’ll slap them. Such blanket statements with so little to back it up and so little understanding of the code they make claims about.

    I admire that John pushes for personal bests but I think he might also understand how I and possibly Alex feel when a community filled ignorant unfounded statements needs a graph to show them their claims may need some rethinking.

    I don’t care who is fastest at any one moment. Only that there is real competition, that everyone is striving for it and innovating along the way.

  14. Posted February 1, 2010 at 3:59 am | Permalink

    I’d like to second @Tony

    The whole my library is faster than yours is a bit anal, especially considering querySelectorAll is now widley implemented.

    jQuery won the popularity contest a long time ago; but it isn’t the best choice for every situation, or suited to every programming style preference so live and let live library fanboys

  15. Sarah G
    Posted February 14, 2010 at 4:11 pm | Permalink

    Still haven’t seen these JQuery performance improvements released yet.

  16. Les
    Posted February 17, 2010 at 5:33 pm | Permalink

    jQuery 1.4.2 is now available.

    http://ajax.microsoft.com/ajax/jQuery/jquery-1.4.2.min.js

    Is it faster than Dojo?

  17. Gonzalo
    Posted February 24, 2010 at 7:56 am | Permalink

    seems to, and now?