Come for the whole day, drop by for a bit, or just join us for dinner/drinks afterward. In the ramp up to 1.4, there’s some great engineering happening in nearly every area of the toolkit, including some great new visual improvements that I expect to see and hear a lot about tomorrow.

As usual, folks will be on IRC throughout the day should you not be able to join us in person, and in a first, we’ll have a live feed of the event going.

Hope you can join us!

What does this buy you? You get to use the Zend helpers for Dojo as you normally would, simplifying how you pull in code, declare components, and build your UI. What this new integration saves you is the tedium of figuring out which components you’re using everywhere, building a layer file for it, kicking off a build, and remembering to re-visit the layer definition when you project adds or removes modules. Hopefully ZF 1.9 should lower the barrier to taking advantage of the full range of Dojo-based optimizations, making it easier to prototype quickly and deploy easily. Exciting stuff!

Thanks, Will!

I won’t address the background point regarding software patents here. I’ll only to say that reasonable people can disagree on this, particularly when it comes to proposed solutions. What I would like to focus some attention on is the background that this patent filing is made against.

IBM has executed a CCLA with the Dojo Foundation. This agreement gives Dojo (and the rest of the OSS community) a license to whatever patent rights may be embodied in contributions of code. While IBM may file patents on things they build and contribute to Dojo, there’s no risk to any users regarding use of that code or “submarine” issues of patent infringement. As a result of the Dojo Foundation’s insistence that ALL code come with CLAs, Dojo is more trustable in terms of IP than most of the JavaScript you can choose to use. A similar patent claim in a less rigorously developed toolkit would indeed be apocalyptic, but the Dojo community has adopted a mature process for dealing with IP that both makes the concerns plain and then works to eliminate them, step-by-step. That’s what licensing agreements are, after all: links in the chain that together help you trust that your anchor is indeed set.

It’s clear to me that IBM filed this patent fully aware that they were giving away all follow-on rights to enforce it in anything but a defensive way for the benefit of the Foundation and users of Dojo. After watching IBM counsel decimate SCO in court, does anyone in the OSS world really think that IBM’s lawyers are fools? And if so, to what end?

It’s sad that Slashdot hasn’t, for a decade of coverage of IP issues, learned that licensing is harder than the zealots would have you believe and that malice isn’t always the intent of those who participate in communities with a commercial interest.

The good news here, of course, is that IBM is just as generous today toward the OSS and Dojo communities as they were yesterday. We have the legal documents to prove it.

I’m not entirely sold that the right solution going forward is to use the toolkits we already have, but if you’re building a mobile app today and you’re going to use a toolkit, it looks like (as on desktop browsers), Dojo should be your first choice. The webkitMobile variant of Dojo points to one possible way out of the “what to do about mobile?” conundrum regarding size and IE-induced bloat, but I have some hope that WebKit will improve quickly enough to make the toolkits of today obsolete on mobile phones entirely. Time will tell.

Hat tip: the Uxebu blog

If you haven’t tried out ZF, it really does make building sophisticated Dojo-based UI’s really simple. A bit part of that is that it helps automate a lot of the stuff that may be confusing when you’re first starting out with Dojo.

If you’re already using Dojo, this should be a no-brainer upgrade. It’s out-and-out better. As a quick example, dojo.create("tagname", { /*properties*/ }) is now the preferred way to build DOM nodes quickly. Its simple API will be natural to anyone who has used dojo.attr(). Even better, Pete’s exciting PlugD version of dojo.js has been updated to 1.3 as well.

1.3′s Core features the new “Acme” CSS selector engine which provides a big boost in speed for many operations in the fast-path. I blogged before about the work we did to make Acme fast, and rest assured it is (in aggregate, across all use cases) quicker than any other selector system you can get your hands on today. But selector performance isn’t where it’s really at, and I’ve been saying that for a long time.

Luckily, Pete Higgins decided to prove it and has been working on a new set of benchmarks with the help of other toolkit vendors (to ensure fairness) called “TaskSpeed“. Dojo 1.3 wins by a wide margin. Across all the reported browsers so far, Dojo is at least 2 times faster than other toolkits on common DOM operations. We’ve worked very hard over the years to make sure that Dojo’s APIs don’t encourage you to do things that will hurt you later, and TaskSpeed finally shows how much this philosophy pays off:

Given that DOM is the primary bottleneck in most apps DOM is a big bottleneck in today’s apps, usually just behind network I/O and these tests demonstrate how Dojo’s approach to keeping things fast pays off not just on micro benchmarks like CSS selector speed, performance improvements to single toolkit functions, or even file size – but on aggregate performance where it really matters. Dojo’s modern, compact syntax for these common operations doesn’t slow it down, either. For instance, if you go check out the TaskSpeed reporting page, you’ll see that where browsers are slowest (IE6/7/8, etc.), Dojo’s focus on performance pays off most. Why use a toolkit that’s going to hurt you when it really counts, particularly when Dojo so easy to get started with? Dojo’s Core has been designed from the ground up with APIs that encourage you to do things that are fast and keep you from doing things that are slow unless you really know what you’re doing. In some cases, we’ve made hard size-on-the-wire tradeoffs in order to keep actual app performance speedy. That hard engineering doesn’t show up in micro-benchmarks or single test release-over-release improvements or the “my toolkit is smaller” comparisons that some would prefer that web developers focus on. It’s easy to win rigged games, after all. It’s only when you see APIs composed together in real-world ways, across browsers, that you can start to see the real impact of a toolkit’s design philosophy. Dojo is designed to help you make things that are awesome for users, and that means they need to be FAST.

Other toolkits have released performance numbers of late, and most of them have been either reported badly or run without much rigor, so it’s exciting to see everyone finally pitching in to build end-to-end tests that show how library design decisions interact with real-world realities of browsers. The TaskSpeed tests have been designed to be both even-handed and reliable (no times below timer resolution, etc.). The reporting page is also designed to make the results understandable and put them in context. A lot of care has been taken to keep this benchmark honest. JavaScript developers have suffered at the hand of chart junk for far too long.

I can’t do 1.3 justice in a single blog post, so I recommend that you check out these resources and then just dive in:

• Pete’s release announcement
• The Official 1.3 Release Notes
• The new Documentation (at Dojo Campus)
• The new 1.3 Core Cheat Sheet
• PlugD: An even easier way to get going with Dojo
• Run the TaskSpeed tests for yourself then see how the toolkits stack up (hint: everything you thought you knew about JS library performance was probably wrong.)
• Check out integrations for Dojo that you can probably drop right in to your workflow, no matter what stack you’re using: , Spring Web Flow (Java), Dojango (Django/Python), DRails (Ruby/RoR), Tatami (Java/GWT), Zend (PHP), or Dojomino (Domino Server).

Big thanks to the folks who tried out the betas and RC’s and helped make 1.3 solid.

So why should you be switching to Dojo or upgrading to 1.3? You can dig through the nearly 500 fixed bugs or the tentative release notes yourself, but broadly speaking, we’ve hit all of our major stated targets for 1.3: IE 8 compatibility, major performance improvements throughout the whole system, and enhancements that make the APIs you already know that much more powerful and useful. I don’t want to get into specifics since Pete’s release announcement will include much more detail, but I’ll outline some of the changes to just the CSS query engine since that’s the system I worked on most for this release (continued after the jump).

As you may know, Dojo’s CSS query engine has always been wicked fast. Indeed, our original design target for doing CSS queries was that we wouldn’t do an engine until we could be sure that it would be so fast that using it wouldn’t slow down applications. Release after release, the query engine in Dojo has continued to improve, consistently beating all the other major toolkits, particularly on queries that get a lot of heavy use by Dojo developers. The world is changing, though, and over the Christmas break, I spent some time upgrading dojo.query to take advantage of the changes in browser landscape. For 1.3, my goal was to remove the XPath branch from the system and re-enable QSA in a robust way. The motivators were:

1. Firefox 3.0′s days are numbered. Since the DOM branch has been used on WebKit-based browsers for some time (it has been shown to be faster than XPath over the HTML DOM in tests), and since querySelectorAll is becoming available on IE 8, FF 3.1 3.5, and is now usable on WebKit browsers, the only customer for the XPath branch was FIrefox 3.0. With the imminent release of Firefox 3.1 3.5, it stops making sense to cart around a second selector engine target for the query compiler. The win is that we get to keep the speed for selectors that can be run through QSA, and since we drop the XPath branch, the core is both smaller and we have more room in our “byte budget” to optimize the DOM branch further.
2. querySelectorAll is nearly usable. As has been noted elsewhere, querySelectorAll is good but fatally mis-designed for the rooted-query case. As a result of this and many, many, many QSA bugs on various browsers (yes you, IE and gaps in CSS3 selector support, determining when to run a query via QSA and when to defer to the DOM branch isn’t actually straightforward. Engines like Sizzle try to detect query failure and re-run on DOM, but that leads to serious performance deficiencies on browsers with half-hearted QSA implementations. It’s also not accurate since silent errors are, well, silent. Enabling QSA, then, requires both a lot of work to handle rooted queries as well as feature and bug detection code to work around query engine differences.
3. Portability. The new selector engine is affectionately named “Acme” (aka, “query.js”) to indicate how generic it is. The engine is 100% stand-alone and can be integrated into other toolkits without much work at all. This is made possible thanks to some build-system magic in Dojo which keeps us from duplicating any functionality and by careful and judicious use of Dojo Core features. query.js can be used stand-alone without any extra work.
4. Better infix operator handling could make things faster. Instead of using regular expressions to parse CSS selectors, Dojo’s query engine generates an AST-like structure that represents a query. This system has always allowed us to have great visibility into what parts to optimize and how, but with 1.3 the AST generator now changes the way that infix operators (“>”, “~”, and “+”) are tokenized, making it possible to avoid inefficient queries which are then mostly thrown away and to skip error prone look-ahead code. This class of changes lead to huge speed wins.

I won’t cite specific performance numbers here since, as I’ve noted in the past, query engines are a commodity. That’s part of the reason we called the new engine “Acme”. Fast enough is fast enough, and the bottlenecks are elsewhere in toolkits today. I’m proud that the new engine sets a high water mark for performance, and I encourage other query engine maintainers to look through query.js and adopt the approaches we’ve taken to achieve eye-popping aggregate performance across different browsers and types of queries. The code is clean and commented to the hilt with notes about overall design and specific implementation decisions. I’m also happy to answer questions that other engine authors have.

Acme is just one – and not nearly the biggest – reason that Dojo 1.3 is an outstanding release. I’ll post more about some of the things I’m most excited about when 1.3 final is announced, but until then, I again encourage you to start working with the beta. While you’re at it, you might also want to check out plugd, drails, Dojango, and Persevere. They’re making it easier and faster to build great apps with Dojo and might save you tons of time.

Pete, Bill, Dustin, Becky, Adam, Kris, James, Bryan, Sam, Nikolai, Wolfram, Neil, Dylan, Tom, Chris (and you too, Chris), Tobias, Shane, Cougar, Jared, Doug, Josh, Bob, Roberto, and the rest of the Dojo community have a lot to be proud of with 1.3. Nice work, everyone!

As far as I know, it was a total surprise to the Dojo community (myself included). I can’t wait to get started writing apps for this thing and see what device APIs Palm has surfaced.

Maybe.

These things matter only in regards to how well they represent end-user workloads and how trustworthy their findings are. The first is much harder than the second, and end-to-end benchmarking is pretty much the only way to get there. As a result, sites like Tom’s Hardware focus on application-level benchmarks while still publishing “low level” numbers. Venerable test suites like SPECint have even moved toward running “full stack” style benchmarks which may emphasize a particular workload but are broad enough to capture the wider system effects which matter in the real world.

Marketing departments also like small, easily digestible, whole numbers. Saying something like “200% Faster!” sure sounds a lot better than “on a particular test which is part of a larger suite of tests, our system ran in X time vs. Y time for the competitor”. Both may be true, but the second statement gives you some context. Preferably even that statement would occur above an actual table of numbers or graphs. Numbers without context are lies waiting to be repeated.

With all of this said, James Ward’s Census benchmark makes a valiant stab at a full-stack test of data loading and rendering performance for RIA technologies. Last month Jared dug further into the numbers and found the methodology wanting, but given some IP issues couldn’t patch the sources himself. Since I wasn’t encumbered in the same way I thought I might as well try my hand at it, but after hours of attempting to get the sources to build, I finally gave up and decided to re-write the tests. The result is Census 2.

There are several goals of this re-write:

• Fairness. Tests need to be run multiple times for them to be representative in any way. Likewise, systems not being directly tested need to be factored out as much as possible. C2 does this by reducing the number of dependencies and running tests (at least) 5 times and discarding outliers before reporting an average. I’ve also worked to make sure that the tests put the best foot forward for all of the tested technologies.
• Hackability. Benchmarks like Census serve first as a way for decision makers to understand options but second as a way for developers to know how they’re doing. Making it trivial to add tests helps both audiences.
• Portability. The test suite should run nearly everywhere with a minimum of setup and fuss. This ensures that the largest numbers of people can benefit from the fairness and hackability of the tests.

The results so far have been instructive. On smaller data sets HTML wins hands-down for time-to-render, even despite its disadvantage in over-the-wire size. For massive data sets, pagination saves even the most feature-packed of RIA Grids, allowing the Dojo Grid to best even XSLT and a more compact JSON syntax. Of similar interest is the delta between page cycle times on modern browsers vs their predecessors. Flex can have a relatively even performance curve over host browsers, but the difference between browsers today is simply stunning.

Given the lack of an out-of-the-box paginating data store for Flex, RIAs built on that stack seem beholden to either Adobe’s LCDS licensing or are left to build ad-hoc pagination into apps by hand to get reasonable performance for data-rich business applications. James Ward has already exchanged some mail with me on this topic and it’s my hope that we can show how to do pagination in Flex without needing LCDS in the near future.

The tests aren’t complete. There’s still work to do to get some of the SOAP and AMF tests working again. If you have ideas about how to get this done w/o introducing a gigantic harball of a Java toolchain, I’m all ears. Also on the TODO list is an AppEngine app for recording and analyzing test runs so that we can say something interesting about performance on various browsers.

Census 2 is very much an open source project and so if you’d like to get your library or technology tested, please don’t hesitate to send me mail or, better yet, attach patches to the Bug Tracker.

Update: I failed to mention earlier that one of the largest changes in C2 vs. Census is that we report full page cycle times. Instead of reporting just the “internal” timings of an RIA which has been fully boostrapped, the full page times report the full time from page loading to when the output is responsive to user action. This keeps JavaScript frameworks (or even Flex) from omitting from the reports the price that users pay to download their (often sizable) infrastructure. There’s more work to do in reporting overall sizes and times (“bandwidth” numbers don’t report gzipped sizes, e.g.), but if you want the skinny on real performance, scroll down to the red bars. That’s where the action is.

To my great surprise, I’ve been at SitePen two and a half years. It has been nothing short of wonderful which may explain why it doesn’t feel like it has been that long. When I look back at what we’ve accomplished it’s also surprising that we’ve been able to do all if it in such a short timeframe. Between the huge client projects and re-building Dojo from the ground up, it has been busy bordering on nutty.

It already makes me sad to leave behind working with the SitePen crew, many of whom I helped to hire in and who I count among my closest friends. But I won’t be entirely gone. I’ll still be contributing to Dojo in my new role, if less frequently. Not that it’ll slow the project down any. Pete, Bill, Adam, and Tom have Dojo well in hand and have been driving things forward at a furious rate. Dojo has always been a team effort, and I’m excited about the improvements coming in 1.3. I’ve gotten a dis-proportionate amount of the credit over the years (and not enough of the blame), and as Dojo evolves from here it will continue to be because companies like SitePen, Uxebu, AOL, and IBM have all been able to contribute to make it happen and that leaders like Pete Higgins have stepped up to lead and teach and learn with the community. My deepest thanks go to Dylan and SitePen for having let me be a part of that process on a daily basis for the last couple of years.

So what could possibly pry me away from such a sweet, sweet gig at SitePen?

In a word, Chrome.

Three years after many of my friends joined Google, the appeal of getting to fix the “web as platform” problem from the inside has finally proven irresistible. There’s much to do, and the WebKit platform seems like the best shot that we have (collectively) at forging a future that’s not just open, but also markedly better. At SitePen I’ve had the chance to make the web a better place through Dojo. At Google I’ll have a chance to do it from the browser itself.

To the friends I’m leaving, it was a privilege to work with you. To the friends I’m joining, thanks for your trust and faith.

The new site is important for a couple of reasons. First, it’s a good step in the direction of working to make clear that Dojo-the-javascript-toolkit and Dojo-the-foundation are completely separate entities with different goals and different leadership. The Foundation will take any and all projects that want a good home and are willing to meet the basic criteria that let others trust the code and process behind Foundation projects. It’s been weird that most of the documentation about Foundation policies and procedures has lived on the Toolkit site for so long, and having them be totally separate should help clarify.

Secondly, the Foundation site helps show off the projects which aren’t the toolkit. The Foundation has more than doubled in size in the last year (in terms of projects), and looks set to continue that expansion. That the Toolkit project gets the most attention has been a long-term irritant to nearly everyone involved, and it’s my hope that the Foundation site will help to fix that.

Lastly, the Foundation site is beautiful. Built with the Toolkit and Dojango, the site is a great showcase for the kinds of things that are easy to build with Foundation-sponsored technologies. Nice work everyone!