Infrequently Noted

Alex Russell on browsers, standards, and the process of progress.

Zend + Dojo: More Than The Sum Of The Parts

In the past several months, more and more integrations of Dojo with server frameworks have been shipping, and we couldn't be happier about the Zend + Dojo integration that was announced yesterday.

Fundamentally the Dojo and Zend teams really "get" each other. Both are deep packages that give you an opinion about how best to do something but also all of the tools you'll need to make it work at scale. The "use at will" term that the Zend folks use made immediate sense to us. Like Dojo, Zend doesn't saddle you with more than you're really going to need up-front, but at the same time, when you need something awesome which is well tested and integrated, it's right there. No digging around on google to find a "plug in" that will get you where you want to go...both Zend and Dojo give you a full stack of great components to work with out of the box.

There have been some questions on IRC today about why Dojo and not something else, and we know that the Zend folks are committed to continuing to allow Zend to work with other frameworks as well and we fully support them in that. There seem to be lots of choices of Ajax frameworks which Zend could have integrated, but when you look at the requirements of serious products which need to ship tested solutions to really hard problems, the field whittles down very fast. In response to the needs of our users, both Zend and Dojo take seriously our responsibilities to provide integrated, high-quality, unambiguously licensed products that will let user scale both up and down. Key to this is understanding the full spectrum of use-cases, informed by past experience, and striking a balance between enterprise-ready features and close-to-the-metal primitives. The Zend Framework has a larger view of the server-side than Dojo can, and as a result there are new opportunities to optimize and improve the user experience for all classes of users through this integration. This isn't just about including some scripts on a page, this integration is about improving user and developer experiences, and both Dojo and Zend bring a lot to the table which can compliment the other. Dojo's strengths in progressive enhancement, packaging, localization, and accessibility all have obvious allegories in the ZF world where a complete integration can more value based on what developers are already doing. All of those features of Dojo will work better when the server-side knows how to "hint" things for the client and work with, not against, the client to deliver better experiences.

I'm perhaps most excited about the data-driven opportunities in the Zend/Dojo integration. Dojo's data infrastructure is second-to-none, and the design of the dojo.data and dojo.rpc layers provide Zend Framework integration the ability to take advantage of systems like the incredible Dojo Grid and the client-side charting package. There's more to look forward to, and figuring it out together with the folks at Zend is a great opportunity for the Dojo community.

Pete Higgins and I will be participating in next Tuesday's discussion with a broader chunk of the Zend Framework world, but until then (and long after) we'll be hanging out in the #dojo and #zftalk channels on irc.freenode.net. We're looking forward to working more with the ZF community to build great experiences, and are hugely excited about the direction things are already taking!

Blatantly Mis-Quoting Sublime

100 points to Freedom.

Dojo, DWR, CometD, OpenRecord, and the newly accepted Psych Desktop project are all 100 for 100 on Dion's scale because we've done the hard work of building a Foundation, ensuring the IP provenance of every checkin, built a community structure that gives everyone who's significantly invested a real voice, and have made the sacrifices that ensure that Dojo Foundation projects aren't just "open", but that they're trustworthy.

Dion's right:

The license is a great starting point, but it isn’t enough.

The Dojo Foundation has open door for any web projects that want to join and will do what it takes to get their score all the way up to 100. If the projects you're using aren't 100 for 100 — particularly given that we make it so easy — isn't it worth asking "why?"

App Engine: Most Of The Stuff I Want, None Of The Stuff I Don't

I'm not sure who or how, but I got an invite to yesterday's Camp Fire One event at Google where they announced their new App Engine platform. The event itself was small-ish, with lots of interesting people invited (both from Google and not). I had no idea ahead of time what the announcement would be, and frankly I forgot all about the event until the day of. I'm glad I remembered (note to future self: next time, pack gloves and a hat).

As they started talking about the platform and what's part of it (and what's not), I couldn't escape escape the feeling that they'd gotten it right. It is absolutely the case that for most apps, scaling requires some amount of re-architecting. Systems like Rails are built with such a dependence (intentional or not) on the features of relational data stores that they quickly hit bottlenecks because frameworks aren't keeping developers out of the gutter. This is nearly the same lesson that the security community collectively came to when it started to beat the average developer about the head regarding the awesome power of defaults. What systems do and don't do for you "cheaply" defines their character, and in many systems those choices aren't made consciously, or if they are, they don't have the benefit of a different perspective which might de-emphasize certain traits. Call it libertarian paternalism, choice architecture, or pure condescension, but whatever it is, systems and platforms today are in the explicit business of making some things easier than others.

As the presenters showed how to make an app quickly, I knew I was looking at something I hadn't seen the likes of since Jot. We all know that Big Table is a column-oriented data-store, but since most people are still stuck on the likes of MySQL, it's hard to appreciate how liberating it is not having to think about how adding properties to models will affect a schema. The way App Engine is constructed lets the data store layer provide something called expando models. These models give us the kind of flexibility that I've only ever enjoyed before inside of Jot. Want a property? Just add it. No migration, no schema version...just data finding a happy home, and as your app's skeleton "solidifies" and you figure out which properties really do need to be faster, you can migrate that data into fixed properties with indexes and types and all that jazz. It's like a gradual type system, only for data stores. It's agility nirvana for application development.

Speaking of application development nirvana, they also had the good sense to start with a great language (Python) and a great webapp framework (Django) as the basis for the new system. For all the Rubyists and Java heads out there who are surely crying bloody murder, I suggest that they just try it. Seriously. Django's template system is freaking sweet, and Python (despite it's lambda-related warts) is close enough to being executable pseudo-code as to still hold the second place in my toolbox of languages.

There's a lot which I'm sure others will (and have) covered about how App Engine is going to change the game for startups and players like Amazon, but I think that if someone else had launched this system it would still survive on its merits alone. The only wrinkle in the whole thing will be seeing what's done about pricing over the long haul. It really shouldn't be hard for Google to beat S3 on price, and I'm sure there will still be a market for EC2 for non-traditional tasks, but fundamentally I think App Engine has all the makings of something really, truly better than the current (assumed) stack.

After more than a year of mourning the effective loss of Jot as a platform, writing apps on the server just got interesting for me again, and that may be the highest praise I can offer and framework or platform.

Dojo 1.x: Industrial Strength Open Web Tech

...just ask AOL. James Burke just pinged me to let me know that AOL Webmail has been updated with a slew of new features built using Dojo 1.0. A quick inspection of the app shows all kinds of great stuff including tons of custom widgets as well as extensive use of the Grid. To keep loading of the app quick, AOL is using custom builds to pull Dojo from a CDN and a number of application-specific layers in order to defer loading of code until it's needed. Congrats to the AOL Mail team! It's inspiring to see a site that does billions of page-views a month using Dojo so effectively.

Dojo 1.x is now powering the UI of the world's largest mapping service, one of the world's largest email services, the most useful personal information service anywhere, and the front-end of my favorite RSS reader.

The proof of Dojo 1.x's quality really is in the pudding. Congrats again to the AOL Webmail team!

Update: Travis Vachon also just sent mail to let me know that OSAF's Cosmo project has also released their 0.14.0 version which has also made the upgrade to Dojo 1.0. Congrats, Cosmologists!

Dojo 1.1: Some Awesome For You App

I could go on for a long, long time about what's great in Dojo 1.1...but I'll spare you most of that. James, Pete, Dylan, and the release notes can give you a strong sense of why Dojo 1.1 is the most polished, fastest, and easiest-to-use release of Dojo we've ever done. For the impatient, you can already start using it from the CDN without downloading anything.

I should mention a couple of Core features from 1.1 that might otherwise go overlooked, though. The first is a lack of visible change. Dojo Core and Dijit from 1.1 are fully backwards compatible with 1.0. We promised that the fundamental incompatibility between 0.4.x and 0.9.x+ was a one-time change, and Dojo 1.1 keeps that promise. We've already had reports during the RC cycle of people swapping out Dojo 1.1 for 1.0 without any changes to their apps. It takes dedication and discipline across the entire team to achieve that kind of API stability while still taking risks to deliver better features, reliability, and performance.

Next up, we updated the animation APIs significantly. In Dojo 1.0, we moved to a unified timing loop for animations which helped to significantly improve the performance of Dojo animations. In 1.1, we've again improved the performance of the animation system but have also added some great syntactic sugar to the already powerful APIs which we expose. As always, start and end coordinates for an animation can be values or functions which return calculated starting and ending positions. Now, though, you can elide away the { end: 30 } structure and just provide a value. This lets us go from:

dojo.animateProperty({
    node: "thinger",
    duration: 500,
    properties: {
        width: { end: 500 },
        height: { end: 500 }
    }
}).play();

To just:

dojo.animateProperty({
    node: "thinger",
    duration: 500,
    properties: { width: 500 , height: 500 }
}).play();

But to get even more terse, we'll need a different API structure that doesn't require so much explicit argument packing. Dojo 1.1 adds just such an API:

dojo.anim("thinger", { width: 500, height: 500 }, 500);

Note that in addition to being able to drop a lot of stuff out of the function call, dojo.anim also doesn't require that we call .play() on its returned animation object since in the common case you'll just want to play right away.

Next up, Dojo now has a unified dojo.xhr() function that covers a lot of bases and gives you a single entry point into the various bits of XHR goodness contained in dojo.js.

As I've noted elsewhere, Dojo is also now supporting querySelectorAll on the browsers that support it sanely. Dojo's CSS engine has always been fast, and by keeping our query syntax to just what CSS provides, we've avoided getting ourselves into a situation where we'll always need to be shipping such a query engine down the wire. Sooner or later, dojo.query() will become nothing more than a call into querySelectorAll plus some syntactic sugar on the returned array. Best yet, API won't change and you can get the speedup on the browsers that support it now, knowing full-well that things will only get faster and smaller in the future. An investment in a toolkit that is pays attention to the evolution of the web is already paying dividends for Dojo users.

Lastly (for now), there have been some fun additions to the API of dojo.NodeList, the thing that's returned out of every call to dojo.query(). dojo.attr() and dojo.anim() are now available on groups of nodes. For instance, we can make a group of elements tab-focusable on browsers that support it and then draw some attention to them:

dojo.require("dojo.NodeList-fx");

dojo.query("#nav > .focusable"). attr("tabIndex", 0). style("border", "1px solid transparent"). anim({ "borderColor": { start: "yellow", end: "white" } });

There's also a new empty() method on node lists which makes removing children simpler, and the instantiate() method which helps you create instances of a class for every item in the list. Lets say you want to encapsulate some behavior in a class. You could use the hotness that is dojo.behavior or you could use the traditional combination of the Dijit base classes plus the Core parser system. Or you could strike out on your own, particularly for implementing something like a microformat handling library:

// define a Card class which we can create instances of
dojo.declare("mf.Card", null, {
    constructor: function(props, node){
        this.setAddrs(dojo.query("> .adr", node));
        this.setName(dojo.query(".fn, .n", node));
        // ...
    },
    name: null,
    addrs: [],
    tels: [],
    emails: [],
    // ...
});

dojo.query(".vcard").instantiate(mf.Card);

Taken together, these extensions make dojo.query significantly more powerful. There's also a growing set of extensions to node lists, such as those for easy templating of generated content via dojox.dtl. We've also taken care to ensure that anywhere that Dojo returns you an array of nodes, it'll probably be handing you a dojo.NodeList. That means you can treat these arrays as regular, intrinsic Array instances, or you can use these chainable methods for added node-manipulating power.

An Aside

The team that put together Dojo 1.1 is really, truly, amazing and it shows in the end product.

When it became clear that we had a responsibility to the a11y of Dojo application, they built it and shipped solutions. While others still aren't thinking about localization, they built it and shipped solutions to right-to-left as well as string handling (including optimizing those processes for deployment). When it became clear that the CSS file structure for building themes was too hard, they fixed it...in a 100% backwards compatible way. When Dojo was "dinged" for poor documentation, they did what they knew was the right thing and kept working on what is now the world's best JavaScript documentation tool.

What's most impressive to me about the Dojo team, though, is that this isn't a centrally planned, in-house operation by a giant company. There isn't a product road-map other than what's decided at the Developer Day events and in the weekly IRC meetings. Dojo is the collective work of volunteers, committers, and companies that understand that building things together in a truly open project is better for everyone. Having the source of a product be available under an OSS license gives you certain rights in an instantaneous sense, but for those advantages are usually just advantages for people already "inside" of a project. To really convert on the benefits of openness, the process of developing the product needs to be just as open as the license. Open Source business models all tacitly acknowledge that most pithy of truisms: the people on a project are its greatest asset. If new people can't join or "get in", then the project is effectively closed, and the benefits of openness can't be fully realized. Users win when advocates for their needs can emerge from more places, developers win when they can get involved and make a real difference, and a project benefits its community is really that: a group of people who show up because they want to make things better. I'm proud to be working with just such a dedicated group on Dojo.

During the 1.1 release cycle, Wolfram Kriesing and Nikolai Onken earned their committer stripes. They didn't start making Dojo better because they happened to work somewhere or because their company sells Dojo...they joined the project the way most of us did: they saw a place where they could contribute and because Dojo is run as an open project, they were able to have a huge impact. Nikolai's artwork and theme wrangling are largely responsible for Dojo 1.1 being the most attractive Dojo ever and Wolfram's code contributions have made interfacing services with dojo.data stores easier than ever before.

We've got an ambitious set of improvements slated for 1.2, and I'm confident that while it always looks slightly chaotic, the advantages of working in the open with whoever shows up will continue to pay off for everyone. If you've been a Dojo user and have thought about contributing, there's never been a better time to come find us on IRC or the forums and help to shape the future of the toolkit. You can be part of this amazing team too. It's as simple as that, really.

I can't wait to work with you to make our corner of the Open Web better.

Older Posts

Newer Posts