Infrequently Noted

Alex Russell on browsers, standards, and the process of progress.

Kris Zyp Joins SitePen

Some time back I posted about an opening in SitePen's R&D "group". The responses I got back were astounding, and I couldn't be happier that Kris Zyp is joining me to help work on things that we feel are important to the future of the open web. This is yet another high point in our recent string of great hires. SitePen has never been a better place to work, and that's saying something.

The W3C Cannot Save Us

Things are finally moving over in CSS-land. On the positive side CSS column layouts are looking pretty nice, having dropped their dependency on the the janktastic "advanced" layout module and there's some initial movement on improving the CSS-OM.

But all is not well, nor has it been for a long, long time. No work on hbox or vbox or mixins/inheritance. There's also no indication that the WG has taken stabs exposing the expressive power that Microsoft exposed with CSS expressions.

More importantly, Andy Clarke is pretty disgusted by what he sees of the process and participants and so, apparently, is David Baron. Anyone who has met David in person will probably understand how much of a big deal this really is. It signals the effective end of the CSS WG as we (don't) know it. Rebuilding credibility in the WG is going to be much, much harder now that Mozilla's representative has effectively given up on the closed-door process. The working group's secret cabal style of operation is imploding. It was inevitable, but the timing is still a surprise.

But why was it inevitable? And should we take Andy's suggestion seriously and expect a re-chartered WG to do better? After thinking about it for a while, I think the answer is that we can't expect any standards body to do what is being asked of the CSS WG; namely to invent the future by committee. It's particularly unreasonable to expect progress when deployed browsers provide so much momentum that the safest thing to do is to solve the little problems and ignore the large ones. This is also a well-worn stalling tactic used by vendors playing for time as to marginalize the standards process or solidify their lead such that it becomes a de-facto standard. This is a good reason to reject closed working groups, but the web standards to which advocates now cling were developed in those same closed systems. Closed clearly can't be all bad nor can it be a primary cause for the lack of progress. We seem to be hugely confused and conflicted about the relationship between standards, vendors, and how we get to a better future.

Before I lay out all the forces involved, let me first say what everyone knows but few truly accept:

In order for the future to be better by a large amount, it must be different by a large amount.

I think that statement alone is enough to indict Opera's anti-trust actions as stupid and ill-considered. But we should also recognize that it forms the basis of Opera's grievances. We should all be pissed off that the discussion today hinges on how we will get MSIE to improve by slight degrees (or should we expect more?). Opera could have done better, though, by shipping Gears and working with Google to make it a host for pluggable renderers...like Opera. Too bad Opera has prioritized proving a point over actually improving the situation. But I digress.

So what was so different about the late 90's that it allowed a closed process to make huge gains in a short order while we can't even get basic architectural issues addressed in a timely fashion today?

The first major reason is that web developers in the 90's were looking forward, not backward. I remember being excited about getting the chance to use new features and not caring who gave them to us. As a community, web developers hadn't collectively "picked sides". I think the market as a whole still has that essential optimism built in, it's just that the more that self-identified web developers focus on how standards compliant things are (or aren't), the more we lose the sense that it can get better. So platforms not tied to standards race ahead. Why is anyone surprised that Adobe has essentially kicked HTML's ass with Flex or that Microsoft feels it can do the same with Silverlight in a couple of revs? If you buy either Adobe or MS's lines about how these platforms aren't there to replace HTML, I'd like to sell you some prime property in the Pacific.

But, I hear you yelling, it sucked to build things back then! It was hard! We didn't know what bits of the technology we could use portably, and the W3C saved us from that mess!

Oh really?

Try teaching a good programmer without a web background to build anything reasonably sophisticated with web technologies today. Doing so will teach you painfully, embarrassingly that there are huge tracts of the HTML, CSS, and DOM specs that you simply can't use. IE's bugs combined with its market share conspire to ensure that's true, and we wouldn't get off the hook should IE 8 magically transform into a perfect reference implementation. Mozilla, Opera, and Safari all have their own warts as we get to the edges of what's theoretically possible with current specs. And that's not even taking into account how utterly wrong, broken, and silent the specs are in several key areas. Even if the specs were great, we'd still be gated by the adoption of new renderers.

It's clear then that vendors in the market are the ones who deploy new technologies which improve the situation. The W3C has the authority to standardize things, but vendors have all the power when it comes to actually making those things available. Ten years ago, we counted on vendors introducing new and awesome things into the wild and then we yelled at them to go standardize. It mostly worked. Today, we yell at the standards bodies to introduce new and awesome things and yell at the browser vendors to implement them, regardless of how unrealistic they may be. It doesn't work. See the problem?

Andy is probably wrong to suggest that a new working group (no matter the structure) can succeed in fixing the impasses of the existing CSS WG if only because no working group at the W3C has the power to effectively and definitively introduce new things into the market. Only browser vendors can do that and no amount of buy-in at the W3C can force vendors to implement. The last decade is proof enough of that. We need to wise up to this key point: standards bodies are downstream of implementations, and that's the only time when they work well.

Until we get some great new (non-standard) CSS features out Mozilla, Opera, and IE nothing will get better to the extent that we will again be optimistic about the future (Safari earns a pass). The size of the improvements they deliver in the future are directly tied to our expectations of how different the future will be. Only when there are large and divergent ideas of how to proceed expressed through competing, successful implementations will standardization really work to whatever extent that it can reasonably be expected to.

Let that sink in a bit. To get a better future, not only do we need a return to "the browser wars", we need to applaud and use the hell out of "non-standard" features until such time as there's a standard to cover equivalent functionality. Non-standard features are the future, and suggesting that they are somehow "bad" is to work against your own self-interest.

Web developers everywhere need to start burning their standards advocacy literature and start telling their browser vendors to give them the new shiny. Do we want things to work the same everywhere? Of course, but we've got plenty of proof to suggest that only healthy browser competition is going to get us there. Restructuring the CSS WG or expecting IE8 to be "fully standards compliant" is a fools game.

Put simply, Zeldman is hurting you and only you can make it stop. Neither the CSS WG nor the HTML 5 WG nor, indeed, any W3C working group can define the future. They can only round off the sharp edges once the future becomes the past and that's all we should ever expect of them. As much as they tell us (and themselves) that they can, and as much as they really would like to, the W3C cannot save us.

Update: the implosion continues apace. Hixie outs Microsoft's rep as stalling on embeddable fonts and runs up the flag on why he thinks it's just not working in general. Interestingly, one of the MS reps has seemingly seconded dbarons "lets do this in public" sentiment as Hixie smacks down Bert Bos's leadership of the CSS WG. It's easy to get caught up in the horse race on this, but I want to again suggest that none of this will get better until the browsers start taking risks. Hopefully openness and transparency will allow the world to judge the actions of the WG until then without removing the ability for the WG to function when the cavalry does arrive.

The Non-Relational DB Strikes Back!

When I started at Jot, one of the things I fell most in love with about the platform was the way that application developers on the system never, ever had to think about "the database". You just had nodes (JS objects which could serialize themselves to XML) and nodes had properties. Setting a property on the node persisted it and created a new version of the node.

Instead of thinking about how you were going to represent your problem in objects and then figuring out some way to map them to an RDBMS, you just started figuring out your problem in code, letting the normal development cycle iterations of what a "thing" is continue without stopping after every change to babysit the database or perform some sort of lame (or brittle) "migration". Recently I've been excited to see this kind of work start to evolve in the Django world while aspects of the non-relational data store have been finding more mindshare through projects like CouchDB and ERlang's built-in Mnesia persistence layer, although they all still feel relatively primitive in comparison to the "experimentation is free" environment that Jot offered. Sometimes folks ask my why I don't get into RoR, and every time I look into it again I'm alway struck how....backward it is. Hopefully the rumored gemstone port to ruby will plug up some of the remaining conceptual leaks that the RDBMS addiction has tortured the RoR and DJango app development process with.

Adding to the non-RDBMS data storage action is the announcement by Amazon of their SimpleDB service. It shares some of the best features of the Jot model (easy key/value setting, no schema, query anything) but doesn't yet seem to have the ability to version individual records. Even if SimpleDB doesn't do it, I expect it to pop up in another form somewhere else soon.

I'm tremendously excited about these sorts of services and data stores. It's been clear for some time that most data storage tasks for even departmental applications are main-memory tasks. It'll be interesting to see how the language environments respond to these changes. Microsoft's LINQ integration into .NET languages is the first major stab in this direction, and I expect the next up-and-coming language will probably develop something similar in order to one-up Java and Ruby by making "schema evolution" look more like adding properties to an object or a class prototype (in JS parlance).

Hopefully soon all of this work will soon yield a web framework for general consumption that will show the rest of the world what Jot got dead right: that when your data and your program can evolve in harmony and without friction or risk, you are truly liberated. When storage is free (and it nearly is), "screwing up" should mean starting a fire in your data center. Everything else is just a version rollback.

If You're Not Already Subscribed To Mike Shaver's Blog...

...this would be a good time to go add it to your feed reader of choice. His latest post on Adobe's attempts to increase the social acceptability of their closed platform does a great job at distilling some of the history and strategies being employed.

DWR Joins The Dojo Foundation, Joe Walker Joins SitePen

Dylan has the short-and-sweet writeup of what's happening with DWR and the Dojo Foundation and Joe Walker has a bit more Q&A. I can't really add much to the "news" bit of the news other than to say that I'm tremendously excited about it. The DWR community has been amazingly level-headed in its deliberations, and I can't wait to work with them in the future.

I've been concerned a bit that it may appear as though Joe joining SitePen may carry with it the perception that the Dojo Foundation is an arm of SitePen in some way or that DWR will now need to become Dojo-centric. Luckily neither is the case, although reading assurances to that effect on this blog should be taken with a grain of salt. The Foundation has an open door for deserving projects which need a good legal umbrella and don't want a lot of process or formality, and we've extended personal invitations to many non-Dojo-centric projects over the years to join (including direct competitors).

For anyone still worried about the DWR/DF arrangement, working it backwards from both perspectives should yield some comfort. It does very little good for Dojo to be shoved down someone's throat by the choice of some orthogonal tool in the same way that it would be tremendously foolhardy for the Foundation to lose its independence in any way. Neither would be very meritocratic and in particular the independence of the Foundation and its track record of providing a level playing field is most of what it has going for it. To that end, putting DWR at the Foundation and not under the getahead legal entity should help to make the distinction between commercial interest and community development even clearer than it previously was.

An umbrella organization to help support projects in meeting their goals is the under-appreciated bit of what separates successful stand-alone projects from projects which can't escape the yolk of either a closed development process or a sponsoring company that just can't let go of assumed control or brand affinity. Having your work backed by a legal entity is essential if you're not going to pick a GPL-ish license, but having that entity be a brand-neutral known quantity helps get projects adopted by organizations which have both lawyers and some experience with OSS. For component software like DWR, Cometd, and Dojo that's nearly all of the users which an OSS license alone won't convince. The independent nature of the Foundation also isolates users from the employment decisions of key contributors. Through the Foundation, Dojo's licensing has survived wholly in-tact through changes in employment status of nearly all of its most prolific contributors.

Having "external" committers, a reasonable process for minting new ones, and a peaceful atmosphere where developers can build trust along with software isn't rocket science but it does require some amount of prioritization of those concerns over personal and corporate directives. We're not perfect at this at the Dojo Foundation, and we haven't yet encountered the wrenching attempts at "brand drafting" which Apache deals with on a regular basis, but we're committed to keeping the process as hands-off as we can and working to ensure that the umbrella doesn't imply more than it really provides.

Our door is open. All that we require of new projects is that all committers on the project sign a CLA, that the lineage of the code be "clean" (within reason), that the project community is relatively healthy, and that you can convince the existing committers that yours is worthy of Foundation support. It has always been the intent of the Foundation to host projects that have nothing to do whatsoever with Dojo and it's my sincere hope that the DWR announcement drives this home, but we're not going to leave it to chance. More on this in a forthcoming post.

Until then, my congrats again to Joe and the entire DWR community. Thanks for giving the Foundation the opportunity to make good on the trust you've invested in us.