Infrequently Noted

Alex Russell on browsers, standards, and the process of progress.

Yahoo Pipes Support in Dojo

As many a Dojo user is aware, we've been supporting Yahoo's forward-thinking JSON-P style endpoints for their various web services for a long time. We even have a nice RPC wrapper and simple description format for most of them. And no, you don't have to download YUI to get at this's all pure Dojo. We make it easier to work with Yahoo's services than Yahoo does.

And that now includes Yahoo Pipes. It's cool and interesting to be able to call out to Pipes from your server-side web-app, but what the mashup-mad word REALLY needs is to be able to do the same thing from a browser. Despite not really being in the docs anywhere, Yahoo's Kent Brewster points out that Pipes supports a JSON-P callback argument. Awesome!

The structure of Pipes URLs are different than every other Yahoo service (much like flickr. ugg.), so there's no Dojo RPC for it yet, but you can easily query a pipe using and the ScriptSrcIO transport:

// get news results for Cometd
dojo.require(""); // the x-domain magic
dojo.require("dojo.debug.console"); // firebug integration{
// grab this URL from the pipe you're interested in
url: "",
mimetype: "text/json",
transport: "ScriptSrcTransport",
jsonParamName: "_callback", // aha!
content: {
"_render": "json",
"textinput1": "cometd"
load: function(type, data, evt){
// log the response out to the Firebug console

Pipes is a bit slow right now, so you might not want to use this for main page content, but it's a great way to decorate pages, blogs, and other kinds of data with a lot more context from all over the web.

Cutting Back

One of my goals for 2007 is to spend less time on the road, and SitePen has been helping to make sure that I get real blocks of time for R&D. This is all the more important given the aggressive Dojo 1.0 roadmap. As a result, I'm skipping some of my favorite conferences (SxSW, ETech, etc.) and doing less speaking in general. But that doesn't mean I'm not traveling at all, just that we're trying to "make it count".

Here's an abbreviated version of my conference schedule until about June:

Believe it or not, that's a relaxed schedule from last year. If you're going to be heading to any of these and want to catch beer, don't hesitate to drop me a line.

War Powers

I don't blog much, if at all, about politics but I spend a lot of time thinking about it. Recently, someone told me that members of my generation don't care or know anything about politics. I was aghast at the suggestion but upon reflection I think I can understand where he was coming from: having grown up in an environment of growing disenfranchisement leads to a belief that it's impossible to effect change, even if you think you know what to change. I don't even remember a time when congressional districts weren't gerrymandered beyond recognition or when incumbency was unassailable. Unless something changes, my little brother will only vote in presidential elections where untrustworthy electronic voting systems are the norm.

Despite all of that, however, I vote every chance I get. Jennifer and I study for elections, usually reserving most of a weekend beforehand to pore over the hundred-plus pages of voter information booklets that get shipped to every California voter. We spend time researching, trying to pick the best person for the job, peering through the morass of private interest and political machinations and not always coming away feeling like we really understand all that's at stake. I've never voted a straight party ticket in my life, mostly because I don't think anyone really has all the answers. I expect my elected representatives to duke it out to a good compromise. I want the kind of slow, deliberative government that leaves everyone slightly bruised and no-one very happy.

And I worry a lot about the health of the fourth estate. I'm something of an NPR junkie and my favorite show is without a doubt On The Media. It says something about me that I'm not sure I want to acknowledge that I try to make sure I'm by a radio every Sunday a 2pm to hear it. Even though I already get the podcast.

Which is why the administration's saber rattling toward Iran scares the living shit out of me. The Senate can't seem to shake its limp-wristed, morally-equivocting response to the last war while an executive that couldn't be bothered to plan for the aftermath of a war of choice attempts to foment another. The disastrous consequences of our failure to win the peace in Iraq and Afghanistan brings me close to tears. We've shattered hundreds of thousands of lives, returned Afghanistan to the warlordism and feudalism which gave rise to the Taliban in the first place, and wrought untold damage in Iraq. It was our responsibility to ensure law and order after ousting the existing regimes of these countries. This isn't the America that liberated Germany, this is the America that fucked Guatemala for half a century and overthrew an elected democracy in Iran. That overthrow, incidentally, has led us into a trumped-up "conflict" with a bruised, malignant, and blighted society next door to an American quagmire. It frightens me that Bush administration shows no signs of wanting to understand the causes of Iranian intransigence toward American foreign policy dictates. One can only imagine the lack of perspective and historical context in the discussions surrounding American/Iranian policy. The Bush administration wants influence in the mideast and something, anything to draw attention away from 30% approval ratings and a war that is now receiving something that might one day resemble congressional oversight.

Those are both pretty thin reasons to go laying the groundwork for yet another conflict in a region that is both critical to our national interest and growing more unstable by the day. As Juan Cole keeps pointing out, the agro directed in the general direction of Tehran doesn't even make any sense. And it's not like there are any good policy options for a president intent on trying to bully Iran anyway. Sure, there might be some purely military options that would perhaps be available to the US given that we've now got permanent bases in Iraq, but as James Fallows has illustrated, none of them actually make policy sense. Which is why congress needs to find its hands and then use them to find its ass, and quick. Spun the right way, the over-broad authorization of force against Iraq could be twisted to justify limited tactical strikes against Iran. When it blows out of proportion (as it inevitably would), will congress have the cojones to say no? I doubt it. They can't even get to a debate on a freaking non-binding resolution against a horribly unpopular war backed principally by a lame-duck president with approval ratings not seen since the Carter administration. Even with elections looming. Such is the jingoistic power of potentially being seen as "weak on defense" or "not supporting the troops".

And so the media fails us again, the administration makes allegations that aren't even plausible, and the attention that should be directed to a thousand other places gets focused on what can only be described as the wrong things. How long can our flirtation with authoritarian incompetence last? And what will we pay in the decades to come for our duplicitous national arrogance and cowardly ignorance?

JS RegEx's Are Slow

I've been trying to avoid regular expressions in dojo.query. It's a difficult tradeoff since regexes are very space efficient vs. lots of indexOf operations. A little digging on the point turned up a fascinating article on why regexes in most languages are slower than they need to be.

Now if only there were some explanation of why regular expression string replacement, not just matching, is brutally slow across the board.

When Utility Isn't Enough

I finally blogged the work I've been doing on dojo.query. As you can guess, it's a CSS query engine for Dojo, and as you can probably also guess, it's fast. Or at least as fast as you can make a system that needs to hoist itself by its own petard thanks to the network latency of sending the darned thing down the wire every time. As web developers continue to expect ever more out of their tools, it's becoming clear to me that the dynamics of the "Ajax platform" are a deck that's heavily stacked against novices to the advantage of experts. For a long time we've been trying with Dojo to help tilt that balance back in favor of those who haven't co-evolved their skills to suit the peccadilloes and vagaries of the web. But I don't know that we (or anyone else) is really succeeding at it. The "Other Language to JavaScript" compiler crowd is building a very leaky abstraction that is going to take years to get right. Meanwhile, folks trying to do "something interesting" tend to look at the current state of play in pure-browser apps and run headlong into the arms of Macrodobe or Microsoft. Flex is a surprisingly easy sell for organizations that don't have the health of the web in mind.

Ajax is getting us "old apps ++", but even with Comet in the stew, too much complexity is still laid at the feet of end developers. Complexity == cost. Cost == opportunity. Is it any wonder that the most interesting startups today are either retreads of old concepts or simple integration of new data paths into the old ones?

So long as it's still this hard for everyone to build web apps that don't contain nasty surprises, the web's formidable advantage in openness will always be at risk of being dominated by other factors. As the Linux world has painfully discovered, being open isn't a guarantee of value to everyone you wish to entice. It's an opening gambit in convincing people of top-to-bottom value and not even a strong one.

I'm starting to focus more and more on the "sharp edges" of the web development experience because I find that web developers discount mental overhead and workflow impediments without much thought to the real costs. Jot's design hit a chord with me precisely because it rounded off some of those sharp edges: the hours of database and web server setup. It's those kinds of assumed costs that keep tripping us up. They're fine for organizations that want to be experts or think they can get competitive advantage out of it, but how is it good for everyone else?

One of the big goals with dojo.query wasn't only to be relatively fast compared to the other query engines that you can chose from, but it's to help even out the performance peaks and valleys. It's still possible to write poorly performing queries with dojo.query, but for the most part you're going to have to do more typing to get a slow query than a fast one. It may not be the kind of thing that webdevs will notice, per sae, but rounding off the sharp edges is an exercise in usability: things are only useable when they do what you expect them to. A system that hurts you more than you expect isn't useable.

I'm not sure that I have a point with all of this other than to hope that others will either point out a great big hole in my logic or, like me, start to work on rounding off those edges. If I'm right, the web needs us to question our sacred cows and continually sunk costs. Openness might not have strong value on an individual basis, but in the aggregate it's important for everyone.

Older Posts

Newer Posts