So I'm procrastinating (I mean...erm....thinking) on some code for a talk I'm giving in a couple of days, and as usual, I wind up doing mental gymnastics on every topic but the one at hand when I'm staring down a deadline. This time, I can't get my head off of mobile apps.
At a conference a couple of weeks ago, I got to see a version of KHTML on a phone, and for the first time in a long while my interest in doing mobile app development has been rekindled. The Series60 Python port was good, but I think that this several orders of magnitude more important.
Back when J2ME as something new, I got to spending a summer trying to get phones to do enough crypto to pass themselves off as smart cards. Now, this shouldn't be hard, if only because most phones include actual smart cards. Problem is that the carriers force device manufacturers to seal off access to all of the interesting hardware on cell phones from apps running on them. We couldn't get access to the real crypto co-processors already embedded on the phones. These closed platform antics of carriers and operators are in part to enforce monopolistic control of the user base, in part to exert cartel-like control over the handset manufacturers, and in part to enforce monopolistic control of the upstart application and content authors who they need so desperately but can't bring themselves to treat as anything other parasites.
They're the phone company, after all.
While I can't say much more than that about the project, I can say that we were able to get SSL working from a phone in 2001 without hardware or OS-level software access. Which only served to teach me how utterly evil telcos are. With the proof-of-concept project done, it came time to ask ourselves how we'd get our app into the hands (literally) of people that could benefit from it. The stunningly bad answer was "you don't". At least not without lawyers, businessfolk, and a small mountain of cash. There are two major "platforms" for building apps that get delivered to phones: BREW and J2ME. Both suck.
Carriers who are sufficiently in bed with Qualcomm to have built CDMA networks are that much more likely to be enforcing BREW as their app delivery "solution". The system works something like this:
- developer buys handset
- developer submits to cavity search and pays Qualcomm for the privilege
- developer builds app
- developer submits app to 3rd party for "verification"
- developer agrees to every demand of carrier he/she wishes to launch apps on. Of course, developer can't even get a look at these before submitting to aforementioned cavity search
- developer tries to negotiate some sort of a deal with carrier that will give them reasonably good placement on the captive application portal through which all applications on carrier's handsets must be installed (silly developer, you're not a media conglomerate!)
- developer thanks carrier for such handsome service and ease of development with a steep cut off the top. Note that at this point the carrier is enforcing a billing system so that it's both impossible and irrational for Open Source software to ever make it into the hands of users
Now, admittedly, BREW is worst-case-scenario for mobile app deployment, but things are hardly much better over in J2ME-land. MIDP 1.0? MIDP 2.0? CLDC? KJava? How in the hell is anyone supposed to target a "platform" when there are so many permutations and versions of these castrated "profiles" with vendor-specific extensions hanging off the edges all over the place? It's not like Java is a forgiving environment when the right API isn't available at the right time. That wasn't part of the spec. Well, ok, to be fair, it wasn't part of any of the myriad applicable specs, not that you'd be able to find that out easily. And that's not even addressing getting the app onto the phone if the device supports some variety of J2ME. Carriers that ship J2ME-enabled handsets aren't somehow more enlightened when it comes to demanding their pound of proverbial flesh. They just don't have a unified system with which to extract it.
And that, in a nutshell, is why doing mobile phone development is the software equivalent of the life of Job. Just when you think it got bad, you look over and your cattle are dying of famine and the APIs you were counting on have withered on the vine of carrier apathy and manufacturer helplessness.
Which is why the web is our trojan horse (again). This is gonna sound like old news, but remember when there was this entrenched monopoly that controlled what kinds of apps could succeed (or be bought and then re-introduced) and which would be crushed under the fist of the almighty marketing dollar and vaporware announcement? The web routed around some part of that damage. Open Source did too, but it depends much more heavily on the concept of a liberated hardware platform, which is why RMS is fighting so darned hard to keep "trusted computing" out of the bootstrap of commodity computing devices via GPLv3. I don't agree with RMS most of the time, but I think his fundamental understanding of the problem regarding closed hardware and Free Software is correct.
Which is why the web is doubly important to mobile devices. Instead of being inert data, it turns out that the web has all of the characteristics of a programming environment that can succeed in the face of device foibles, carrier antics, and handset manufacturer impotence. WEP failed and the heir apparent is the web as we know it today.
If we can get the UI figured out.
I've been noodling a bit on the problem of "stackable" interfaces for mobile devices. It seems to me that for as good as the KHTML demo I saw was, it lacks the ability to easily build spatial or temporal associations with a piece of data in a page. The S60 KHTML makes it easy to relate pages to each other, but it's hard to scan a page for important information to dig into if everything is "below the fold" or if the page is zoomed out so far that you can really only make out visual landmarks. What we need is a way to promote things that are essentially "headlines" of the page into that spot in a remixed "folded" view of the page. This stack or list of "headlines" might represent Augment's outline view or a stack of cards with a fast "flip" function, but the content has to be the regular-old-web. It's one applications where I see micro-formats or a Dojo-style extra-attribute syntax as being essential. Instead of air-headed <:div> tags to separate things out such that CSS can have it's pretty way with the content, it would be much better if sections of a page which contained related data could be marked as such. The developer incentive to do this would be that page sections that do this would be "hoisted" into the stack whereas those that don't could be ignored by a browser (or script) that creates this sort of UI. This way, advertising content could be placed in the stack along with regular content, keeping everyone happy and still giving users a better search-and-drill experience than they currently have.
All while reducing carriers to the glorified purveyors of bandwidth that they should have been relegated to years ago. The web is our end-run. Our fulcrum.
I suspect we haven't even begun to see the start of the browser wars that will actually matter for most of the world's population. Add some limited local storage to a remixable, stackable web interface, throw in bluetooth or proximity networking and you have village-level networking. Negroponte's immoral gamble is likely to fail at the hands of good-old worse-is-better web tech. And the best part is that it starts with kicking the telcos in the groin here at home.
Either that, or I need to start getting more sleep.
Wikis, as a rule, suck.
I say this as someone who spent the last year and a half working at what is generally considered to be a "wiki company". I hated wikis before I went to Jot, and now having seen the strange cult of wiki even closer (Jot's competition), I haven't been dissuaded from my view. In return for a single interaction win, WikiWords, wikis throw away most of what is obvious and basic usability design. Had Jot not been committed to doing WYSIWYG in a wiki right, I would never have gone to work there. Heck, the one thing that 37signals got right about writeboard is that users don't want a wiki. They totally missed the point that end-user simplicity sometimes comes down to hard engineering, but that's for another day.
Users pick wikis (and tools like sharepoint) because they want to share the stuff they're writing. Not because they think that wikis are cool or somehow superior to everything that came before and/or after. Sounds obvious, huh?
WikiWords as the potential basis for an interesting collaboration tool: good. Wikis as a religious creed about some sort of artificial "simplicity" related to one-off markup languages: very bad.
Today Jot quietly launched a new major version, 2.6. This version includes a brand-new editor component implemented as a Dojo widget. That it's Dojo-based isn't something to be proud of in itself, but the new editor provides fixes to some of the most serious issues that plague WYSIWYG on the web today. Instead of taking you to an "editing" interface for a page, the new system uses Ajax-based editing to "upgrade" the page in-place so that editing is as fluid and natural as reading.
No more forgetting what it was you were going to change as you wait for the editing interface to load. Just change it.
Perhaps more importantly, though, the new editor component does everything it possibly can to avoid data loss. Browsers do a pretty good job of keeping data that's been filled out in a form around in case you accidentally hit a link or otherwise go to a page and want to come back to where you were. WYSIWYG editors, by comparison, totally flub this since they aren't native components that the browser knows to save state for. Jot's new editor fixes this with some sleight of hand, and the hope is that you won't even notice it. That it Just Works (TM) is its best feature.
The new editor works on IE, Firefox, Safari, and Opera 9 (modulo some Opera bugs), doesn't break the native undo stack, instantiates much faster than other WYSIWYG controls, and doesn't impose the lame "dueling scrollbars" scenario on users. Instead of having independent scroll bars for editor content and for the page, Jot's editor supports a "fixed toolbar" system that allows it to grow in height with the content, but never loose the editing controls at the top of the page when in editing mode. All of this has been made available as Open Source software in Dojo 0.3.0 via the
While the Jot editor is a thing of technical beauty, hopefully users will never think about it. Having been working with it for months, I can say that while I don't think about using the new version, I certainly become annoyed when I run across a wiki that's not using it. The experience isn't as fluid...everything else just feels clunky by comparison. The new editor is the system we all envisioned a year ago when Paul first started evaluating how to improve the WYSIWYG experience. By doing our homework about what makes a great experience and by staying true to the mantras of "never break undo" and "WYSIWYG, not semantics", Jot now sports an editor component that is second to none.
Of course, the Jot team should also be tremendously proud of the Tracker v2 (screencast) release that is now integrated into Salesforce. I'm not sure when it'll roll out to current Tracker users, but having a look at the screencast should make users of Tracker v1 salivate for the new version. For most interactions, it's nearly indistinguishable from Excel.
From here on out, I'm just an observer to the stuff Jot is building, but the new Tracker and Wiki releases help reassure me that our time was well spent. These products get out of the users way and help them get their work done or help them to express themselves better, but do it with the advantages of the web built in from the ground up. These aren't products that can't think outside the monopoly, these are tools that make people's lives easier...because that's what they were designed and engineered to do.
I'm proud to have played a small part in building them.
If you're doing serious web development and you're not running Firebug 0.4 yet...well...you should be.
Depending on how the debugger UI turns out, I may be ditching Venkman for everything but profiling. Huzzah!
It's usually bad algorithm choice or an inability to move things that are actually slow to a faster path. I think this is part of why my love affair with Python hasn't ended despite the proverbial writing having been on the wall for a long time. I mainly blame the craptastic XML-SIG, earlier versions of Zope, and Guido's weird malaise about web development for this, but that's another sad story for another day. In Python, you don't worry about language speed. You develop fast and then when you need to optimize, you profile. If things really aren't tractable (which is rare), you just move those critical sections to C. This is a common theme across most modern scripting languages. In Java, you can still get to C via JNI, but it's a second class citizen. Perhaps it's all the "pure java" marketing BS. Perhaps it's how difficult JNI makes it to get across the border and back. Who knows, but Java has certainly suffered as a result.
Phobos is starting to fix this by acknowledge that instead of making the Java mistake (again) and attempting to boil the ocean in yet another new language, we should be riding on the investment in libraries that are already available and make it possible for application authors to work at a faster, higher level and drop down to "the metal" where it makes sense. Obviously Phobos won't solve all of Java's myriad problems, but it will at least allow the Java platform to compete on developer productivity. The tools haven't saved us, and I doubt they ever will. Time to look up the stack. Kudos to Sun for having the foresight to free developers from the shackles of expressive mediocrity that Java has imposed for so long.
Which brings me to GWT. I got asked about this at the smackdown panel and of course I hadn't had nearly enough time to dig into GWT to form a coherent answer. I'm not sure that just looking at the code and playing with the demos has given me enough depth to give a useful answer either. From where I sit, GWT looks like the physical manifestation of hiring managers' frustration when trying to hire for "ajax developer" or "UI engineer" positions. The truth of the matter is that there just aren't very many of us in the world and interestingly, Google employs a fair number of them. That they did GWT smacks of recruiting desperation and the need to better leverage the Google SSFPU (super-smart fungible programming unit).
Turns out you can't just drop a good hacker into UI programming and trust that it'll work out. Not only does someone in the UI engineering role need to have a solid appreciation for the constraints of the web, they need to be multi-language clued, know when to defer to visual designers, have the balls to push back on stupid requirements, and empathize with users. And that's the baseline. No wonder Ajax is sexy and hard to hire for. The great apps in the Ajax word are developed by great engineers and designers.
Not coincidentally, great engineers and designers value great tools. Great tools allow them to express themselves better, faster, and with less effort. This is why the smartest hackers I know want either a fully dynamic language or C. Everything else (yes, you Java and C#) are bad compromises to these folks. They raise the average by enforcing mediocrity. This is a Good Thing (TM) on average but a truly awful thing for people who need to move faster than the herd. Traversing the stack with ease, closure to slab allocator, is what defines great hackers. A high-profile investor and former hacker has written broadly on the subject, so I won't try to rehash his observations. What I do find interesting though is that folks who think they want great hackers are shocked by how bloody honest they are when they finally land one. These people call bullshit because they're constitutionally unable to stomach dumb. Before I moved to the valley, I thought that the shorter average tenure of people at tech companies here was the result of competition, but the more time I spend here the more I realize that for the best-of-the-best, it likely has more to do with the impossibility of moving Mt. Stupid than with competitors offering a better package or employers folding up shop.
So what does that have to do with GWT? Well, I think that like Java and C#, it's going to raise the average. This is also a Good Thing (TM). I'll go out on a limb and say that it's a safe bet that Bob, Cal, and Andy won't be picking it up any time soon. If they do, it'll probably be to build tools for other people. Oddly that might even be a sign that it will succeed in a broader market.
The obvious next question is "what does that mean for Dojo?".
At the end of the day, both approaches are trying to get to a better level of productivity for building more responsive apps, and that can only be good for everyone.
Update: just saw that the Ajaxians have been thinking about GWT as well.
I was supposed to give a talk in Amsterdam yesterday, but as the airlines would have it, there was no such luck.
I got as far as Houston in my trans-continental journey before being stymied by aggregated air travel stupidity. Luckily, Andy Smith of Flock fame stepped into the breech and gave the talk in my place.
The talk was written out ahead of time and Andy can read better than I ever could, so I assume it went as well as possible. At least he didn't report any sharp objects being thrown at him afterward.
The text of the talk is here.