View-Source Follow-Up

One of the points I made during last Saturday’s panel was that the further down the path we go with JavaScript, the more pressure there is to use it in ways that defeat view-source. Brendan called me out for seemingly not being aware of beautifiers which can help correct the imballance, but I think that response (while useful) is orthogonal to the thrust of my argument.

Indeed, I hadn’t personally used the excellent jsbeautifier.org, instead using my own hacked-up copy of Rhino to beautify when I need to, but neither tool sufficiently reverses the sorts of optimizations being employed by GWT and the even more aggressive Closure Compiler. Far from the mere obsfucation of ShrinkSafe and its brethren, these new compilers apply multi-pass AST-based transforms on an entire body of code, performing per-browser optimizations, type inference and annotation based optimizations, loop invariant hoisting, function inlining, dead-code removal, and global code motion optimizations that produce code different not only in style but in flow of control. The results are nothing short of stunning. The Closure Compiler can deliver code that’s much, much smaller than I can wring out by hand and that performs better to boot. It’s also totally unrecognizable. De-obsfucators have no hope in this strange land — brand new tools akin to symbol servers and WinDbg-style debuggers are needed to work with the output in a meaningful way. I argued in the panel and in the comments of my last post on the topic that when we get to this place with JavaScript the product is functionally indistinguishable from a bytecode format like SWF or Java and the effects on the learning-lab nature of the open web are the same: less ability to easily share techniques, a smaller group of more professional users, and a heavier reliance on tooling for generating content.

If we assume that the furthest down the code-centric path we’ll go are the Dojo and JQuery style augmentations of existing content, then a simple de-obsfucator is sufficient. But I’m afraid the current generation of high-end sites and apps points in a different direction, one that is less hopeful, and one that implies to a greater extent that the browsers must lead the way out of the wilderness by creating new tags and CSS properties to help re-democratize the process of creating applications. We’ve already seen the alternatives, and while they may be elegant, they lack leverage and the second-order beneficial effects that have made the web as successful as it is today.

If HTML is just another bytecode container and rendering runtime, we’ll have lost part of what made the web special, and I’m afraid HTML will lose to other formats by willingly giving up its differentiators and playing on their turf. Who knows, it might turn out well, but it’s not a vision of the web that holds my interest.

15 Comments

  1. Posted March 17, 2010 at 1:23 pm | Permalink

    It’s a good point and I agree with the sentiment, though seemingly like you I don’t have a solution that I’m satisfied with or that will satisfy everyone at stake.

    Providing the uncompressed, unoptimized JavaScript alongside the “production-environment” one is unfortunately only an option. It also doesn’t easily allow folks to step through the code, watch events, etc on the live site (unless you provide two versions of the app as well).

  2. Posted March 17, 2010 at 1:27 pm | Permalink

    I should also state, as I’ve stated before, that I think the state-of-the-art needs to “move up the stack” in terms of browser environments, tooling and abstraction.

    Perhaps this isn’t a contradiction.

    Perhaps as time goes on, I will be able to deliver my plain, commented JavaScript over HTTP and the browser will do the compiling into byte code directly. This might even be another competition point between browsers.

    From the declarative side, I would expect XBL, XUL and that type of thing to one day take hold on the web side meaning that the ‘lower-level’ building blocks of HTML and SVG do kind of fade away.

    I talked about this a couple of years ago here: http://www.codedread.com/blog/archives/2005/12/15/putting-the-pieces-together/
    Food for thought, anyway.

  3. Jason!
    Posted March 17, 2010 at 1:59 pm | Permalink

    You seem to think you’re almost entitled to view what might be proprietary source code simply by virtue of the fact that it’s been customarily transmitted “in the clear”. Very Stallmanesque here.

    The web _is_ special, in that much of the JavaScript source code from everyone has been available to everyone generally unmolested. Almost all other languages people program in get compiled, and you never see the source code unless it’s specifically Open Source.

    I’d imagine that even with the advent of more JavaScript code obfuscators (since that’s really what they seem to be, adding no functionality, and removing legibility), people will make conscious choices about how “Open” their JavaScript gets to be.

    This is not a necessarily bad thing. People have managed to learn C/C++/Java/etc. without all the code out there being available for anyone to see at their whim.

    The main problem I see is code developers mistaking obfuscation for security.

  4. Posted March 17, 2010 at 2:07 pm | Permalink

    I’m slightly more hopeful than you. Javascript is not quite the same as bytecode formats like Java and Flash. Those frameworks *enforce* obfuscation. Javascript makes it optional and indeed obfuscating requires more effort not obfuscating. I may be totally mistaken but I would guess that the vast vast majority of JS is not compressed. Even if a site includes dojo the site-specific code is likely to be fully open. Beginners don’t learn from dojo code or Gmail code. They learn from the little snippets of functionality found all over the web. As long as this situation persists, view-source should be ok.

  5. Posted March 17, 2010 at 3:31 pm | Permalink

    Jason:

    Um, no. By way of background I helped set up the Dojo Foundation because licensing and property rights *matter*, even for things you can read. The remedies are legal, not technical, though. In compiled languages decompilers can help folks understand what’s going on, so if that’s your threat, then compiling isn’t gonna solve anything.

    Now that we’re past you reading too deeply into my perspective, lets talk about folks caught in the middle: people trying to just Make It Work (TM). For them, learning from snippets they find on the web in order to learn a technique is invaluable. Poll any average group of web developers and they’ll tell you straight-up that that’s how they learned. Not be stealing, but by dissecting and inspecting and tweaking and then re-implementing a technique in a new and unique situation. What we’re contemplating with ever-heavier use of JS is shutting many of the least experience out of this process entirely. That might be OK in the short run but the long-run effects aren’t positive.

    Rowan:

    I totally agree that this isn’t one-or-the-other, but I see two risks: the first is that the web stops being competitive for any given amount of developer effort. Everything that requires JS today is by default *harder* to do than the same thing if it were provided by CSS or markup. The second risk is that more of the content will go the way of gmail code out of necessity as tools and compilers become part of the assumed toolchain in response to increased expectations about interactivity and UX. I think we’ve got some years before we’re totally hosed here, but we need to act today to make sure things end up in a good place; encouraging browser vendors (like me) to add new native capabilities and ask for forgiveness instead of permission.

    Regards

  6. Posted March 17, 2010 at 4:06 pm | Permalink

    What about promoting CoffeeScript? Its white space cannot be trimmed by the optimizers/obfuscators, because, just like Python, is significative. So it can be considered like a warranty for the maintenance of the clearity of the source code.

  7. Posted March 17, 2010 at 4:10 pm | Permalink

    CoffeeScript is just another code-centric solution to a problem. It doesn’t have any more leverage than, say, any of the existing JS toolktis do. My sense is that we’ll move forward by adding APIs and capabilities to the platform that let us write LESS code, not more or different styles of code.

  8. santos
    Posted March 18, 2010 at 12:14 am | Permalink

    I agree with your sentiments, but not with your conclusions. JS is just a nifty toolkit of shims and duct tape. It’s a way to experiment with new functionality that can’t be implemented without a good turing machine. It’s helpful for figuring out *what* we need the next generation of browsers to do. Then, we can figure out how to give a democratic API for that functionality. I agree that procedural programming is too difficult for the average web citizen and non-democratic.

    Saying that “ever-heavier use of JS is shutting many…out of the process” is like saying that Chuck Yaeger shut everybody out of flying on airplanes. The heavy use of JS is trying to solve a different (and complementary) problem.

    On the other hand, I’m not convinced that ever-more-complicated HTML markup is the next step either. I’m really excited about some of the cool fx that webkit people are doing with CSS. Or maybe it will be a new markup language entirely.

  9. Posted March 18, 2010 at 1:56 am | Permalink

    > Poll any average group of web developers and they’ll tell you straight-up that that’s how they learned. Not be stealing, but by dissecting and inspecting and tweaking and then re-implementing a technique in a new and unique situation.

    Not that he’s average, but I believe that’s a big part of how Bill Gates learned to program too — going along to his local university’s computer science department, and reading the print-outs of what they were writing there.

  10. Posted March 18, 2010 at 10:51 am | Permalink

    Hey Nick:

    I totally agree that the right solution needs to the tiered process you recommend, with the steps roughly outlined as:

    1.) build in script where possible
    2.) wait and see what gets popular
    3.) translate to native API and ship in browsers
    4.) duke out “official” shape of native API at standards body

    I don’t mean to suggest that there’s not *any* natural place for script, only that if we think “the plan” should continue to require ever-larger piles of script, it’ll mean that steps 3 and 4 just aren’t happening at an acceptable rate.

    Put another way, we need compilers and minifiers only when we need so much script that we can’t just write what we mean in a couple hundred lines. When the platform does most of what we need without huge piles of script, the pressure to use tools that defeat view-source (either plugins or huge piles of script) goes down markedly. Maybe that means we need new extension points in the platform ala HTCs, maybe it means we need to be able to write a lot of behavioral code in fewer charachters (the DOM sucks today), and maybe it just means we need new APIs to do common-case stuff.

    In any case, I’m not arguing that minifers or compilers are *bad*. They keep the web competitive until such time as the cavalry shows up with new features in the platform. But they absolutely should have a sell-by date.

    Regards

  11. Posted March 18, 2010 at 2:22 pm | Permalink

    Alex,

    I completely agree with you that view-source is important. It’s how I learnt most of my web stuff. However, obfuscation is not necessarily what site owners want, right? Performance and latency are of very high importance, and I wouldn’t care two hoots about obfuscation, yet I’d compress. If Closure Compiler can help me improve experience, I’d go with it. If Packer does a better job, that’s the way to go.

    I don’t know how to solve this problem. Even if jQuery, dojo, etc. were bundled with browsers, I’d still compress my code. Maybe the only way to solve this problem is to invent a new mechanism of distributing code without it being dependent on latency of any form. Compression then would have no value, and people would start shipping source, comments and all, without thinking too much about it. I’m not smart enough to see how this could happen, though.

    I already run a site where I’m distributing JS without compression since the value of code compression on the little JS there is is so little that I can overlook it. Gzip does the trick well enough. But I can see that it doesn’t apply to everyone.

  12. Posted March 23, 2010 at 1:31 pm | Permalink

    Alex, Santos, et al.

    regarding re-democratizing the web, new CSS properties, JS to native API adoption, etc.

    Layout is one of the things “that requires JS today [that] is by default *harder* to do than… if it were provided by CSS.”

    In several past rants by some folks bear this out although CSS3 helps to some extent.

    I have developed a layout spec in CSS that incorporates JS. I know that Alex commented on it before expressing his reservations last July. however, my description and treatment of it at the time was too superficial for an adequate hearing. I have since updated it to a large extent at http://blogs.eforceglobal.com/dkarisch/archive/2009/07/16/536.aspx

    I think it not only addresses continued weakness with layout in CSS, but also fits in with the general tenor of the desired qualities that I referred to at the beginning of this post.

    I would have preferred that this reply be under the CSS 3: Progress! post from August, but I’m too late for that.

    nevertheless, this post is at least somewhat apropos.

    alex’s comment back in July, the slightlyoff character at http://ajaxian.com/archives/css-scripting-layout, states that “it [layout script] can’t interact with the browser’s (potential) caching and batching of layout operations.”

    Alex, can you please tell what you mean by this statement?

  13. Justin Meyer
    Posted March 25, 2010 at 9:50 pm | Permalink

    Great article. However, I’m not as pessimistic. When I started on JS 5 years ago, view source was criitical. However, there is little I want to do anymore that can’t be found in dojo, some jquery plugin, or article describing what I need.

    And all I do is write complex JavaScript code. When I’m doing something that hasn’t been done, the browsers or w3c have good enough docs for me to figure it out.

    I think view source fostered the right culture that will be carried on by great frameworks.

  14. Neil Kandalgaonkar
    Posted March 28, 2010 at 4:33 pm | Permalink

    There seems to be two problems here:

    1) Browser-delivered apps are more complex. There’s no way to fix this. But many of us are about to develop in a thin layer over mature libraries, so that problem is somewhat mitigated.

    2) Obfuscation and compression. Do you think the world could use a standard attribute in HTML, to link to the uncompressed source? To somehow give the recipe for the page?

    Our colleagues who work on native OS open source applications have always had this problem, and have devised many cultural workarounds, including the GPL. Even then, setting up an editing and building environment for a new open source project is still a tremendous pain.

  15. Posted April 28, 2010 at 12:35 pm | Permalink

    alex,

    the reason I ask you to elaborate on your comment regarding browser caching and batching of layout operations is because what I have implemented does include layout caching in the browser. so it is not the case at all that scripting precludes caching of layout operations.

    you can download the POC from http://downloads.eforceglobal.com/CSS/CSSScriptLayoutR21525.zip and check out the performance yourself. you can resize the browser and note the resize performance of the container and nested containers in the examples provided. the executable should work on any windows box.