Monday, October 26, 2009

The Complicated Software Stack

To aspiring Web application developers or people looking to put together their own Web application: the road to building a modern working Web application is a long and complicated journey.

Today's Web application developer is nothing short of a jack-of-all-trates, requiring deep knowledge of everything from HTML and CSS to Java and SQL. Everything from common CRUD tasks to sophisticated work-flows requires knowledge of half a dozen computer languages along with their quirks and variations across platforms and applications.

Today's software is built using a mix of programming paradigms and data models. Every level in the software stack requires explicit data mapping between paradigms. Many Web applications include the following levels in their software stack:
• Relational for persistence,
• Object oriented (class-based) in the model,
• Aspects peppered throughout,
• Resource (or activity) oriented Web services,
• Functional template engines,
• Markup using key/value pairs, and
• Prototype based objects for UI behaviour.

The above complication comes at a price. Software takes longer to develop and is more expensive to maintain than it used to be. This is causing a greater divide between small tools and large software systems.

Applications, like Microsoft Excel, which combine data processing and persistence using a consistent programming paradigm, have grown in popularity as a cheap alternatives to the complexity of modern Web applications.

While the market for Web applications has grown, the scope has decreased, favouring large high volume systems. Smaller Web applications are too often over-architected and over-budget. There is a large (and growing) opportunity for software vendors to fill this divide and create a new platform that combines data processing and persistence, using a single programming paradigm, for Web applications.

Can Web applications be built to use a single programming paradigm?

Reblog this post [with Zemanta]


  1. Thanks for the link Mike. Do you have any comments on JS persistence and HTTP services and serialization?

    JS uses a graph structure (in memory), but the primary serialization format (JSON) is a tree structure. Have you overcome any interesting solutions for this? Do you recommend using JSON for persistence?

  2. Very true, James. I also wish the programming threshold was lowered so anyone could create web apps.

  3. While I don't think we can reach a completely unified single-paradigm solution for the whole Web stack there are some solutions that get close to it.

    Using JavaScript would be one. You have it on the client side, there are implementations on the server side and there's JSON for communicating between both. With CouchDB you can even use JSON for persistence. Although as you point out it is somewhat restricted.

    Another solution is the XRX architecture (using XForms, REST and XQuery). Here the data can flow without need for translation but I'm not sure sure if - programming-wise - that's all you need.

    Whatever the solution I think it will always have the "resource-oriented" component because that's how the Web itself is defined, so everything has to adapt to this. I found that very often on the server side you can drop the object-oriented bit. So instead of having "resource" classes which operate as controllers over domain-model classes you can just have the resources interact with the storage directly. I do realise that this might seem a bit scary for some because there's no centralised business logic and validation for the domain model entities but I think it's more natural to the Web.

  4. Interesting point on resource-oriented, I think you are right that the desired solution should use a RESTful protocol. However, too often that view point is not shared by tool providers.

    Google, for example, has been working on getting Java's (class based) objects on the client (via Google Web Toolkit (GWT)) and there is already a lot of effort to try and persist (class based) objects, either in an object-store or through an object-relational mapper. With both of these developers are able to eliminate the need to think about relational and prototype based objects -- simplifying the stack.

    The problem I see with GWT's approach is the proprietary protocol they built on top of HTTP is not RESTful, but simply uses HTTP as yet another RPC. It is not transparent and often obfuscated, making it difficult to debug or understand when something goes wrong.

    I have been working on finding a solution to networking objects together using REST and will blog about it soon.