Programming The Web

Blob Store

2012-01-15T11:39:00.000-08:00

In release 2.0-beta14 (I know, this is the late beta release) AliBaba introduced a new BLOB store. The blob store integrates with the RDF repository ObjectRepository to synchronize transactions. This allows both the BLOB store and the RDF store to be isolated and always consistent with one another. This is done using two-phase commit transactions in the BLOB store.

The BLOB store also has a few other advantages over a traditional file system. First every change is isolated until it is closed/committed. This prevents other readers from see an incomplete BLOB and help prevent inconsistency between the BLOB and RDF stores. In additional, as disk space is generally considered cheap, all past versions of BLOBs are keep on disk by default. This allows any previous versions to be retrieved (and restored) using the API.

The BLOB store API is fairly simple. Here what some code might look like using the BLOB store.

BlobStoreFactory factory = BlobStoreFactory.newInstance();

BlobStore store = factory.openBlobStore(new File("."));

String key = "http://example.com/store1/key1";

BlobObject blob = store.open(key);

OutputStream out = blob.openOutputStream();

try {

// write stream to out

} finally {

out.close();

}

InputStream in = blob.openInputStream();

try {

// read stream from in

} finally {

in.close();

}

More API options can be see in the JavaDocs:

http://www.openrdf.org/doc/alibaba/2.0-beta14/apidocs/org/openrdf/store/blob/package-summary.html

Web Developer Review of BlackBerry PlayBook

2011-06-02T10:51:00.000-07:00

Most reviews for the PlayBook focus on the same issue: very few downloadable apps in app world. As a web developer - I couldn't care less.

First Impression

Websites render fast, and due to the high dpi, look really nice. With its compact form, it fits well in my hands, easy to type and very portable. With a flash plugin included, streaming video is smooth and full screen works. Videos look really slick when plugged into a HD TV. Each app can only open one window, so the browser supports tabs and allows you to keep multiple tabs open at once.

Honeymoon Ends

Tabbed browsing works on the desktop, but not on the PlayBook. Only the open tab can be actively loading. Opening a new tab before the page loads can cancel the page from loading. Opening a new tab while watching video pauses the video. This makes watching commercials really frustrating because you can't turn away or it will pause. Watching videos in the browser is also frustrating, as after five minutes the PlayBook goes into suspend. (There are some tricks to stop this, but not if in full screen mode.)

In addition, despite all the fuss about multitasking, the PlayBook can't multitask. Most specifically you can only have one web page active at a time, and this includes webapps.

Surprisingly, the PlayBook is much less web developer friendly than I expected. The script engine is incomplete. There is no offline support for webapps. There is no support for turning a webapp application into a chromeless app. Webworks development requires a series of confusing bat commands that don't work the first time. All of this makes it really hard to develop for the PlayBook.

What's Left

The apps I use include Browser, Wi-Fi Sharing, Word To Go, Slides To Go, Videos, Pictures, aVNC, and ReelPortal. All of them work, but I expected more from almost every one of them.

All that being said, I am going to hold on and put up with the current limitation of the PlayBook. I really like having a portable web browser, and I believe there is still a lot of potential for this device. I am looking forward to seeing what the next software update has to offer.

Five Steps to a More Secure Web App

2011-02-14T06:39:00.000-08:00

There are a number of different authentication methods available to choose from when launching (or updating) a Web application. Choosing the wrong method can leave the system (or worse, the users) vulnerable to cyber attacks or identity theft.

Below are five rules that should always be obeyed (regardless of the method). By considering these rules and how your users will use your system, you can better understand the security requirements of your Web application and can choose the right method.

1) Never send clear user passwords over an unencrypted channel.

When passwords are sent over an unencrypted channel, anyone who has access to the network (and a little know how) can read them. This should never be done with user supplied passwords (not even for intranet websites). Users often use the same password for multiple systems. Exposing a user's password in one system puts them at risk in another.

Both basic authentication and form-based authentication are vulnerable to this and should never be used when users can choose their own passwords. Digest authentication and encrypted logins do not send clear passwords, and can be used when users can choose their own passwords.

HTTP basic authentication and HTML form-based logins can be used in secure networks to restrict Web access as long as the passwords are pseudo random, unpredictable, and unique across other systems.

For systems that allow user created passwords, care must be taken to ensure the passwords are not readable by others by using HTTPS or digest during logins.

2) Never send session tokens unencrypted over a shared network.

Unencrypted session tokens are visible to anyone who has access to the network. Although session tokens don't expose the user's password, they do allow hijacking accounts with unlimited access. This should never be used over a public wifi network (or other shared network) to access private information or make changes.

Cookie based authentication over HTTP is vulnerable to this. Digest authentication and HTTPS sessions are not vulnerable.

Digest authentication uses a unique "salt" for every request and digest systems prevent the same "salt" being used more than once (although this is optional). By never using and never allowing the same authentication token twice, digest authentication prevents account hijacking.

HTTPS requests are encrypted and prevent eavesdropping from others on the network, preventing access to any request tokens that might be present.

Only allow HTTPS using keys from a certificate authority, HTTPS with self signed keys, HTTPS with mixed content or digest authentication should be used to exchange private information over shared networks.

For more information about the vulnerabilities of using session tokens see Weaning the Web Off of Session Cookies.

3) Always verify information sent over an insecure network.

Insecure networks may be vulnerable to malicious attacks such as DNS posioning, or a trojan Web proxy. These attacks are often called man-in-the-middle and can manipulate the content from the server before it reaches the client (and vice-versa).

Most unencrypted HTTP communication is vulnerable to this. Even mixed content of both HTTPS and HTTP is vulnerable to man-in-the-middle because compromised HTTP content can read and manipulate HTTPS content.

Although digest authentication includes an optional integrity check to prevent this, most browsers either don't check or don't indicate to the user if the content has been verified.

All Web browsers verify HTTPS content (when not mixed) and this should be used for insecure networks. For mobile devises that often connect from potentially insecure networks HTTPS (self signed or CA signed) should be enabled by default for any private information.

4) Never give confidential information without verifying authenticity of the server.

Well disguised URLs and familiar looking pages can trick users into visiting and pseudo-logging into illegitimate websites. If your website asks your users for confidential information, ensure there is a clear way for your users to verify the authenticity of the site before logging in. Otherwise, your users might give confidential information to untrustworthy third parties without even knowing it.

HTTPS using previously distributed keys (such as keys from an established certificate authority) allow the user to verify the organization in their browser (near the address bar). This allows the user to quickly verify authenticity of the server.

HTTPS with self sign certificates cannot be used to verify authenticity unless they have been previously distributed through a secure channel.

Although digest authentication can include authentication-info to verify authenticity, most browsers either ignore it or don't indicate to the user when the site is verified. However, most browsers do show the host name and realm to the user for review before logging in and this does give the user a chance to check to domain name before logging in.

Always use HTTPS for confidential, or sensitive information.

5) Never access sensitive information over an unencrypted channel.

HTTP traffic can be viewed by any who has access to the network. It is vital that all sensitive information is never sent over unencrypted HTTP. Sensitive information should always use HTTPS.

Only exclusively HTTPS with known certificates should be used to exchange sensitive information with its users.

Always use HTTPS for confidential, or sensitive information.

In summary

By obeying these five rules you can pick the right authentication method and prevent your system and users from being vulnerable to cyber attacks and identify theft.

Status Code 200 vs 303

2010-11-28T16:37:00.000-08:00

The public LOD has been dominated by discussions on using 303 in response to a GET request for distinguishing between the requested resource identifier, and a description document identifier.

Some resources can be represented completely on the Web. For these resources, any of their URLs can be used to identify them. This blog page, for example, can be identified by the URL in a browser's address bar. However, some resources cannot be completely viewed on the Web - they can only be described on the Web.

The W3C recommends responding with a 200 status code for GET requests of a URL that identifies a resource which can be completely represented on the Web (an information resource). They also recommend responding with a 303 for GET requests of a URL that identifies a resource that cannot be completely represented on the Web.

Popular Web servers today don't have much support for resources that can't be represented on the Web. This creates a problem for deploying (non-document) resource servers as it can be very difficult to set-up resources for 303 responses. The public LOD mailing list has been discussing an alternative of using the more common 200 response for any resource.

The problem with always responding to a GET request with a 200 is the risk of using the same URL to identify both a resource and a document describing it. This breaks a fundamental Web constraint that says URIs identify a single resource, and causes URI collisions.

It is impossible to be completely free of all ambiguity when it comes to URI allocation. However, any ambiguity can impose a cost in communication due to the effort required to resolve it. Therefore, within reason, we should strive to avoid it. This is particularly true for Web recommendation standards.

URI collision is perhaps the most common ambiguity in URI allocation. Consider a URL that refers to the movie The Sting and also identifies a description document about the movie. This collision creates confusion about what the URL identifies. If one wanted to talk about the creator of the resource identified by the URL, it would be unclear whether this meant "the creator of the movie" or "the editor of the description." Such ambiguity can be avoided using a 303 for a movie URL to redirect to a 200 of the description URL.

As Tim Berners-Lee points out in an email, even including a Content-Location in a 200 response (to indicate a description of the requested resource) "leaves the web not working", because such techniques are already used to associate different representations (and different URLs) to the same resource, and not the other way around.

Using any other 200 status code for representations that merely describe a resource (and don't completely represent it) causes ambiguity because Web browsers today interpret all 200 series responses (from a GET request) as containing an complete representation of the resource identified in the request URL.

Every day, people bookmark and send links of documents they are viewing in a Web browser. It is essential that any document viewed in a Web browser has a URL identifier in the browser's address bar. Web browsers today don't look at the Content-Location header to get the document URL (nor should they). For Linked Data to work with today's Web, it must keep requests for resources separate from requests for description documents.

The community has voiced common concerns about the complexity of URI allocation and the use of 303s using today's software. The LOD community jumped in with a few alternatives, however, we must consider how the Web works today and be realistic on further Web client expectations. The established 303 technique works today using today's Web browsers. 303 redirect may be complicated to setup in a document server, but let's give Linked Data servers a chance to mature.

HTML-Oriented Development

2010-09-13T10:21:00.000-07:00

The heart of all Web applications is the user interface (UI) design - this is what its user interact with. As any consultant knows: clients are more satisfied with a well designed UI and mediocre business logic then they are with a poorly designed UI with minimal transparency and fully automated business rules.

What is surprising (when you think about it) is that most Web application frameworks orient around the business model and treat the HTML like a second class citizen. The conceptual model may be important, but even more important is the representation of the model in HTML. Good UIs provide the user with full transparency to the state and operations of the underlying model. It doesn't matter how well the model is if the HTML is too confusing or too obscure; users will avoid using it.

The HTML of Web applications is surprisingly rich with domain concepts. Most well designed UIs contain all the classes, relationships, and attributes found in the underlying model and present them to the user in a language everyone involved can understand. There is a lot of emerging standards that can help turn this human readable data in HTML into machines readable data using RDFa, microformats, or microdata.

Recently, David Wood and I started the project Callimachus; it has taken a different approach to Web application design/development. Callimachus reads the domain model from your HTML templates! In Callimachus there is no need to maintain multiple models, no SQL schema, no query languages, no object-relation mapping, it's all embedded in HTML using RDFa.

RDFa allows your HTML to include resource identifiers, their relationships, and properties using additional attributes such as: about, rel, and property. Consider the following HTML snippet. Using RDFa the data is readable by both humans and machines alike. It says that James Leigh knows David Wood using the relationship "foaf:knows" and the property "foaf:name".

<div about="james">
<span property="foaf:name">James Leigh</span>
knows
<div rel="foaf:knows" resource="david">
<span property="foaf:name">David Wood</span>
</div>
</div>

Written using a Callimachus HTML template it might look like the snippet below. Here is an embedded query asking who knows "david" and what is their name.

<div about="?who">
<span property="foaf:name" />
knows
<div rel="foaf:knows" resource="david">
<span property="foaf:name" />
</div>
</div>

Callimachus provides the framework necessary to create HTML templates to query, view, edit, and delete resources. This technique allows Web developers to save time and maintenance costs by applying the DRY principle (Don't Repeat Yourself) to Web application development.

For more information about Callimathus see http://callimachusproject.org or turn into my live Webcast on Wednesday at http://www.wilshireconferences.com/semtech2010/email/email-webcast-091510.html

The Future of RDF

2010-05-27T06:36:00.000-07:00

At the end of June, immediately after SemTech, I'll be attending the W3C RDF Next Step Workshop. This workshop has been set up with the goal of gathering feedback from the Semantic Web community to determine if (and how) RDF should evolve in the future. I'll be presenting two papers with David Wood which I hope will generate good discussions...(To review the papers or for more information on the workshop, go to NextStepWorkshop.)

The first paper I'm presenting will show a new RESTful RDF Store API supporting named queries and change isolation. (I blogged about this earlier this year.) This proposed API would combine basic CRUD operations over RDF constructs (graphs, services and queries) and mandate RDF descriptions of services. With the ability to modify an RDF store's state in SPARQL 1.1 comes the challenge of managing store versions and the need to manage them (and their differences) over HTTP.

The other paper is a proposed alternative handling of rdf:List in SPARQL. The way we currently deal with ordered collections in RDF, whether through tools or in SPARQL, is so difficult that it limits adoption of RDF. So much of data retrieval, which is currently dominated on the Web by XML, includes the notion of ordered collections - RDF must align the RDF representation with the conceptual notion of ordered collections if it has a chance of making inroads into already established networks.

Where do you think RDF needs to go in the future? Does it need to change if it is going to stay viable?

Reinventing RDF Lists

2010-03-08T09:06:00.000-08:00

Last month the SW interest group discussed alternatives to containers and collections as part of a discussion around what the next generation of RDF might look like. Below is my opinion on the matter.

RDF's simplistic approach makes it possible to encode most data structures, both simple and complex. The challenge people have with RDF, coming from other Web formats, is the lack of basic ordered collections (a concept common in XML). In RDF you are forced into a linked list structure just to preserve resource order. The linked list structure known as rdf:List is difficult to work with and highly ineffective within modern RDF stores.

Most RDF formats provide syntactic sugar to make it easier to write rdf:Lists. In turtle this is done using round brackets (parentheses); in RDF/XML this is done using the parseType collection attribute. However, because rdf:List is not a fundamental concept in RDF, no RDF store implementation preserves them, instead opting to use the fundamental triple form -- a linked list.

RDF is made of the following fundamental concepts: URI, Literal, and Blank Node. A fundamental list concept should be added to make it easier and more efficient to work with ordered collections. This would not have a significant effect on RDF formats, as their syntax would not change, but would have a significant impact on the mindset of RDF implementers.

With this change RDF implementers would strive to ensure that lists are implemented efficiently and provide convenient operations on them, just as they would other fundamental RDF concepts. The triple (linked list) form should be kept for compatibility with RDF systems that don't preserve lists, but the goal would be that RDF systems would not be obligated to provide a triple linked list form that has proven to be ineffective.

By making lists a fundamental RDF concept, there is no required change for RDF libraries to continue to be compatible with existing standards. Most libraries and systems may already understand list short hand and some may also preserve it.

Improving RDF/XML Interoperability

2010-03-01T07:06:00.000-08:00

All the permitted variations in RDF/XML make working with it using XML tools difficult, at best. Most of the time assumptions are made about the structure of the RDF/XML document. These are based on particular RDF/XML implementations. However, there is no standard spec that says what this simplified structure should be. The next generation of RDF specs should correct this and create a subset of RDF/XML for working with XML tools.

A good place to start is with the document Leo Sauermann has started. I like the design, but feel the rules could be improved, based on my experience.

The design of SimpleRdfXml, as proposed by Leo, is:
1. be compatible with RDF/XML
2. but only a subset
3. restrict to simplicity

The rules I try to follow when serializing RDF/XML for use with XSLT are:

1. No nested elements (references to resources must be done via rdf:resource or rdf:nodeID).
2. No property attributes.
3. All blank nodes identified by rdf:nodeID.
4. Only full URIs, no relative URIs.
5. No type node elements, always as rdf:type element.
6. The same rdf:about maybe repeated on multiple nodes.
7. Never use rdf:ID.
8. Always use rdf:li when possible.
9. Always use rdf:parseType="Collection" when possible.
10. All rdf:XMLLiterals written as rdf:parseType="Literal".
11. Never use rdf:parseType="Resource".
12. White-space is preserved within literal tags.

By standardizing on these (or another RDF/XML subset), interoperability between XML and RDF tools becomes possible. This allows existing shops to reuse their current XML skills to work with RDF, easing their transition.

Beyond the SPARQL Protocol

2010-02-17T04:34:00.000-08:00

The SPARQL Protocol has done a lot to bring different RDF stores together and make interoperability possible. However, the SPARQL Protocol does not encompass all operations that are typical of an RDF store. Below are some ideas, that would extend the protocol enough that it could become a general protocol for RDF store interoperability.

One common complaint is the lack of direct support for graphs. This is partly addressed in the upcoming SPARQL 1.1, which includes support for GET/PUT/POST/DELETE on named graphs. However, it is still missing the ability to manage these graphs. What is still needed is a way to assign a graph name to a set of triples as well as a vocabulary to search and describe the available graphs. The service could accepted a POST request of triples and responded with the created named graph to support construction. The graph metadata could be available in a separate service or as part of the existing SPARQL service, made available via SPARQL queries.

The use of POST in SPARQL ensures serializability of client operations. However, it prevents HTTP caching (with reasonably sized queries), which is necessary for Web scalability. This can be rectified by introducing standard named query support. By providing the client with the ability to create and manage server side queries (with variable bindings), many common operations can become cachable. These named queries can be described in their own service or as part of the existing SPARQL service. The named query metadata would include optional variable bindings and cache control settings. The queries could then be evaluated on HTTP GET to the URI of the query name, using the configured cache control, enabling Web scalability.

Another requirement for broad RDF store deployments is the ability to isolate changes. Many changes are directly dependent on a particular state of the store and cannot be represented in a update statement. Although SPARQL 1.1 allows update statements to be dependent on a graph pattern, many changes have indirect relationships to the store state and cannot be related directly within a WHERE clause.

To accommodate this form of isolation, separate service endpoints are needed to track the observed store state and the triples inserted/deleted. Metadata about the various available endpoints could be discoverable within each service (or through a dedicated service). This metadata could include such information as the parent service (if applicable) and the isolation level used within the endpoint.

To support serializable isolation, each endpoint would need to watch for Content-Location: headers, which would indicate the source of the update statement in the POST requests. When such a update occurs, the service must validate that the observed store state in the source endpoint is the same as the store state in the target endpoint before proceeding.

By standardizing graph, query, and isolation vocabularies within the SPARQL protocol, RDF stores would be much more appealing to a broader market.

RDFa Change Sets

2010-02-09T07:45:00.000-08:00

With so many sophisticated applications on the Web, the key/value HTML form seems overly simplistic for today's Web applications. The browser is increasingly being used to manipulate complex resources and an increasingly popular technique for encoding sophisticated data in HTML is RDFa.

RDFa defines a method of encoding data within the DOM of an HTML page using attributes. This allows complex data resources to be connected to the visual aspects that are used to represent them. RDFa provides a standard way to convert an HTML DOM structure into RDF data for further processing.

Instead of encoding your data in a key/value form, encode your data in RDFa and use DHTML and AJAX to manipulate the DOM structure and in turn manipulate the data. The conversion from HTML to data can be done on the server or client using existing libraries.

There are a few ways that RDFa can help with communication to the server. The simplest would be to send back the entire HTML DOM for RDFa parsing on the server. However, an HTML page might contain an excessive amount of bulk and therefore this would not be appropriate as a general solution. Instead, using an RDFa parser on the client, the resulting RDF data can be sent to the server, ensuring only the data is transmitted back. This would reduce excessive network traffic and move some of the processing to the client.

In a recent project, we went further and used rdfquery to parse before and after snapshots on the client to prepare a change-set for submission back to the server. In JavaScript, the client prepared an RDF graph of removed relationships and properties and an RDF graph of added relationships and properties. These two graphs represent a change-set. By using change-sets throughout the stack, enforcing authorization rules and tracking provenance became much more straight-forward. Change-sets also gave more control over the transaction isolation level, by enabling the possibility of merging (non-conflicting) change-sets. Creating change-sets at the source (on the client) eliminated the need to load/compare all properties on the server, making the process more efficient and less fragile.

RDFa on the client and submitting change-sets can help streamline data processing and manipulation and avoid much of the boilerplate code associated with mapping data from one format to another.

Why isn't the Web Object-Oriented?

2009-11-02T09:00:00.000-08:00

A big part of the Web is web services, but often these services are not modelled using an object oriented paradigm, even though it is well suited for complex behaviours. Web services are often modelled using a simple request/response paradigm or a service oriented paradigm using a RESTful framework, but many of these resource oriented frameworks can be adapted to support some object oriented concepts.

Many people think of classes and methods when they think of Object-Oriented Programming (OOP). However, I like to think of OOP as message passing with class specialization. This is particularly helpful when designing Web services, which also use a message passing model. Even RESTful Web services use forms of message passing between nodes.

Consider the simple URL below. When followed a GET request is sent to a Google server. This can be thought of as sending Google's search object a message with the given search term parameter (using the Google network as the authority). The search object (in this case a proxy) responds with an HTML page with the search results.

   Object Authority
 _________|_________
/                   \
http://www.google.com/search?q=Why+isn%27t+the+Web+Object-Oriented%3F
\__________________________/  \_____________________________________/
             |                                  |
       Object Identity                       message

All HTTP requests can be thought of as messages being sent to remote objects. The request method, query parameters, headers, and body make up the message, and the request URI identifies the message's target object. The HTTP response is the message's return value.

However, OOP is more than simply message passing. A big part of OOP is the association of behaviour with data. The relationship between behaviour and data drives at the difference between service oriented and object oriented paradigms. A service oriented model is like an object oriented model, but all objects are stateless singletons with their own unique behaviour. Because of this, pure service oriented systems can be more efficient (less data access), but is more expensive to maintain, as each service must consider all possible variations at once. In contrast, OOP supports behaviour specialization and can more closely reflect the structure of systems 'in the real world'.

While many services are identified by a single request URI (scheme+authority+path), most RESTful frameworks allow data to also be associated with the URI. JAX-RS, for example, allows path parameters that are often populated with a unique entity ID. By incorporating the entity ID in the URI, data is associated with the behaviour in the same way as in an OOP paradigm. However, most RESTful frameworks fail to provide any support for object or resource behaviour specialization -- a feature that is incredibly powerful in class-based OOP.

The Web is actually fairly close to seamlessly supporting an object-oriented paradigm. Processing efficiency seems to be the only barrier. However, with the growing costs of maintaining complex Web systems, I'm not sure how long this argument can hold up. When do you think we'll have an object oriented Web framework and what would it look it?

The Complicated Software Stack

2009-10-26T07:23:00.000-07:00

To aspiring Web application developers or people looking to put together their own Web application: the road to building a modern working Web application is a long and complicated journey.

Today's Web application developer is nothing short of a jack-of-all-trates, requiring deep knowledge of everything from HTML and CSS to Java and SQL. Everything from common CRUD tasks to sophisticated work-flows requires knowledge of half a dozen computer languages along with their quirks and variations across platforms and applications.

Today's software is built using a mix of programming paradigms and data models. Every level in the software stack requires explicit data mapping between paradigms. Many Web applications include the following levels in their software stack:
• Relational for persistence,
• Object oriented (class-based) in the model,
• Aspects peppered throughout,
• Resource (or activity) oriented Web services,
• Functional template engines,
• Markup using key/value pairs, and
• Prototype based objects for UI behaviour.

The above complication comes at a price. Software takes longer to develop and is more expensive to maintain than it used to be. This is causing a greater divide between small tools and large software systems.

Applications, like Microsoft Excel, which combine data processing and persistence using a consistent programming paradigm, have grown in popularity as a cheap alternatives to the complexity of modern Web applications.

While the market for Web applications has grown, the scope has decreased, favouring large high volume systems. Smaller Web applications are too often over-architected and over-budget. There is a large (and growing) opportunity for software vendors to fill this divide and create a new platform that combines data processing and persistence, using a single programming paradigm, for Web applications.

Can Web applications be built to use a single programming paradigm?

Chrome Frame: Love It Or Hate It

2009-09-29T17:18:00.000-07:00

Image via Wikipedia

Google has clearly struck a nerve among browser makers with the announcement of Chrome Frame. Microsoft was awfully quick to down play any thoughts about installing Chrome as a plugin for IE considering it refers to the WebKit's market share as a "rounding error". Mozilla has also recently become vocal about putting down any notion of a browser-in-a-browser solution. This is all quite bizarre as both of these players are big into browser plugins of some form or another. Microsoft with its alternative Silverlight application engine and Mozilla, which acquired its market share through extensible plugins of its own.

It is actually quite common to have multiple rendering engines within the same browser: flash, silverlight, and Java being the most obvious, but there is more. IE has had a number of browser plugins in the past, including Mozilla ActiveX Control and Google's SVG plugin. IE8 ships with multiple rendering engines that get triggered based on HTML tags or user actions. Nescape 7, although short lived, shipped with both the Gecko and IE rendering engines. Mozilla has previously encouraged this type of action in the past, with Google's ExCanvas and Mozilla's, now inactive, Screaming Monkey initiative. Today Mozilla still makes IE available as a Firefox plugin.

I think it is ridiculous to ask users to only use particular browsers for particular websites. Choosing the best available rendering engine should be the choice of the website authors and I would welcome a mega-browser that seamlessly switches between Gecko, Trident, WebKit, and Presto based on the preferred engine of the author. More precisely, I trust website authors will choose standard compliant engines more then I trust users to choose standard compliant browsers.

I find Mozilla's reaction particularly interesting as it comes at a time when I find myself, an old Gecko fan, looking at WebKit more seriously. Recently in a project, due to an old outstanding Gecko issue, I had to put Firefox support on hold while Trident, Presto and WebKit continued to operate without much trouble.

I know it is true with IE, but perhaps it is true with Mozilla as well, that they view the engine as just something a browser needs and not a feature in and of itself. Perhaps I have been wrong all along and XUL is actually Mozilla's doom.

Accept Headers: In The Wild

2009-09-20T09:30:00.000-07:00

As web agents (including browsers) become more diverse there is an increasing need to distinguish between their types. The User-Agent header can be used for this task, but requires the server to know in advance all the possible agents and what type they are. This is not possible as both the diversity and quantity of agents is growing too quickly for any single registry to track.

According to the HTTP specification, the Accept header can be used to determine the type of agent. For example:
• HTML browsers should include "text/html" within the Accept header,
• XHTML browsers include "application/html+xml",
• RDF browsers include "application/rdf+xml",
• XSLT agents include "application/xml",
• PDF agents include "application/pdf",
• Office suites include "application/x-ms-application" or "application/vnd.oasis.opendocument", and
• JavaScript libraries include "application/json"

This allows the server to better redirect the agent to an appropriate resource.

Obviously, if a service will only serve HTML browsers, the type of agent is not necessary, as is the case in the Web 1.0 days when everything on the Web was HTML. However, as HTTP is becoming a more popular protocol for non-HTML communication, the need for distinguishing between types of agents is becoming important.

Consider the situation when an abstract information resource (like an order or an account) is identified by a URL. When the server receives a request for an abstract information resource, it needs to know which type of agent is requesting it, so it can better redirect the agent to an appropriate representation. If the agent is an HTML browser, the server should redirect to an html page displaying the order or account information; if a JavaScript library, the server should redirect to a json dump of the order/account summary; if a PDF agent, the server should redirect to a order/account summary report; if an office suite, the server should redirect to a spreadsheet of the details.

This works very well in theory, but because the Web was built with only HTML browsers in mind, most browsers don't properly implement the HTTP specification (because they don't have to). Even worse is that most non-HTML browser agents either don't include an Accept Header at all or use */* and say nothing about the type of agent. Below are some of the default accept headers from popular user agents on the web.

FF3.5 is an HTML and XHTML browser first, XML/XSLT agent second
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

IE8 is a media viewer (apparently)
image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, */*

IE8+office is a media viewer and office suite
image/gif, image/jpeg, image/pjpeg, application/x-ms-application,
application/vnd.ms-xpsdocument, application/xaml+xml,
application/x-ms-xbap, application/x-shockwave-flash,
application/x-silverlight-2-b2, application/x-silverlight,
application/vnd.ms-excel, application/vnd.ms-powerpoint,
application/msword, */*

Chrome3 is an XHTML and XML/XSLT agent first, HTML browser second, and text viewer third.
application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Safari3 is an XHTML and XML/XSLT agent first, HTML browser second, and text viewer third.
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Opera10 is an HTML and XHTML browser first, XML/XSLT agent second.
text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1

The MSN bot is an HTML browser, text viewer, xml client and application archiver.
text/html, text/plain, text/xml, application/*, Model/vnd.dwf, drawing/x-dwf

Google search bot is a jack of all agents, master of none
*/*

Yahoo search bot is a jack of all agents, master of none
*/*

AppleSyndication is a jack of all agents, master of none
*/*

See Also:
Unacceptable Browser HTTP Accept Headers (Yes, You Safari and Internet Explorer)
WebKit Team Admits Error, Downplays Importance, Re: 'Unacceptable Browser HTTP Accept Headers'

Dereferencable Identifiers

2009-08-24T06:58:00.000-07:00

A document URL is a dereferencable document identifier. We use URLs all over the Web to identify HTML pages and other web resources. When you can't give out a brochure you can share a URL. Instead of sending a large email attachment, you might just send a URL instead. Rather then creating long appendixes, you can simply link to other resources. It is so much more useful to pass around URLs then it is trying to transfer entire documents around.

This model has worked well for document and is now being adopted for other type of resources. With the popularity of XML, using URLs to identify data resources is now commonplace. Rather then passing around a complete record, agents pass around an identifier that can be used to lookup the record later. By using a URL as the identifier these agents don't need to be tied to any single dataset and are much more reusable.

From the HTML5 standardization process has risen the debate on the usefulness of URLs as model identifier. Most people agree that a URL is a good way to identify documents, web resources and data resources. However, the debate continues on the usefulness of using a URL as an identifier within a model vocabulary. One side claims that a model vocabulary should be centralized and therefore does not require the flexibility of a URL. The other side claims the model vocabulary should be extensible and requires a universal identifying scheme that URLs provide.

To understand the potential usefulness of using a URL as a model identifier, consider the behaviour difference between a missing DTD and a missing Java class. A DTD is identified using a URL and a Java class is not. When an XML validator encounters a DTD it does not understand it dereferences the identifier and uses the resulting model to process the XML document. When a JVM encounters a Java class it does not understand it throws an exception, often terminating the entire process. Now consider how much easier it would be to program if a programming environment used URLs for classes and model versions. Dependency management would become as simple as managing import statements. As the Web becomes the preferred programming environment of the future, we must consider these basic programming concerns.

Although I enjoy working in abstractions, I certainly understand how things always get more complicated when you go meta: using URLs to describes other URLs. However, this complexity is essential to continue to maintain the flexibility and extensibility of the Web.

See Also: HTML5/RDFa Arguments

97 Things Every Project Manager Should Know

2009-08-23T11:47:00.000-07:00

If the projects you manage don't go as smoothly as you'd like, 97 Things Every Project Manager Should Know offers knowledge that's priceless, gained through years of trial and error. This illuminating book contains 97 short and extremely practical tips -- whether you're dealing with software or non-IT projects -- from some of the world's most experienced project managers and software developers. You'll learn how they've dealt with everything from managing teams to handling project stakeholders to runaway meetings and more.

This is O'Reilly's second book in its 97 Things series. My contributions included tips to Provide Regular Time to Focus and Work in Cycles.

SPARQL Federation and Quints

2009-07-31T08:47:00.000-07:00

There are currently a couple popular way to federate sparql endpoints together:

1) In Jena the service must be explicitly part of the query, and therefor the model,

2) In Sesame the basic query patterns must be associated with one or more endpoints before evaluating the query, or

3) Hack the remote query into a graph URI: http://gearon.blogspot.com/2009/05/federated-queries-long-time-ago-tks.html

Although both can be used to achieve the same results, Jena's solution puts more responsibility in the data model, and Sesame's put more responsibility in the deployment. Both have their trade offs, but I believe the query is suppose to be abstracted away from underlying services. The domain model (and therefore the queries) should not be aware of how the data is distributed (or stored) across a network. Therefore, I prefer to describe which graph patterns and relationships are available at each endpoint during deployment and make the application model independent of available service endpoints.

Furthermore, I think it is a bit silly to add yet another level of complexity to the basic query pattern. Adding the service level turns the basic query pattern from a quad to a quint.

To fully index a quint (with support for a service variable, which Jena does not support) would take 13 indexes (nearly double what a quad requires). Below is a table of some complexity levels and how many indexes they require to be fully indexed (variables could appear in any position within the pattern). I have included a theoretical sext that would allow you to group services in a network (just as graphs can be grouped in a service).

Level	#ofIdx	Term	Data Structure
double	2	subject	directed graph
triple	3	predicate	labelled directed graph
quad	7	graph	multiple labelled directed graphs
quint	13	service	replicated multiple labelled directed graphs
sext	25	network	trusted replicated multiple labelled directed graphs

Switching from triples to quad provides a big functionality leap (the ability to refer to an entire graph as a single resource). However, I question how much functionality a quint (or a sext) has over a quad. Couldn't the same functionality be put into a property of the graph (or embedded in the graph's URI authority). An inferencing engine/query could also conclude graph relationships like (subGraphOf), which would still allow a large, but precise, collection of graphs to be queried more effectively.

Hopefully, this topic will have more time to mature before the SPARQL working group makes any official decisions on the matter.

Enterprise Information Systems and Web Technologies

2009-07-22T05:34:00.000-07:00

I recently got back from speaking at the Enterprise Information Systems and Web Technologies conference in Orlando Florida. There I presented my paper on an Object-Oriented rules engine. In the talk I shared examples of when businesses need to coordinate, track data, and policy check between organizations. Such as in transportation, satellite data tracking, and contract management. I outlined the following requirements and went into detail on the various components of the system.

Requirements
Reduce the investment costs and time
Policies understandable by domain experts
Rules must not inadvertently interfer with one another
Model complex domains
Easily adapted to change
Policy rules have access to external services
Track all state changes both their cause and effect

The talk was well received and posed some interesting discussions.

Panel: Linked Open Data

2009-07-08T13:45:00.000-07:00

The SemTech 2009 Videos have been posted, including the Linked Open Data Panel.

The "data commons" is a cornerstone of the semantic web vision. The Linked and Open Data movements are progressing beyond the early adopter phase and preparing to cross the chasm. Enough experience now exists to reflect on how this data set is being used, how useful it is, and where we can take it from here. Beyond the basics, the panel will discuss issues such as quality of service, stability, and longevity. They'll also explore the evolution of the semantic web with a particular emphasis on modes of data use, reuse and aggregation.

Paul Miller, The Cloud of Data
Jamie Taylor, Metaweb Technologies, Inc.
Leigh Dodds, Talis
James Leigh, James Leigh Services, Inc.
Kingsley Idehen, OpenLink Software, Inc.

http://www.semanticuniverse.com/semtech-panel-linked-open-data.html

HTTP Servlet Caching Filter

2009-06-30T09:19:00.001-07:00

The nice thing about the HTTP protocol is how easy it is to implement an trivial HTTP server. At some point, however, just responding to HTTP requests is not enough and response caching must be introduced.

If you search for "servlet response caching", you will find advice to ensure you use the correct response headers to facilitate HTTP caching and suggestions to use a servlet filter to cache the response. If you are like me, you would continue to search for a way to use both - a servlet filter that caches based on the correct response headers.

With the HTTP protocol so well supported and J2EE so popular, finding a caching servlet filter that adheres to the HTTP spec should be easy, but it isn't. In fact, it is really hard to find any Java implementations that caches based on the HTTP response headers (servlet filter or otherwise). This seemed like an interesting problem that is fairly common, so I spent some time to see how far I could get with a servlet filter that understands HTTP caching.

Despite my general knowledge of the HTTP spec, implementing it is a lot more difficult. For example, the If-Modified-Since and If-None-Match headers are fairly easy to understand, but when you try and implement this logic, things get a little more complicated. In working through this, I realized that there are nearly 20 possible scenarios that need to be handled by the server. The request might not have an If-Modified-Since header, it might have been modified, and it might not have been modified are three states the server must handle for that header. The If-None-Match may not be present or it might or might not match and it might have a '*' tag, which may not may not have an existing entity. You can't just process these one at a time either, but once you write down the edge cases it can all be handled fairly compactly within a precondition check.

Another area that surprised me was the request Cache-Control directives. There are five boolean directives and three with a specified value. All of these directives are fairly easy to understand and are used to determine if the cache can be used and if it needs to be validated. However, that is a lot of variables to manage and combined with the possible server directives, its gets really harry tracking their state. There were many occasions when adding support for a new header/directive that I inadvertently broke an earlier unit test (couldn't have done it without them).

The HTTP spec is fairly clear is some areas, but less so in others. One area that has had various interpretations is entity tags. It is fairly clear how ETags should be used with static GET requests, although I had to digest their implications on caches before I could understand how to use them with content negotiation. However, their recommended use with PUT and DELETE is still a bit of a mystery. When an entity has no fixed serialized format (such as a data record), it has many entity tags (one for each serialized variation and version). So, which entity tag should be used after a PUT or a DELETE that effects all variations?

This get even more complicated when some URLs are used to represent a property of a data record. If the property is a foreign key, the response has no serializable format, its a 303 See Also response. What does a PUT look like when you want to reference another resource? Furthermore, a DELETE of a property, just deletes the property, but the data record still exists and there is still a version associated with it, shouldn't the client be given the new version?

In the end I have a new appreciation for why there are so many interpretations of the HTTP spec and I have a fairly general purpose HTTP caching servlet filter on top of it.

Resource Oriented Framework

2009-06-17T16:45:00.000-07:00

What happens when you put an Object oriented Rules Engine in a Resource Oriented Framework? After three years of research, I think I have found the answer and have released it as AliBaba.

AliBaba is separated into three primary modules. The Object Repository provides the Object Oriented Rules Engine. It is based on the Elmo codebase that has been in active development for the past four years. The Metadata Server is a Resource Oriented Framework built around the Object Repository. Finally, the Federation SAIL gives AliBaba more scalability.

In AliBaba, every resource is identified by a URL and can be manipulated through common REST operations (GET, PUT, DELETE). Each resource also has one or more types that enable it to take on Object Oriented features that can be defined in Java or OWL. Each object's properties and methods can be exposed with annotations as HTTP methods or operations. Operations are used with GET, PUT and DELETE HTTP methods by suffixing the URL with a '?' and the operation name. These operations are commonly used for object properties, while object methods are commonly exposed as other HTTP methods (POST) or as GET operations. This HTTP transparency allows the Metadata Server's API to hide within the HTTP protocol and not dictate the protocol used - allowing it to implement many existing RESTful protocols.

AliBaba provides a unique combination of Object Oriented Programming and a Rules Engine, available in a Resource oriented Framework. I believe it combines some of the most promising design paradigms commonly used in Web applications. Its potential to minimize software maintenance costs and maximize productivity by combining these paradigms is very exciting.

Intersecting Mixins

2009-06-08T08:14:00.000-07:00

As software models become increasingly complex, designers seek additional ways to express their domain models in a form that more closely matches their design concepts. One way this is done is through Mixins.

A Mixin is a reusable set of class members that can be applied to more then one class. It is similar to inheritance, but does not interrupt the existing hierarchy.

Suppose we have a class called "Invoice" within our domain model and we want to "enhance" this class with operations to fax, email, and snail-mail it to the customer. To prevent the reduction of Invoice's cohesion, we want to define this behaviour in a separate construct. We could subclass Invoice, but the ability to send a document is common among other classes as well. We could put this logic in a super class, but that only works if there is an appropriate common super class among them. An alternative is to create mixins, called Faxable, Emailable, and Mailable, that are added to all the classes that can be sent.

Suppose some of our documents require a unique header. If this behaviour is common, but no appropriate super exists, a mixin would be a desirable choice. Mixins allow classes to be extended with new behaviour, but what if you want to alter existing behaviour? Unfortunately, many mixin implementations do not allow calls to the overridden method, and the ones that do require it to be done procedurally (by changing an open class at runtime).

When using inheritance, a subclass can call the overridden method to intersect and alter the existing behaviour, but a mixin does not inherit the behaviour of its targets and there are possibly multiple mixins wanting to alter the same behaviour, so there is no single "super member", but one for every mixin that implements it.

Most languages allow a mixin to override the target's behaviour, but don't allow it to be intercepted. Some languages, like Ruby and Python, allow the target class to be altered by renaming and replacing members. This allows the programmer to simulate an intersection, but is a much more complex way of handling it.

In AliBaba's Object Repository a mixin, also known as a behaviour, can declare precedence among other mixins and allows them to control how method execution proceeds. For example if a mixin has the annotation @precedes, it will be executed before any of the given mixin classes are executed. By declaring the method with a @parameterTypes annotation, with the overridden method parameter types, and a Message as the method parameter, the mixin can call msg.proceed() to execute other behaviours and retrieve their result. This allows mixins to call the overridden methods and provides intersecting other methods.

By extending the basic mixin construct to allow them to co-exist and interact, a mixin can be used to address other aspect oriented problems in an OO way.

Time to Rethink MVC on the Web

2009-06-04T11:35:00.001-07:00

Typical web applications don't do a very good job at separating the model, view, and controller logic. The view is particularly confusing as logic is split between server side and client side. This is a side effect from trying to support "dumb" (or nearly dumb) clients (browsers that only support static HTML). Furthermore, logic that would be more appropriate in the model, often gets put into the controller to avoid unnecessary access to the database.

While these design decisions lead to faster web application servers, they also lead to what I call coincidence coding - an unwanted code separation that has no documentation or declared interface. The worst part is that this separation often happens in the most visible part of the system: over HTTP and SQL. This prevents the system from becoming a block box utility, because the internals rear their ugly heads. Any attempt to standardize the protocol (or to use well documented services) is impossible because the protocols get stuck between various components of the view or model and cannot be separated cleanly and maintained across versions.

It is not all doom and gloom: the web is a different place then it was when many of these web frameworks were designed. I am pleased to report that all modern desktop web browsers have good support for client side templating using XSLT (finally!). This means web applications today don't need to support these dumb clients. Furthermore, web proxies have also been improving enough that it is now common for production deployments to plan on use a proxy server in front of the web server(s). Accessing the persisted data is not as much of a problem as it used to be, as database caching is usually planned when systems are being designed.

With these (somewhat) recent developments, its time for a new breed of web frameworks to emerge. These frameworks could implement MVC on the web like we have never seem before - really separating the model from the view and from the controller. This could have a significant impact on the maintainability of web application software.

Yesterday, I introduced what I believe to be the first web application framework that does a decent job of separating MVC. It is called the AliBaba Metadata Server and can be downloaded from OpenRDF.org.

The view logic can only exist in static HTML, JS, CSS, and XSLT files. The model/data is transformed into standard data formats, like RDF and some forms of JSON without model specific transformations. You can't put any model logic into the controller, and all service logic must be put into the model. Stand alone, it would be slower than most web application servers, but with proxy and caching on both ends, I believe it will perform just as well and, most importantly, yield more maintainable web applications.

Standard RDF Protocol

2009-06-01T09:08:00.000-07:00

The SPARQL Working Group's first meeting has come and gone. Part of that meeting discussed the scope of what should be undertaken as part of SPARQL2, including what features should be added to the protocol. SPARQL is a standard query language and protocol.

I am a big fan of standards, as most people are in the Semantic Web community. However, I feel that a database protocol does not provide much value and in fact, should be tailored to the evaluation and storage mechanism used by the implementation.

Standards are intended to be interoperable between implementations, and SPARQL must abstract away from the storage mechanism used in order to achieve the desired level of interoperability. This leads SPARQL to result in a less than efficient query/retrieval operation, when compared to storage specific mechanisms like SQL. However, this still has significant advantages, as a single query can be used with a wide array of storage mechanisms and remains unchanged between significant storage alterations. This does come at a cost to both the ability to create efficient queries and the ability to evaluate queries effectively using the SPARQL protocol.

The cost of query parsing and optimization can always be improved and can be tailored to specific models, without any loss of interoperability at the query language level. However, a standard protocol cannot be optimized the same way.

When a database backed application has performance problems, 9 out of 10 times it is due to excessive communication between the database and its clients. Therefore, I question the value in abstracting this communication protocol away from the storage/evaluation mechanism, because this will exacerbate communication overhead.

While I think the SPARQL query language is developing nicely and should be considered in any project that wants to ease the maintainability of their queries, I also think any project that is concerned about the performance of their queries should consider using a proprietary protocol to optimize its communication with a database.

Speaking at SemTech

2009-05-25T08:00:00.000-07:00

I will be speaking at the Semantic Technology Conference in San Jose again this year from June 14-18, 2009.

"Modelling Objects in RDF" talk will be given for attendees that want to learn how to get started quickly building applications that use an RDF store.

Paul Gearon and I will compare and contrast the features and structure of modern RDF stores to help you find the stores that works best for your environment and data.

I will be discussing "Linked Open Data" in a panel to give you a chance to ask questions and get answers about how to share and use interconnected data across the web.

Thomas Tague and I will demonstrate how you can you use OpenCalais (at no cost) to extract metadata and named entities into the Linked Data cloud from english text.

As a speaker, I am authorized to share registration discounts up to $200. If have not registered yet and are interested in attending, please contact me.

Toronto College of Technology TechTalk

2009-05-21T13:36:00.000-07:00

I will be giving my "Modelling Objects in RDF" talk again at TCT next week on May 30th. If you missed the last JUG meeting and are interested in learning/talking about RDF, come by for the talk.

More details can be found here:
http://www.torontocollege.com/Display.action

A Map can be found here:
http://www.torontocollege.com/BI/location.jsp

Annotation Properties

2009-05-21T10:40:00.001-07:00

In Java annotation properties are grouped in annotation classes, which are grouped in packages. Since Java does not consider properties a top level construct (they must be in a class), this organization makes sense for Java. However, often burying the property this deep can make the syntax hard to read.

Compare the annotation syntax of JAX-RS vs Spring's annotation-based controller configuration. Spring makes full use of the annotation class/property grouping and can often look confusing as many annotation properties are crammed into the same declaration. Whereas JAX-RS tries to collapse annotation classes and properties together, allowing the annotation declaration to be simplified.

Java allows annotation properties to be omitted if they have the name "value". This allows the coder to simply state the annotation class and the property value (omitting the annotation property).

I argue that when using the short syntax, the annotation class (conceptually) becomes the annotation property and therefore should start with a lower case. This is how JavaDoc annotations have always be done (start with a lower case letter) and I think it should be carried over to Java annotations as well.

Conceptually, annotation properties are akin to static final class properties and therefore should be easy distinguished from traditional classes. Creating lower case annotations class files is one easy way to do this.

What do you think? Is it okay to create a Java file that starts with a lower case letter? If so, under what conditions?

Is Google the database of the Web?

2009-05-14T09:50:00.000-07:00

Yesterday Google announced plans to improve its user's searching experience by providing more ways to search and find information on the Web. With Google's new search options and rich snippets, it is putting it self in a position to not simply link to desired pages, but actually provide the information directly.

By adopting RDFa, Google hopes to make it possible to start importing the deeper web into its cluster. It is starting by asking publishers of reviews, people, products, and organizations sites to share the raw data directly, so it can server the information without requiring its users to visit the publisher's site. Although Google is not the first search engine to use RDFa in the results, it is the first to start using it across all sites.

I am happy to finally see Google start to include rich semantics in its interface, but I worry what this might mean to publishers. Will Google become the database of the Web? Will the Web surfers of tomorrow never leave Google's domain? Time will tell, but lets hope this triggers new discussions on how we "browse" the growing web of data.

Toronto JUG Meeting

2009-05-06T07:46:00.000-07:00

My presentation at yesterday's Toronto JUG meeting covered a lot of ground as it both introduced RDF and demonstrated how to use RDF with objects. Many of attendees struggled with the significant mind-shift from relational to RDF and how a data model can be represented as a graph of nodes.

One important point, that I want to reiterate, is that RDF stores are not designed as general purpose databases (as relational databases are), but instead RDF stores are designed for complex data structures. Currently, RDF is mostly used within industries that have complicated data models. However, RDF is becoming more appealing to a wider audience as data models in general are becoming more complicated and interconnected.

If you attended the meeting and/or are interested in talking more about RDF. Read through the getting started guide for Sesame[1] and join the discussions on IRC at irc://irc.freenode.net/sesame

[1] http://wiki.aduna-software.org/confluence/display/SESDOC/GettingStarted

SPARQL Highlighting in GEdit

2009-04-30T09:43:00.000-07:00

In searching for highlighting mode for N3 and gedit, the best I could fine was a blog post from a couple years ago[1]. When writing N3 I am overly cautious about typos, and I could use some indication of syntax errors. So I created a SPARQL syntax highlighting mode for gtksourceview. This will highlight all known xsd, rdf, rdfs, and owl classes, datatypes, and predicates to make creating ontologies in N3 easier.

The file can be downloaded[2] and saved as ~/.gnome2/gtksourceview-1.0/language-specs/sparql.lang. Then the SPARQL mode is available from the View->Highlight Mode->Sources menu.

[1] http://www.semikolon.co.uk/blog/index.php?entry=entry070510-102401
[2] http://bugzilla.gnome.org/attachment.cgi?id=134935&action=view

Speaking at Toronto's JUG

2009-04-28T05:15:00.000-07:00

I will be giving a talk entitled "Modelling Objects in RDF" next week at Toronto's Java User Group. If you are in the city, please come by and introduce yourself.

Today's most used database model is the relational model. However, in today's high performance web-centric world, the relational database has begun to show its age. This talk introduces the Resource Description Framework (RDF) as an alternative database model. RDF is a family of standards to model properties and relationships between resources in a web-centric way. Unlike the relational model, which stores strictly defined records, an RDF model stores semi-structured graphs. This allows RDF stores to model more complex relationships, scale better across the web, and tolerate schema variations for compatibility.

Object-oriented design is the preferred paradigm for modelling complex software; however, most RDF APIs are tuple-oriented and lack some of the fundamental concepts in object-oriented programming. This talk introduces Sesame and how it can be used with RDF resources using object-oriented designs.

Event details can be found here:

http://torontojug.org/

AtomPub as a Discovery Protocol

2009-04-23T05:54:00.000-07:00

AtomPub was designed as a blog API, but with all the advancement the browsers have made as an application platform, it hasn't achieved wide spread use among bloggers. However, that doesn't limit the usefulness of it as a general publishing protocol.

My colleagues at Zepheira and I have been putting AtomPub into action recently. In our latest project we used AtomPub for a marketplace service registry. This allows others to publish the existence of their web service into collections. Because of the thorough documentation of AtomPub, it was easy to get everybody on board. With a simple example everybody started seeing the advantages of using a standard protocol. Generally speaking the AtomPub server was a breeze to maintain as all operations could be done using curl.

In the same project we also used OpenSearch with Atom to integrate multiple (private) search engines into a unified result. With some sugar and spice added to the "self" links we created a very impressive search solution.

Although AtomPub hasn't achieved much popularity within the blogging world, there are lots of benefits in using a standard protocol and AtomPub has proven itself as a great discovery protocol to us.

Long Running Transactions

2009-03-31T13:24:00.000-07:00

I have been experimenting recently with long running transactions and am intrigued by its potential. The term 'long running transaction' is loaded with interpretations, so let me explain. Normally, when one thinks of a long running transaction, they think of a set of operations that can be undone. The implications of which are domain/implementation specific.

However, I am experimenting with atomic optimistic long running transactions - like a database transaction that spans multiple HTTP requests. The key requirement here is that it is optimistic. This means there are no resource locks used across the requests, allowing concurrent access and non-conflicting modification.

What makes this interesting, is when using a web interface to manipulate a complex structure. While other approaches would have the structure copied into session memory or track undo operations, this utilizes the indexes already present in the store (RDF in this case) and there is no need to track or even develop complicated undo behaviour. We are all familiar with the classic cancel/apply or restore/save buttons of local applications that span an application or set of dialogs. With optimistic long running transactions, this same experience can be used in a web application.

Eight Isolation Levels Every Web Developer Should Know

2009-03-19T05:25:00.000-07:00

The ACID properties are one of the cornerstones of database theory. ACID defines four properties that must be present if a database is considered reliable: Atomicity, Consistency, Isolation, and Durability. While all four properties are important, isolation in particular is interpreted with the most flexibility. Most databases provide a number of isolation levels to choose from, and many libraries today add additional layers which create even more fine-grained degrees of isolation. The main reason for this wide range of isolation levels is that relaxing isolation can often result in scalability and performance increases of several orders of magnitude.

Read More Here:
http://www.infoq.com/articles/eight-isolation-levels

RDF Federation

2009-03-09T07:13:00.000-07:00

An RDF store (unlike an rdbms store) usually indexes everything. This means that the store's indexes are often larger then the data that it contains. Such large indexes makes I/O expensive and caching difficult. Scaling vertically by adding more memory for caching and faster disks can make a big difference, but it can get very expensive very quickly.

The alternative is to scale horizontally. This can be done in one of two ways: by mirroring the indexes on other machines, or by partitioning the indexes to other machines. The first option, called clustering, can reduce the I/O load, but will still have difficult caching it. The second option, called federating, reduces the I/O load on each machine, and can allow each machine to specialize, making caching much more effective.

Federating RDF stores is now going to be a lot easier with Sesame 3.0. Sesame 3.0 will support federating multiple (distributed) Sesame Repositories into a unified store. This allows large indexes to be distributed on multiple machines that are connected over a network. The federation supports multiple ways of partitioning the data. It can be partitioned by predicate (property), by subject, or both. When properly setup the federation can effectively proxy queries to the specialized members and join queries among the distributed members. For large-scale RDF stores, federating is becoming a valuable solution for RDF architecture.

Instructions for setting up a read only Sesame Federation can be found here:
https://wiki.aduna-software.org/confluence/display/SESDOC/Federation

What is the Future of Database Systems?

2009-03-02T05:07:00.000-08:00

Last month saw a flurry of activity around the future of relational databases. Although no one can predict the future, I think it is safe to say that many system developers/architects are hungry for mainstream semi-structure databases.

Feb 12 Is the Relational Database Doomed?
Feb 13 Is The Relational Database Doomed?
Feb 13 CouchDB could be a viable alternative to relational databases for storing patient data
Feb 16 The future of RDBMS's
Feb 20 Is the Relational Database Not an Option in Cloud Computing?
Feb 27 How FriendFeed uses MySQL to store schema-less data

RDF Transaction Isolation

2009-02-26T12:32:00.000-08:00

Transaction Isolation in relational databases (for better or worse) is well established. However, the issue of transaction isolation is rarely documented in RDF stores.

The ANSI SQL isolation definitions are UPDATE (write) oriented and do not capture the general use case of RDF, which has no notion of UPDATE. For example, the first ANSI SQL phenomenon, dirty-write, is not even applicable to RDF transaction. Another phenomenon, non-repeatable reads, is defined by records retrieved by a SELECT statement. However, RDF queries (unlike SQL) are pattern based and the results don't have a direct relationship to any internal data record.

Relational database isolation mechanisms do not perform nearly as well when INSERT/DELETE operations are used instead of UPDATE. Furthermore, relational databases often have a lax definition of "serializable", allowing conflicting INSERT operations (assuming that preventing conflicting UPDATE operations is sufficient).

RDF is a different beast altogether. RDF is set oriented. Two RDF transactions adding or removing the same statement do not necessarily conflict with each other, as they would in SQL, because a successful add or remove operation in RDF does not require a state change.

Early RDF use cases required full serializable transaction as many of the inferencing rules used in RDF needed to take the complete store state into account. Because of this, RDF stores generally only provide full serializable transactions. However, full serializable transactions often do not perform as well as lower isolation levels.

RDF stores are now being used in environments that have a much greater, real-time demand, for fast concurrent write operations. These environments don't require full serialization, but currently lack any other isolation levels to choose from.

To address this need Sesame 3.0 introduces five isolation levels that will allow RDF stores to vary the isolation level provided. By providing different levels, significant performance improvements can be made for lower isolation levels. For example:

• Read Committed isolation level permits weak-consistency and allows proxies to cache repeated results without validation.
• Snapshot isolation level permits eventual-consistency and allows store clusters to maintain independent state and propagate the changes during idle periods.
• Serializable isolation provides a higher degree of isolation, but does not require atomic consistency, permitting concurrent transactions.

For more details on the isolation levels supported by Sesame 3.0 see:
http://wiki.aduna-software.org/confluence/display/SESDOC/TransactionIsolation

What variations of transaction isolation have you used in your application?

Sesame 3-alpha1

2009-02-24T12:49:00.001-08:00

The first preview of the new Sesame API is now available. Here is an article explaining the new features: http://www.devx.com/semantic/Article/40987

XHTML: What is it Good For?

2009-02-12T04:54:00.000-08:00

With IE8 nearly upon us, discussion on the future of web standards has once again been triggered. Again lacking in IE8 is support for XHTML, ensuring that IE is the only browser that doesn't support it.

The goal of XHTML is to allow XML technologies to be used with HTML. XHTML has been a standard now for about five years, and IE is still (single handedly?) preventing websites from adopting it. Instead websites are forced to use server side solutions that require more bandwidth and processing, while making it harder for non-traditional agents to participate.

Within recent discussions on XHTML many people seem to fail to understand the potential benefit XHTML has over HTML. An interesting example I came across was combining XSLT and XHTML together.

I have written about XSLT before
http://jamesrdf.blogspot.com/2008/12/mvc-with-xslt.html

This example is from 2002 (around the time XSLT and XHTML were standardized) and can be found in more detail here:
http://www.webreference.com/xml/resources/books/practicalxml/chapter5/

As most of us know, XHTML is also an XML file and as such can be used as the input or output of XSLT. What many do not realize (and what is not covered in the above link) is that XHTML can also be used as the stylesheet. XSLT supports a simplified syntax that allows XSLT tags to be embedded inside an XHTML template file, making the template look a lot more like other server-side templating engines.
http://www.w3.org/TR/xslt#result-element-stylesheet

This allows you to use XHTML for the template and the content, and it works in all browsers, except *one* of them. Actually, you can get this to work in IE (even as early as IE5), but you have to use the XML rendering mode.

The XML rendering mode requires that the pages return application/xml and no doctypes present. Unfortunately by using the XML rendering mode, no HTML-specific features are available: no cookies, no document.write and script tags are parsed differently.

TV Series.com is rendered in XML mode, for example: http://www.tvseries.com/

If IE would get around to implementing XHTML, I think a lot more websites could safely switch to serving static files and the Web would start to become a lot easier to work with. But that probably wouldn't be good for Silverlight.

Elmo to Get a Face Lift

2009-02-02T11:25:00.000-08:00

In 2008 Elmo's interface saw some adjustments to enable more efficient access across HTTP. The challenge with this new interface is to understand how the object operations map to RDF operations and what implications they have on performance over http.

Despite the changes, EntityManager-oriented interface is still inappropriate for RDF/Object mapping, because relational operations do not directly map to RDF operations. Further, object persistence abstraction causes more performance problems then it solves for seasoned developers. Unlike object-relation mapping, RDF-object mapping is much more natural.

Work has already begun on Elmo's successor, AliBaba Object Repository, providing an hybrid RDF/Object interface extension to Sesame. Performance over HTTP is a high priority and was the major inspiration to the new interface.

Early development has already shown significant performance increases for both read heavy and write heavy transactions. For those interested in getting involved, you can checkout the code at
http://repo.aduna-software.org/svn/org.openrdf/alibaba/trunk/

Perfection Is An Unrealistic Goal

2009-01-29T14:05:00.000-08:00

Linda Rising gave a presentation on a topic that is still fairly misunderstood. She tries to address the question "What is the best way for us to work as individuals?" She talks about not deceiving yourself and understanding that everything is a journey without a destination. It is better to build something that is known to be imperfect now, than wait for clarification, as more knowledge can be gain from a system that doesn't work than any theoretical discussions. Software development is like a series of experiments, a series of learning cycles. She then uses our regular sleep cycles to identify our optimal work cycles and emphasises the importance of taking breaks throughout the day to improve productivity.

http://www.infoq.com/news/2009/01/Perfection-Is-Unrealistic-Rising

Modelling Objects In RDF

2009-01-26T06:27:00.000-08:00

Also at The Semantic Technology Conference this year, I will again be speaking on persisting Objects in RDF. This is an introductory session on building object-oriented applications with the Sesame RDF repository. This year we will be going into more detail on the advantages of using RDF as a persistence layer. Including achieving forward and backward data compatibility, OWL's representation richness, longer running optimistic transactions, and using graphs for auditing purposes.

Unifying RDF Stores

2009-01-22T08:02:00.000-08:00

In June I will be speaking at The Semantic Technology Conference with Paul Gearon about some of the integration work we did with Sesame and Mulgara. We will also be comparing a hand full of existing RDF stores and demonstrate how they can be used interchangeably or as a unified federation.

The RDF storage market has become much more divers recently, with many providers tailoring to specific environments and data patterns. This talk will cover some considerations to help you identify what RDF store implementation is best for your data and environment. We will discuss common features found with many providers, unique features found in only a few, and demonstrate some of the new features in Mulgara. We will also be demonstrating a unified RDF API to allow RDF stores to be swapped into applications post-development and how using a provider independent API enables divers RDF storage nodes to be federated together, allowing each node to be tailored to the unique shape of the data being stored within.

Hope to see you there!

Validating RDF

2009-01-15T11:54:00.000-08:00

RDFS/OWL is criticized for its weak ability to validate documents in contrary to XML, which has many mature validation tools.

A common confusion in RDF is the rdfs:range/rdfs:domain properties. A property value can always be assumed to have the type of the rdfs:range value. This is very different to XML, which only has rules to validate tags, but cannot conclude anything. Many of the predicates in RDF are used for similar inferencing, but they lacks any way to validate or check if a statement really is true. This is a critical feature for data interchange, which RDF is otherwise well suited for.

To address this limitation, an RDF graph can be sorted and serialized into RDF/XML. With a little organization of statements, such as grouping by subject, and controlled serialization, common XML validation tools can be applied to a more formal RDF/XML document. Our validation was done with relatively small graphs and we restricted the use of BNodes to specific statements to ensure similarly structured data would produce similar XML documents.

Although TriX could also have been used (it is a more formal XML serialization of RDF), it was considered that the format produced would not be as easy to work with for validation tools.

With a controlled RDF/XML structure we were able to apply RNG to provide structure validation before accepting foreign data and able to automate the export into more controlled formats using XSLT. (We used a rule engine for state validation.) Although RDF is a great way to interchange data against an changing model, XML is still better over the last mile to restrict the vocabulary of the data accepted.

Did Google Just Expose Semantic Data in Search Results?

2009-01-12T06:01:00.000-08:00

Google has been hesitant, in the past, of employing semantic technology, citing trust issues and the lack of quality meta-data in The Web today. However, it would appear Google is warming to the idea of semantics in the area of natural language processing. I have written a couple articles on the subject previously, but it appears that Google is exposing their own text analysis, acording to this blog entry.

http://www.readwriteweb.com/archives/google_semantic_data.php

Building Super-Scalable Web Systems with REST

2009-01-08T06:42:00.000-08:00

I came across this blog posting that I thought was an interesting example of how the services should be oriented around the data and not the other way around.

http://www.udidahan.com/2008/12/29/building-super-scalable-web-systems-with-rest/

The idea here is that the data (in this case weather info) needs to be partitioned in a meaningful way (by location). REST services can then be created around this and utilize the caching available in HTTP.

By creating and organizing services around the domain's data model more efficient services can be created. This reinforces why SOA often ends up failing, because there is not enough emphasis on the data. It also highlights REST support for caching and cache validation built into the protocol. Other service/message specifications (like SOAP) would have more difficulty identify and implementing a caching mechanism.

Object Oriented Rule Engine

2008-12-29T07:19:00.000-08:00

Last week I discussed the desire for a rule engine to allow object oriented domain logic to be browsed, searched, reviewed, and altered outside of an IDE.
http://jamesrdf.blogspot.com/2008/12/object-oriented-rule-engine.html

Earlier this year Zepheira, LLC needed an object oriented rule engine to enable different participants to manage continually changing rules. After developing a solution with Drools, it was observed that there were too many restrictions on the accessible data and the rules were getting too complicated to be effective with multiple participants each with different interests. Together we combined the modelling language - OWL (Web Ontology Language) - with the Object Oriented language Groovy to create a rule engine that was effective over large datasets and included many of the features of OOP including the ability to override selected rules for particular sub-domains within the model.

We developed the model using OWL and extended it by adding the ability to define message types (LHS rule declarations) and method bodies (RHS rule execution). By separating the two and associating every rule with a concept class, we were able to apply OOP features to the rules. The syntax we used can now be seen here:
http://www.openrdf.org/doc/elmo/1.4/user-guide/x457.html#AEN466

By developing Groovy domain logic in an RDF database we enabled many stakeholders access to review and alter the rules to facilitate their own what-if scenarios. It also made it possible to manage much more complex situations by utilizing the strong vocabulary of OWL to describe the concepts that could not easily be represented in a simple object model.

We achieved the flexibility we needed and reduced the complexity of the rules by using an object oriented rule engine. The rule engine has since been licensed under the BSD licence as part of the OpenRDF Elmo project at http://www.openrdf.org/

About Zepheira
Zepheira is a US-based professional services firm with expertise in semantic technologies and Enterprise Data Integration. For more information, visit: www.zepheira.com.

Object Oriented Rule Engine

2008-12-22T07:16:00.001-08:00

A rule engine should be used when a domain model has many rules that may change over time and need to be closely managed by domain experts.

By using a rule engine, rules become much more tangible. They can be browsed, searched, reviewed, and altered in a way that facilitates their management without the complications and restrictions of a full-blown IDE.

Rule engines are often avoided by DDD-ers (Domain Driven Designers) and OOP-ers (Object Oriented Programmers) because they pull domain logic out of the model and into a rule structure, disregarding many of the core features of OOP. (Separating the rules prevents classes from encapsulating their own behaviour, and doesn't allow inheritance or any form of overriding super class behaviour.)

Drools, a popular Java rule engine, uses an OOP language (Java-like) to write the RHS (rule execution) and to some degree, the LHS (rule condition). However, the rules themselves are still very much detached from the model. The rule is more like an /if/ condition with a /then/ procedure. If the code does not belong to an extend-able class hierarchy, it is anything but OO.

Tools could be written to allow a OO model, written in a language like Java, to be indexed and allow it to be browsed and searched - many IDEs already do this to a degree. However, the challenge is that the Java language (like many other languages) does not distinguish between the concepts (the classes and properties) and the behaviours (the rules). In Java, it is particularly difficult to distinguish between a property that retrieves and stores values and a method that applies logic to the state of the model.

To combine the features of OOP with the advantages of a rule engine, the domain model needs to be stored in a more formal structure to facilitate its management. It needs to be in a structure that clearly distinguishes the concepts and their properties from the behaviour rules, while providing OOP features to them. Such a structure would need to be index-able to allow it to be browsed and searched and straight-forward enough to allow alteration of individual rules without the risk of data loss.

Does such a rule engine exist? Check back next week.

Repository Entity Pattern

2008-12-18T18:01:00.000-08:00

The repository pattern is too often under used when developing a domain model. I like to take a spin off the Domain Driven Design repository pattern and apply it to all collections within the domain.

Whenever the model needs a collection of entities, I use a repository. It is basically a glorified collection, controlling access to underlying entities in a type specific way. The main advantage to using a souped-up collection is that as needs change the implementation can be changed while maintaining the same interface. This abstraction also co-locates similar queries and query building logic into a single class structure, minimizing its duplication.

Writing access and bulk update queries involves a significant investment for new models. The abstraction of using a repository interface allows you to focus on the business logic early on, while working with small datasets and in memory collections. As the model interfaces begins to stabilize more focus can be in optimizing entity access.

For complex models that require uniquely optimized data access and updates. The repository pattern allows integrated query building logic to be separated and shared within it own class structure.

An anti-pattern to be aware of when using the repository pattern is to ensure that you don't try and combine aggregates and repositories together. A repository should not have any properties, only a collection of entities. Violating this significantly complicates the implementation and leads to role confusion exacerbating the problem.

A good definition of the pattern by Martin Fowler can be found on his website.
http://martinfowler.com/eaaCatalog/repository.html

There isn't many good examples of the repository pattern, so here is an example interface of what it might look like.

interface Repository extends Iterable {
void add(E entity);
void remove(E entity);
void clear();
E findById(String id);
}

interface SequentialRepository extends Repository {
E get(int index);
int indexOf(E entity);
}

interface ScoreRepository extends SequentialRepository {
List findScoresByCategory(Category cat);
void setScoreLimit(int limit);
Score getMax();
}

The 7 Stages of Scaling Web Apps

2008-12-15T05:46:00.000-08:00

I came across this short presentation on the pains of scaling a web application and thought I would share it here.

http://www.slideshare.net/davemitz/7-stages-of-scaling-web-applications

Rich Internet Applications

2008-12-05T08:27:00.001-08:00

Silverlight, Flex, and JavaFx are all trying to capture a new market of rich Internet applications that promise to be the best of both desktop applications and web applications. However, none of them are accessible to agents. Like desktop applications, they can only be used for the purpose they were designed for and cannot be used for indexing, harvesting, or used in mashups. This is not surprising as none of their businesses depend on open interoperability. The only big web players that depend on open interoperability are Google and Yahoo!, which is why Google funds the Mozilla and Chrome browser projects.

The web's biggest success is the inter-connectivity and vast amount of information available in the same format (HTML). It will be interesting to see if these new RIA platforms will have any effect on the trend of moving applications to the web.

MVC With XSLT

2008-12-01T05:53:00.000-08:00

When Spring MVC (Model View Controller) was first released in 2003, it helped clarify the roles and bring a clearer separation between the model, the view and the controller to Java web development. Complicated HTML pages became easier to maintain and changes were straight forward to implement. However, web applications have changed significantly since then.

With the recent explosion of new JavaScript libraries the separation between MVC has once again become blurred. Many HTML pages today are loaded in stages (a la AJAX), one entity at a time. Traditional Spring MVC seems overly complicated for small single entity results.

Creating rich AJAX web applications can still follow the MVC design pattern, although it might require stepping out of the Java/JavaScript comfort zone.

Consider a typical asynchronous request:
1) User's activity triggers an HTTP request.
2) The server processes the request and may invoke changes to the model and/or return part of the model (an entity) to the client.
3) The script that sent the request, manipulates the response and displays it for the user.

In the above we can still see the MVC pattern: the model is the server's response, the view is the manipulation of the response for display, and the controller is the server. However, this differs from the original Spring MVC of 2003 - the view has moved to the client and the model is (in part) serialized in an HTTP message body.

Part of the confusion with AJAX development is around the role of the "view". It gets blurred between the serialization of an entity model and how JavaScript displays it. Inconsistencies in this area can cause many maintainability issues as the interaction between the client and server becomes confusing.

By viewing dynamic HTTP responses as serializations of entities from a data source (model), and leaving the "view" for the client, clarity and maintainability can be achieved. The only standard display technology that works equally well for large entities and entity fragments is XSLT/HTML. Today's modern browsers all support XSLT 1.0 transformations using JavaScirpt. By using XML for the model interchange and XSLT/HTML for the view display, JavaScript usages can be limited to what it does best: filling in missing functionality of the browser.

By limiting the role of JavaScript, its reusability is maximized. In the future the amount of JavaScript required by web applications should be significantly reduced. In addition, projects like webforms2 (at google code) promise to bring tomorrow's HTML5 and Web forms 2.0 to today's browsers.

XSLT brings more flexibility and reusability to the view layer (vs JSP-like technologies). By using client-side JavaScript/XSLT with other AJAX technologies modern web applications can achieve the richness of desktop applications, while still using best practises of the MVC design pattern.

Open Concepts

2008-11-24T07:32:00.001-08:00

Most examples of open classes are extensions of core system classes (string/float), such as adding a "to_cents" method on Float. Often such extensions are not needed, as a new wrapper class, such as "Money", would serve the purpose better.

Core classes may not need to be open for extensions, although that can sometimes be useful (remember how long it took Sun to add java.lang.String#replace(String,String)). It is important for Model classes to be open for downstream extensions. For this reason I use the term "Open Concepts" to indicate that the concepts in the model are extendable.

The biggest advantage for having "Open Concepts" is that the model can be modified without modifying the original source. This creates many opportunities for multiple team system development. By enabling the model to permit dynamic integration of independent and decentralized designs (open), the code becomes simplified and more useful to a wide array of use-cases. Consider the productivity advantages of allowing downstream development to have more influence to extend the supplied model without necessarily involving upstream in the changes. These are many of the advantages that allowed the Ruby language to gain so much popularity.

There are many ways to enable this, even in more mainstream languages. One popular approach is to implement a plug-in system for key points in the model (although this quickly become complicated with many "key points").

A more generic way is to implement the role class model in all the concepts. This allows every individual to expose itself through multiple concepts. However, anyone that has used Mozilla's XPCOM, knows first hand how the syntax of this pattern can get out of hand.

Today, with the popularity of new languages that support the run-time mixing of classes, new opportunities for creating an expendable model now exist. Such an approach allows both plugin-style and role-style extensions, but still uses the syntax of the host language. This allows the model to be used the same way as if no extensions are present.

Any large system will need some type of extendability be it a plugin-style, a role-style, or an open-style model. It is worth considering the approach early in the development cycle, as many of these approaches are intrusive, by syntax or language.

Storing Business Rules

2008-11-17T10:07:00.000-08:00

Every application has some domain logic or business rules, but how much thought goes into how it is stored? Try asking yourself the following questions to help identify possible ways to serialize the domain logic.

How often will it change? Will each deployment need to have customized rules, will they be changing at run-time?

Who will be managing the domain logic? Will it be managed by people with a particular skill set? Does the domain logic need to have its own version control and distribution channel? Will the domain logic be reviewed by independent parties?

What form will the domain logic take? Can it be defined in a formal structure? Is it data driven?

There are many ways that domain logic could be stored, the most common is not always the best. Consider the following ways to serialize and store domain logic. You might use the same language as the application logic, or You might encode it in a domain specific language.

Domain logic might be stored in flat files that are compiled or in a form more suitable for a rule engine. It could be embedded in one or more XML files, for indexing and quick access. Domain logic might also be stored in a formal structure, for user-initiated rules or dynamic version control.

Domain logic can take on many forms and how it will be stored should be consider carefully. Depending on the makeup of the team or other external influences there maybe a need to make a greater distinction between domain and application logic.

Utilizing a Multi-Core System with the Actor Model

2008-11-13T05:52:00.000-08:00

Demand for multi-core/multi-processor applications is growing, but developing for a multi-threaded application does not require a steep learning curve or an understanding of complicated edge cases. Learn how to develop efficient multi-threaded applications without using synchronized blocks.

http://www.devx.com/Java/Article/39868

What a Modelling Language Should Look Like

2008-11-10T10:13:00.000-08:00

InfoQ gave a summary yesterday of a few blog posts on Modelling languages.
Briefly, Steven Kelly and Juha-Pekka Tolvanen say such a language should:
1) Map to the domain problem concepts and not implementation details.
2) Be formalized and helpful.
3) Have stand alone tooling support.

http://www.infoq.com/news/2008/11/useful-modeling-language

Unfortunately, the discussion focused on a visual modelling language (UML) and did not address any alternative modelling languages, such as the standard Web Ontology Language (OWL), which addresses many of these issues. Although OWL was not designed as a visual language or arbitrary code-generation, it has grown to support such usage.

TopBraid Composer, for example, is a mature editor and visualization tool for OWL.
http://www.topquadrant.com/topbraid/composer/

Module Dependencies

2008-11-06T06:00:00.000-08:00

I was reminded recently of a domain model I used to work with that did not separate interfaces from their implementations. They never considered dependencies to be an issue because javac could compile the entire source tree at once.

When I came on board, their product was nearing it's first release and their domain model's class dependency chart looked like a bowl of spaghetti. Eventually the lack of dependency management become apparent: each team was developing at a different pace, but everyone was forced into monolithic releases.

Much of the circular dependencies were addressed by introducing interfaces or abstract classes. However, the process was slow and full of challenges. Hibernate only supports a single inheritance model, but with so many different perspectives such a restriction proved very limiting.

Many of these problems could have been avoided had more thought gone into the the conceptual class hierarchy earlier and distinct modules been created to force developers into observing and thinking about class dependencies early in the development cycle.

Where are programming languages going?

2008-11-03T10:03:00.000-08:00

Anders Hejlsberg gave a keynote last month at JAOO http://jaoo.blip.tv/#1324214 (published two weeks ago) entitled "Where are programming languages going?". He said that with a little more restriction in languages, compilers could do a much better job at optimizing the code. I think this is an interesting discussion and agree with many of he ideas.

Towards the end of the talk he addressed concurrency. He suggested that a for loop, for example, could automatically execute all iterations in parallel - if few mutable variables existed. This is an interesting idea, although a very difficult task. Unless the code fits into the map-reduce pattern, it will likely modify some shared memory structure or external state (read database) and could not safely be executed in parallel. I think we will have to see more integration between persistence frameworks and optimizers before such as idea could be realized.

What is the Semantic Web?

2008-10-30T12:44:00.000-07:00

I get this question a lot and below is what the semantic web means to me. What do you view as the semantic web?

Software is built around models: logical representations of a system of entities. Models are an important part of software development, as the user's experience is often a reflection of the underlying model. Any flaws in the model will cascade up all the way to the user's interaction with the system.

Today's software programs have many different users with many different perspectives. Attempting to satisfy everyone with a single model leads to complexity that make the model difficult to understand and use.

The semantic web provides standards to express interconnected models by capturing and contextualizing the knowledge about them. Semantic web technologies allows one to access the larger conceptual data model and from more abstract points of view. This enables both people *and* software to navigate between both high level and detailed views so information can easily be examined and modified from different perspectives.

Keeping Classes Small

2008-10-27T11:09:00.000-07:00

Most people would agree that smaller classes are better. It is generally accepted that smaller classes are associated with robustness, reliability, reusability, and understandability. However, what causes a class to become too large?

Many people instinctively try to implement a interface or concept as a single class. However, sometimes the implementation is too complicated to be understood quickly and the class size gets too large. If you think this might be the case, take a look at the class variables. If the number of variables is large, if some of the variables could be "grouped" together, or if some of the variables are only used some of the time, try and split the implementation details into their own classes and allow the original class to delegate to or compose them.

Every time you are creating or adding to a class ask yourself if this could better explained in a separate class. This will help you or a co-developer later find what it is they are looking for faster.

Sesame 2.2.1 Released

2008-10-24T05:44:00.000-07:00

This marks the first stable release of Mulgara's Sesame interface. Creating a unified API to access many specialized RDF stores.

Other RDF stores that support the Sesame API include:
OWLIM
Virtuoso
BigData
AllegroGraph

For more information about Sesame see:
http://www.openrdf.org/

Extracting Meaning from Text with OpenCalais R3

2008-10-23T05:46:00.000-07:00

This article shows how to convert unstructured written text into structured data using OpenCalais, which is a public general-purpose text-extraction service that uses a combination of statistical and grammatical analysis to extract meaning.

http://www.devx.com/semantic/Article/39550

Smaller Class Sizes

2008-10-20T06:57:00.000-07:00

We hear this a lot wrt education, but can it also help programming? I think so!

Code should be written in a way that explains what the code "should" be doing. This can only be done by continually giving the reader a summary of the operations that need to be performed. Think of each class as cheat sheet: it contains what is important to understand its behaviour. A class should not go into too much detail, but instead delegate to other classes for further details, leading to smaller classes. Every computer operation can be divided into sub operations. These sub operations may not be as important to a reader and should not distract or confuse the reader when trying to understand the purpose of the class.

Easy To Read Code

2008-10-14T14:25:00.000-07:00

If we wanted to write code for a computer, we would be using only 0s and 1s, but we don't. Code is written for human consumption. It is more important that a co-developer can easily "compile" the code than any machine. If any code contains a bug, a machine isn't going to fix it - only a human will. Therefore we should take extra care to make the reader's job as easy as possible by making the code as easy as possible to read. Any code worth writing is worth writing well.

Unlike a computer, we can't read one line at a time. Our natural field of focus has a limited width and this needs to be considered when writing and creating APIs. This courtesy leads coders to limiting the length of each line. The easiest way to correct long lines of code is to use more local variables to allow operations to be separated on separate lines. However, too often this is not enough.

Often libraries use overly verbose and repetitive names. This is done in the name of clarity, but at the expense of the readability of their users' code. API developers need to keep not only the readability of their own code in mind, but also the readability of their users' code. Here are four examples that I have seen recently, of APIs that force their users into writing difficult to read code:

ServletConfig#getServletContext() - The word Servlet is redundant in the method and could be removed to shorten calling code.
org.openrdf.repository.RepositoryConnection - Unless it is common to work with multiple connections from different packages, there is no need to repeat the word "repository" in the package and class prefix.
IReadableBinaryStreamRepresentation - Here is an example of a repeating suffix that adds nothing to clarify its use.
context.getKernelContext().getThisKernelRequest().getRequestScope() - If the API forces users in this type of repetitive message chaining, not only is the API forcing code that is hard to read, but it also couples the client to the structure of the navigation and any changes to the intermediate relationships forces the client to also change.

To any API designers (or would be API designers) out there: Please spend some time thinking on how you can reduce the repetition in your API and strive to use short concise names. It will go a long way to making more readable code.