[ engine overview | whitefrost.com | Stephen W. Cote ]
Engine for Web Applications is a client-side framework for applications. This document describes a paradigm for- and implementation of- Engine.
Much attention has been focused on application servers, development environments, and transports (eg: J2EE, .NET and Web Services). The missing component has been a client-side mechanism that both compliments and completes these architectures. Without a mechanism that stitches a browser to these robust features, the browser is little more than a blind client coupled with feature-specific script. Browsers released in 2002, and indeed the majority of active Internet Explorer browsers, include core standards support that makes Engine function.
Engine adds the stitching for web applications by exposing lightweight services such as messaging, task and transaction management, and wiring, to name a few. The binding provided by these services frees developers to create Web applications using their own paradigms, APIs, and graphic user interfaces.
The latest prototype is Engine for Web Applications Implementation #3: Griffin. Browse to the engine prototypes document for other prototypes. You may also be interested in the original design notes (incorrectly labeled for the M3).
[ top ]
The Engine framework creates an environment in which developers can add task- and transaction-based features. At its highest level and most complex level, which is to say the easiest implementation, developers do not need to implement specific parts of the API. Instead, they only need to enable the framework and use one or more of its features to bolt together their Web-based applications.
[ top ]
Engine supports the following browsers:
Note: The Engine architecture should not be used in lieu of good content structure. By using well formed content, compatibility with almost all browsers is maintained, and clients may continue to access your content even when their browser does not meet the minimum requirements.
[ top ]
Refer to the license for this distribution: engine.license.txt.
The Engine for Web Applications is implementated on the client as a JavaScript file and optional XML configuration documents. The following steps describe how to add the engine API to a web page.
<!--
Engine for Web Applications
Composite file "engcomp.js" release: 0.8
Copyright 2002, All Rights Reserved.
Author: Stephen W. Cote
Email: wranlon@hotmail.com
-->
<script
type="text/javascript"
src="{path}/engcomp.js"
>
</script>
Browse the available demonstrations for a detailed approach to basic to intermediate implementations.
[ top ]
A good portion of the Engine framework is rooted in a central question: how to bootstrap multiple actions together so that a particular response is triggered after all dependencies are processed? If you've glanced at the API document at all, the answer should be fairly obvious: The Task Service. However, with the capabilities of the Task Service, another question arises: how to connect core services and completed tasks without forcing developers to wade through an obtuse API structure? The last thing I want to do, and I'm sure the last thing most developers want to do, is to write code against some API whose end result is mostly structural and serves little purpose once its initial objective is complete. The answer to that question is the Engine Service.
This section describes the approach taken to accomplish a few central objectives I had for Engine, and to provide a high-level overview of the methodology behind some of the designs.
[ top ]
Above all else the Engine Service makes the most complex associations with the rest of the application to provide developers with the simplest implementation. Consider this high level comparison.
The following is a sample of an engine declaration.
<!-- engine declaration -->
<div
is-engine = "1"
engine-id = "oEngine"
engine-action = "{path}/tasks.xml"
engine-action-type = "xml"
engine-handler = "engine_driver"
engine-handler-type = "import-task"
>
</div>
<!-- The action and handler attributes can be pre-configured -->
The following is a sample of the task that would be needed to just start the engine object. Additional code would be required for a full comparison.
/* Task declaration */
org.cote.js.message.MessageService.subscribe(
"engine_service_initialized",
handle_engine_init
);
org.cote.js.task.TaskService.executeTask(
"engine_loader",
"xml",
"{path}/tasks.xml",
"import-task",
"engine_driver"
);
function handle_engine_init(s,v){
/* finish processing */
};
In order to continue the comparison down to the next level, a transaction would be created to load the XML file, parse and process the dependencies, and then process the handler to signal completion. Below the point of the transaction, it would be up to the developer to keep track of the various states and dependencies. If only a few actions were involved, this wouldn't be a problem. But consider what happens in more complex web applications, particularly those that define different sets of responsibilities for clients.
[ top ]
A feature I envisioned, which seemed complex to implement standalone (but is now easily expressed with Engine), was a mechanism to link multiple actions together. This objective was different than that of the Task Service in that the linkage had to be more expressive than a single task object permitted. The Transaction Service was quite suited for this, but was ill suited for what should be a simple and declarative implentation. In other words, I wanted to link otherwise unrelated actions together without having to require any special changes to those actions to support the linkage, and make the implementation easy enough to reuse.
At first I started looking at a variety of other designs, such as messaging, events, transactions, and task-based processes. However, I ultimately arrived at the following premise: a) Test, and b) Act if test is true. This is the foundation for the Primitive Wire Service, and is also employed within the Transaction Service and Task Service. The assumption that such a basic condition is complex is asinine.
But, ask yourself: how would you link two unrelated functions together, without modifying the code to provide callbacks? While the following might be one conclusion, f = function(){if (a()){return b()} else return 0;}, or another might be simply, var ret = (a()?b():0); it begs the rhetorical question: did this really accomplish the objective, or is this just the simplest explanation? To my original question, these were the simplest methods, but not the extensible solutions I desired.
The simple answer did not suffice because it did not address a number of conditions, particularly how to maintain the context in which the conditions would be applied. Part of the solution was Wires. However, while Primitive Wire Service and Wires were pretty robust, that also made it needlessly complex to achieve the simplest part of the objective represented above.
An acceptible solution eluded me until I commenced work on Page Configurations for the Engine Service. Part of the Page Configuration design was to include object constructor mappings to element declarations so that both the element and the constructor were part of the same configuration data. Although originally created for the UI components that were later removed, it set the ground work for creating Wire Links. The concept of a Wire Link cemented the design objectives, and thus the solution, to my original question.
If I could simply associate an element with an id that pointed to a Wire construct, how much simpler could that be? Furthermore, the Wire could be reused anywhere else in the application. The object definition for the wire link is listed below. Note that this definition is more or less static.
<object-definitions>
<definition id="wire-link">
<implementation no-recursion = "1">
<package pid="org.cote.js.engine.EngineService.object_config.pointers.wire_service" />
<constructor name="hardWire">
<param value="ora:engine_object" />
<param value="ora:id_attr" />
<param value="ora:xpath-node-value-list:params[@id='action_arguments']/param" />
<param value="ora:xpath-node-value-list:params[@id='handler_arguments']/param" />
<param value="ora:xpath-node-value:action-class/text()" />
<param value="ora:xpath-node-value:action/text()" />
<param value="ora:xpath-node-value:handler-class/text()" />
<param value="ora:xpath-node-value:handler/text()" />
</constructor>
</implementation>
</definition>
</object-definitions>
Within the same Engine configuriation would exist at least one Wire Link declaration. For example, the following declaration is used for controlling access to the View Account Flags configuration screen.
<wire-link id = "wireViewFlags">
<action-class>org.cote.js.snt.SimpleTransferService</action-class>
<action>getFlag</action>
<params id="action_arguments">
<param value="useraccount" />
</params>
<handler>switch_page</handler>
<params id="handler_arguments">
<param value="your_account_flags" />
</params>
</wire-link>
Thus far, these two XML constructs have defined the constructor for a wire and an instruction to the Engine Service to use the constructor to create a Hard Wire; that is a wire with static arguments. To implement this Hard Wire within the configuration, the following three tasks must be met: 1) add a reference, 2) set any visual indicators, 3) process the wire. The third task was easy enough as the Hard Wire implementation for a Wire is two straight forward function calls (thus becoming the simple construct from above). The second task was accomplished by hooking into the onpageconfigload message, and tacking event handlers onto elements with specific attributes. Since that is custom code, it was kept apart from Engine. The first task is also easy, as is represented in the following example. Note that the attributes have nothing to do with Engine and would be processed by external handlers.
<span simlink = "1" pid = "your_account_flags" wire = "wireViewFlags"> View Flags </span>
The customer handler to make use of the wire is represented in the following example.
/* add event listener */
org.cote.js.dom.event.addEventListener(oSpan,"click",fire_wire);
/* ... */
/* handle event for wired link */
if(oWire.invokeHardWireAction(oEngine, sWireId)){
oWire.invokeHardWireHandler(oEngine, sWireId);
}
In the above example, it is the simplest comparison that gets used, and it is fairly straightforward to declare one or more elements to use make use of the wire. It is also pretty simple to manufacture multiple wires for many situations. To see this example in action, refer to Engine for Web Applications Implementation #3: Griffin.
[ top ]
The Engine code base was created in a blind package hierarchy. Pseudo classes are stored in package-like structures for the benefit of a clean namespace, but are not able to import packages not already included in the distribution. Because Engine was designed to be delivered as a single deliverable (read as single file), there are no plans to create a more robust package system.
Accessing Engine members (packages, objects, methods, et al) should be preceded by at least one validation against a package. This will ensure that the code loaded, and the specified package, loaded. I typically just validate one of the last packages, either org.cote.js.template or org.cote.js.engine.EngineService, as any errors in preceeding packages will prevent the latter from loading. If the package exists, then the package has been built from the global namespace, which would be from the window object of the browser.
[ top ]
One-way communication between objects is achieved with message subscriptions, and announcements. The Message Service uses subscription-based messages to communicate variant data (from status updates to object references) between objects and with the host Web page. Objects may also make message announcements that are automatically published to the onsendmessage subscription, and filtered by a threshold.
A subscription applies to a subscription name, and can be filtered to publications made by particular objects. While messages can act like event announcements (and the fact that some message publications are named like events), the publication system is far more generic.
[ top ]
A core feature of Engine is a robust XML utility package that exposes simple mechanisms for developers to get and post XML data to a server (via XMLHttpRequest objects). The utility can be configured to create request objects as needed, or to create a pool of objects for reuse. Also, XML DOM references can be cached in-memory for specific requests so that the browser maintains the document in memory, and a given component may make as many requests against the cache as desired without impeding performance.
Note: The webservices package, which makes heavy use of this utility, is currently not exposed.
[ top ]
The concept of a registry, and specifically an object registry, is pretty basic, and is used throughout the programming world. The Engine implementation is not very different. However, it does raise an interesting question: Why?
A browser exposes a Web page DOM, most likely supports a script engine, and may have support for other extensions such as Java, Flash, and XML. However, when creating objects in JavaScript, where should these objects be kept? Global properties or arrays? Embedded properties on global objects? And heaven forbid, attributes on HTML Elements, or hidden within behaviors?
The Object Registry was created for Engine frankly because there was no better place to store a lot of objects. Yes, the storage mechanism is an array and a few hashes that are embedded properties. However, objects stored in the registry can be created anywhere and free other objects from maintaining object references. Any object can be stored in the registry as long as it exposes the following properties.
The Object Registry exposes the following methods for managing the registry.
A number of Engine features, such as the Transaction Service requires objects to be registered with the Object Registry.
[ top ]
The Transaction Service is used to create an intimate channel between one or more objects. Where the Message Service provides a one way channel to one or more objects, and the Task Service is a process structure that may operate on any object, the Transaction Service utilizes a specific API to enable object communication. Object communication as a transaction takes the form of a Transaction Packet that is passed between each object. A Packet exposes its own set of properties such as a name and id, and stores data defined by the objects participating in the transaction. When an object has completed its operations on the packet, its return value determines whether that object will continue participating with the transaction, and ultimately, whether the packet is closed.
For example, the Task Service uses the Transaction Service to manage individual task items. Each task item references a Transaction Packet which is what gets used to manage task state and interdependencies.
To participate in a transaction, an object must be registered with the Object Registry. Also, an object must define the following API.
The premise of object transactions is that the Transaction Service provides the channel but it is up to the objects participating in the transaction to keep the Transaction Packet in play. Every time the serveTransaction method is invoked, the doTransaction method is called on all participating objects. The owner object is always the first to receive the packet, and may control whether the packet is served to the participants.
Two types of transactions are supported.
The following sample code represents a basic implementation of the Transaction Service.
function TransactionSample(){
this.object_type = "sample_object";
this.ready_state = 0;
this.object_version = "0.0a";
this.object_id = "sample_id_0";
t.doTransaction=function(oService,oPacket){
org.cote.js.message.MessageService.sendMessage(
"Do Transaction for " + this.object_id,
"200.1"
);
return 1;
}
t.startTransaction=function(oService,oPacket){
org.cote.js.message.MessageService.sendMessage(
"Start transaction " + oPacket.packet_name + " for " + this.object_type,
"200.1"
);
oPacket.setBlockStartTransaction(false);
oService.addTransactionParticipant(oOtherObj, oPacket);
return 1;
}
t.endTransaction=function(oService,oPacket){
org.cote.js.message.MessageService.sendMessage(
"End Transaction for " + this.object_id,
"200.1"
);
return 0;
}
}
// create a new TransactionSample
var oSample = new TransactionSample();
// add the new object to the registry
org.cote.js.registry.ObjectRegistry.addObject(oSample);
// register the new object with the transaction service.
org.cote.js.transactions.TransactionService.register(oSample);
/*
Open a new transaction, 'demo_1', where oSample is the owner,
enter some data, and specify a handler.
openTransaction will automatically serve the packet to oSample
*/
var sPacketID = org.cote.js.transactions.TransactionService.openTransaction(
"demo_1",
oSample,
{id:"test_id",name:"test data"}
);
Communication is kept alive by volleying requests to serve the Transaction Packet back to the Transaction Service. The service does not implement a timeout or interval to check status. In other words, it does not poll for object details. The implementation is ultimately responsible for triggering the service to send the packet.
One example of using the Transaction Service would be for handling a customer order. Consider a Web page where a customer has elected to purchase a new DVD. A lot of different activities need to take place for the retailer to confirm the order, and for the customer to have the best experience. There are a number of security concerns as well. Whether this ficticious order is taking place using Web Services or separate form submissions, the Web site developer (or program manager) might (and will probably) decide to lump everything together into one large request, or break the request out into one long synchronous transaction. I'm not merely opining this case, I'm counting on it. Here is what needs to happen to complete the DVD order.
Assuming this process is to take place using Web Services, everybody who wants to do all this on one page raise your hand. Now, of everyone with a paw in the air, who wants to try to make these requests asynchronously from a Web page? Yes, I realize JavaScript is single threaded. But the external objects, such as the XMLHTTPRequest implementation, may not be. If we're going to write Web Services, lets not do ourselves the disservice of writing CGI wrappers around them so we can continue dumping the entire request through a standard form submissions. Sure, such wrappers will be necessary for downlevel compatibility. Its not necessary to push the envelope, but its worthwhile to at least peak inside.
Using the Transaction Service, here is a straightforward implementation to accomplish the order process as a set of partially asynchronous, and partially synchronous actions.
If the Web Services were used over SSL, as they should be for this example, then the security risk or no greater than as from a classic order form.
For an example of the Transaction Service, refer to the Basic Transactions Example.
[ top ]
Tasking is a sequence of synchronous or asynchronous processes, where an individual process may define an optional action, an optional handler, and stipulate zero or more dependencies.
Tasking is a key feature of Engine because it is one of the primary reasons I moved away from the MDI and the more recent M3 code bases. The MDI project addressed its own need for tasking with a convoluted system of polling and internal callbacks, while the M3 project went further by defining a task object for the purpose of bootstrapping certain actions. As I realized I was recreating another convoluted implementation in the M3 project, I took a step back and reconsidered how tasking should be addressed. While working on the original design notes I realized I was interweaving the notion of tasking with transactions. This crossover resulted in the separate Transaction Service and the implementation of the Task Service as a transaction participant.
Note: the actual implementation of the Task Service is somewhat different than that identified in the original notes.
One example of how the Task Service could be applied is to consider Web sites that require a particular loading sequence. Such a sequence might be: a) load a component, b) load data, c) process (b) with (a), z) send notification of success or failure. In addition, there may be other dependencies such as d) wait for window to load, and e) load configuration data to identify network URL for for (b).
One of the very first responses I imagine this example inspires would be to add a window load handler, create an instance of a component, load data from within the component and use a readystatechange or load handler, and then either fall silent while waiting for UI-events or invoke some callback mechanism. If you are partial to using DHTML Behaviors, this might even appear tidy. But in both cases, the issue should be obvious: the management of the process sequence is left to individual components or a containing script. And, what was just an unfortunate fact of life in the above example becomes an architectural problem when developers result to polling to complete the sequence. Or, something I consider worse (and which I did in the other projects), writing convoluted routines specific to that particular component and data.
Let's revisit the previous example in terms of Tasking.
It seems simple enough. We know (z) should be done last. However, there are several interwoven dependencies that are easy to overlook. For instance, (d), waiting for the window to load. This isn't a problem if the process was started after the window loaded, but doing so only delays (if ever so slightly) rendering the content for the user. Also, the data cannot be loaded (b) until the location is discovered from (e), therefore preventing (c) from being completed. Granted, any configuration could simply be included on the page and thus make the process much more straight forward. However, it is that same mentality that leads to the convoluted processes precisely because apparent simplicity is purchased at one location, but at the cost of something wildly more complex in another location.
As a task, this process becomes quite simple to manage. The following task configuration outlines the above process, where ExampleProcess is the entry point, where no action is defined, and where the handler is an invocation of the global function allDone.
<task id="ExampleProcess"
action-type="default"
action="[nothing]"
handler-type="function"
handler="allDone"
>
<task rid = "setup_object" auto-execute = "1" />
</task>
<task id="setup_object"
action-type="default"
action="[nothing]"
handler-type="script"
handler="#cdata"
>
<task rid = "create_object" auto-execute = "1"/>
<!-- dom_event_window_load is returned automatically by the Task Service -->
<depends rid="dom_event_window_load" />
<![CDATA[
// setup object here
t.returnDependency("setup_object");
]]>
</task>
<task id = "create_object"
action-type="default"
action="[nothing]"
handler-type="script"
handler="#cdata"
>
<task rid = "load_config" auto-execute = "1" />
<![CDATA[
// make the object here
t.returnDependency("create_object");
]]>
</task>
<task id="load_config"
action-type="xml"
action="config.xml"
handler-type="default"
handler="[nothing]"
/>
The script to implement the above XML-based configuration is not too complex. However, it illustrates a basic tenet of the Engine design: Why saturate a Web page or Web application with structural code that could be better managed by a background service? In both the previous example and following example, the objectives are the same. The difference is that in the previous example the objective is more artfully stated well apart from the Web page or application. The only code developers needs to concern themselves with is starting the task that loads the XML data, and the allDone function. In the following example, which is the script translation of the previous XML, the developer must be more knowledgeable of the Engine API. Why should developers be concerned with the finer details of the background process (as long as it works) instead of just receiving notification that the process was finished (or, depending on the implementation, didn't finish)?
var oTS = org.cote.js.task.TaskService;
function setupExampleObject(){
// setup example object
// return true to automatically return the dependency
return 1;
}
function createMyObject(){
// create example object
// return true to automatically return the dependency
return 1;
}
var task1 = oTS.addTask(
"ExampleProcess",
"default",
"[nothing]",
"function",
"allDone"
);
oTS.addTaskDependency(task1,"setup_object");
oTS.executeTask(task1);
/* can't use #cdata since there's no XML-based tasks */
var task2 = oTS.addTask(
"setup_object",
"default",
"[nothing]",
"function",
"setupExampleObject"
);
oTS.addTaskDependency(task2,"create_object");
oTS.addTaskDependency(task2,"dom_event_window_load");
oTS.executeTask(task2);
var task3 = oTS.addTask(
"create_object",
"default",
"[nothing]",
"function",
"createMyObject"
);
oTS.addTaskDependency(task3,"load_config");
oTS.executeTask(task3);
var task4 = oTS.addTask(
"load_config",
"xml",
"config.xml",
"nothing",
"[default]"
);
oTS.executeTask(task4);
For an interactive demonstration of the Task Service, take a look at the Tasks Prototype.
[ top ]