The AIR platform defines distinct sandboxes for trusted and untrusted code, and provides a way to talk securely between each sandbox via sandbox “bridges”. This is a lynch-pin in the web-meets-desktop strategy that AIR embodies, but it can also present some of the trickier development challenges, with plenty of head-banging opportunity. I’ll share a few tips to help you avoid those head/keyboard collisions.

The Problem

Any content downloaded from the web is inherently tainted and untrustworthy. Any code running from outside the application directory is blocked from accessing AIR APIs. Nor can you write to the application directory at runtime and load up a downloaded page as a part of your application - with all the system access that implies. That would be bad. But, what if you wanted to make an offline viewer of some of your web content? You can download the files, write them to your application storage directory and load them up into an iframe or nativeWindow, and happily browse them in the bus/train/plane/smallest-room. Making a window or iframe for that content is trivial, but to make a compelling and seamless experience you’ll eventually need to pass events to and from the application and the offline content - and that means live, secure, runtime communication across the two sandboxes. You need sandbox bridges.

Bridging the Sandboxes

Adobe has provided quite a bit of documentation about security in AIR. See this Security FAQ and this entry on the Sandboxes for starters. As a developer, you can create an explicit interface that places functions or objects on the sandbox “bridge” - which appears as a new window property. The process is well outlined in this quickstart guide, so I won’t duplicate it here. Here’s the short version, imagine we’ve got our main application window (in the application sandbox), and a child iframe (non-application, network sandbox).

// make a function on the parent available to the child window
window.parentSandboxBridge.receiveSomething = function(arg) {
  alert("the child sent me something: " + arg);
}

And in our child window:

// make a function on the child window available to the parent
window.childSandboxBridge.receiveSomething = function(arg) {
  alert("the parent sent me something:", arg);
}

To call them: from the application window

myIframe.contentWindow.childSandboxBridge.receiveSomething(5);

..And from the child:

parent.parentSandboxBridge.receiveSomething("a string");

The two sandboxBridge objects already provide some measure of protection by ensuring intentionality — there’s no ad-hoc access to content in other sandboxes — everything must get channeled through the bridge. There are some gotchas though that require a little more understanding of how the bridge works. Think of it like two prisoners in adjacent cells, with a sturdy iron grill set into the wall between them. They can talk, but pass nothing potentially dangerous to each other. Strings fit just fine through the bars, so do Numbers. Objects and Arrays also fit, but their contents have to meet the same conditions; content passed across the bridge is serialized and de-serialized. That doesn’t sound terribly problematic until you consider what gets blocked: Date objects, RegExp objects, Errors - the list goes on. If its not an Array, Object or one of the basic primitive object types, it’s likely to come out the other end as a neutered object. At first glance it might appear that you can pass DomNodes across. innerHTML works, so does tagName. But these are just more examples of string properties; if you attempt to call getElementsByTagName() on what gets passed across, it will fail.

The result - and the goal from Adobe’s point of view - is that you have to think about what you pass between sandboxes. I’ve explored how values get passed across the sandboxBridge, and provided some reference code for setting it all up in a tester AIR app, downloadable at the end of this article. Examine the code carefully and compare to the result values, and add and change tests to get comfortable with what’s coming across.

Event Sequence

The other gotcha that’s waiting for you is one of timing. Say you have an iframe thats loading content from a non-application sandbox, but you want to get notification of events in the parent window: Code in the child window needs to be able to rely on functions on the parentSandboxBridge being defined already as its interpreted, and code in the parent window needs be able to call functions on the childSandboxBridge when they are ready, but not before. A new iframe attribute ondominitialize makes it possible - it gives you a hook to do setup work in the parent page before any of the script elements in the iframe’d document have even been read. Note that its not a window event, its an iframe attribute - it can be set once from the parent, without the loaded document having to know about it - and it will fire whenever a new document is about to load.

SandboxBridging in Practice

Here’s a couple examples from the Dojo Toolbox where we put it into practice:

API Viewer Screenshot
In the Dojo Toolbox’s API Viewer - we browse downloaded content, drop some files alongside it and talk across the bridge to signal things like loaded events. When the child document is loaded, this event is passed to the parent which manipulates the history array/pointer accordingly.

Builder Screenshot
In the Dojo Toolbox’s Builder - we ask the user what directory to conduct the build in, and the build process needs the ability to load and eval code, so it must be done from a local sandbox. We build an iframe with the required url and set ondominitialize to point at the _setupBridges method of our module. That method can run knowing that iframe.contentWindow is available, and set up the parentSandboxBridge so that when the child document loads, it is immediately available and in scope. When both parties are ready, the module (application sandbox) can call a method in the child to start the build. Progress indication is a function of the module, but the child needs to provide updates, so it does so via a method on the parentSandboxBridge.

The iframe creation code ends up looking something like this:

var html = '<iframe id="UI"'
+'	  src="http://localhost/ui.html"'
+'	  sandboxRoot="http://localhost/"'
+'	  documentRoot="app:/"'
+'	  style="background-color: #ffe; width:100%; height:100%">'
+'</iframe>';
 
document.getElementById("icontainer").innerHTML = html;

Setting the sandboxRoot to a non-app:/ hostname/url puts content in my iframe in a network, non-application sandbox - even though in this case its technically loaded from the application directory. I create the iframe dynamically because a) Sometimes you need to, and b) because it gives me the full range of before/during/after events around the window creation.

Summary

Let’s review. You don’t need to worry about sandbox bridges if:

1. All your content is built into the installer, and available under the app:/ root (i.e. the directory you install into, or when testing, the directory containing the application decscriptor file.)

You do need to use the sandbox bridges if:

1. You want to load up content that was not a part of your application as installed - be it from the storage directory, or out there on the web - AND - you want some to access select parent-window properties and functions from the child (parentSandboxBridge) or child-window properties from the parent (childSandboxBridge)
2. You have code that uses eval, or the new Function("statements as string") technique.

Here’s what you can expect to get across the bridge unmolested:

• Strings, Numbers
• Objects, Arrays - though their contents must also pass the same tests
• Functions (!) my tests show you can pass a function across the bridge, though you may lose the intended scope, however simple closures seem to work.

You’ll run into problems with:

• Date objects
• RegExp objects
• Any kind of AIR/Flash objects
• DOM Nodes

Finally

The security sandboxes - and the bridging facility across them - make possible some sophisticated interactions across application and non-application code. They are at once a PITA and a power-tool. They allow you as the developer to establish degrees of trust rather than just a good/bad determination, and provide a safe api to remote content.

Please download the sandbox tester code (HTMLSandboxBridge.zip) here. Run as usual with adl HTMLSandboxBridge/application.xml. This code builds on and owes a credit to Jeff Schwartz’ HTMLSandboxBridge, available from his QuickStart article mentioned earlier.

SitePen’s Support service is built using a variety of interesting techniques and technologies. Read on to see how we built a system that treats the web browser as a real client tier and bridges the worlds of JavaScript, Python and PHP seamlessly to provide a great experience for our customers.

Starting with the User

Even though this article is about the technology used to implement SitePen’s Support service, it doesn’t make any sense to talk about technology without talking about the user experience. The technology is there to do something. But, what?

With the SitePen Support service we provide support for the Dojo Toolkit, DWR and Cometd open source projects. We’re out to provide customers with the help they need when they need it. At the highest level, we needed to:

1. Collect support requests from our customers
2. Act on them
3. Keep the customer informed
4. Follow along with the terms of our support contracts

Without those things, we wouldn’t have a service at all. In addition, there are other requirements for making it the kind of service we’d be proud of:

1. The customer user interface should be very responsive
2. The user interface should be less like a content-oriented web site and more like an application
3. People in a company should be able to work together easily (data should be shared)
4. Customers should be able to bring themselves up-to-date on what’s happening in their account at any time
5. Email is still a super convenient user interface

Note that our goals were all about providing a great support service, and not about creating software. If there was off-the-shelf software that would do all of the above for us, at the level of quality we expected, we would certainly have used it. But, there wasn’t. Which brings us to…

The Tech of SitePen Support

To realize those goals for the service, we employed a bunch of different tools and techniques:

• Dojo runs the client-side
• The client drives the whole interaction
• The browser speaks JSON-RPC with the server for most operations
• The server is built on Python’s WSGI standard
• Client-driven apps have very little “obscurity” to try to hide behind, so we needed to be sure we followed best practices
• Off-the-shelf help desk software (HelpSpot) handles part of the work for us

The Client-side Runs the App

A typical web app today looks something like this:

Typical modern web app model

Most interactions are decided by the server. The client makes a request, the server gathers data and uses some sort of template engine to format that data and present it. That cycle is repeated over and over again, with the server always deciding what comes next and how the next bit will be displayed. Many apps today add some Ajax to that (that little JavaScript box at the top of the diagram), but very often the formatting of data is handled by the server and the client just uses .innerHTML to drop the fresh content in place.

For our support application, the model looks more like:

One model for rich client web apps

With this setup, the server is not responsible for the presentation layer at all. The server sends up static HTML files, and the browser does all of the work in displaying the data to the user. This approach was the topic of my PyCon 2008 talk: Rich UI Webapps with TurboGears 2 and Dojo.

If plain HTML is like the modern day equivalent of a green screen terminal, this design approach helps make your browser usable as a real thick client.

Dojo Runs the Client

The entire user interaction in the Support application is driven by the JavaScript client-side code. Dojo is a natural fit for this style of working, with its built in module system, RPC support, dojo.data interfaces and powerful Dijits.

We set up a very simple “PageModule” system where we use Dojo’s dynamic loading to load a new JavaScript module from the server and then call “initPage” on the code in that module. That will load up any HTML it needs to display and initialize everything for the user. A simple call to spsupport.core.loadContent is all we need to do to move the user onto the next piece of functionality:

loadContent: function(moduleName, params) {
	// Loads a new content section (if necessary).
	// A content section is defined by a Dojo module that
	// has an "initPage" function in it. The module is loaded
	// if need be and then initPage is called.
 
	if(spsupport.core.currentPage && spsupport.core.currentPage.cleanupPage){
		spsupport.core.currentPage.cleanupPage();
	}
 
	var module = dojo.getObject(moduleName);
 
	spsupport.core.currentPage = module;
 
	if(module && module.initPage){
		// initPage is like _Widget startup(), though you can safely
		// call it often. each "content" resource should implement it's
		// own _started mechanism in initPage, and treat initPage() as
		// if it were a "selectChild()" call
		 module.initPage(params)
	}else{
		var thedot	= moduleName.lastIndexOf(".");
		var packageName = moduleName.substring(0, thedot);
		var moduleRemainder = moduleName.substring(thedot + 1) + ".js";
		dojo.xhrGet({
			url: dojo.moduleUrl(packageName, moduleRemainder).uri,
			preventCache: true,
			handleAs:"javascript",
			load: function(js) {	
				if(js && js.initPage){ js.initPage(params); }
			}
		})
	}
},

With this kind of modular architecture, we could have a giant application in which everything gets loaded on demand. Combining this setup with Dojo’s build system gives us quite a bit of control over exactly when things load, allowing us to balance initial load time with interactive responsiveness. Any significant “single page” application will need this. The Support application is by no measure a “giant” application, but using good application design techniques like this allows us to add whatever features we need to the application without impacting its load time or responsiveness.

URL Dispatch: Not Just for Servers Anymore

The page content is decided by the JavaScript in the client, not by the server.

URL dispatch is one of the core features of a server-side web framework. It turns out that putting the client in charge moved some of the burden of URL dispatch to the client! When you hit the front page of the Support site, the JavaScript figures out where you really want to go:

• /: send the user over to the support page on SitePen’s main site
• /?login: give the user a chance to login
• /?signup-(someplan): give the user a signup page with a plan selected

When working with server-side frameworks, you get used to URLs being divided up by slashes and everything after the ? denoting extra query parameters. With the client in control, the slashes tell the server what static file to serve up, and everything after the ? tells the client what to display. Given that we only have three different possibilities there, we didn’t have to get fancy with our URL dispatch. You certainly could write a client-side framework that has many of the same features you get from a server-side framework, if that’s what your application needs. The support application only needed to interpret a very small number of URLs.

Full-stack Framework? Not Anymore!

Our support application doesn’t use server-side templates for the user interface and doesn’t really do URL dispatch. The “full-stack” web frameworks in use today (Rails, Django, TurboGears, CakePHP, Grails, to name a few) are basically defined by their URL dispatch, templates and database support. Given that we didn’t need two of the three of those, we could go a lot simpler on the server than a full-stack framework.

Our server-side code is written in Python. Many components and libraries are now built around the Web Server Gateway Interface (WSGI) specification, making it easier than ever to put a collection of webapp components together. WSGI works well enough to have spawned a version in Erlang.

In our main web stack, we gathered up the following components:

• CherryPy’s WSGI server, used both in development on its own and in production behind Apache
• Luke Arno’s Static WSGI app, which serves up static files. This is used primarily in development, and Apache serves up our static files in production
• Mikeal Rogers’ wsgi_jsonrpc for responding to the RPC requests from the client
• Ian Bicking’s WebOb for the small number of dynamic operations that couldn’t be JSON-RPC
• Mike Bayer’s SQLAlchemy for database mapping

JSON-RPC

In the past, I’ve often used plain old HTTP requests returning JSON results as a convenient and simple mechanism for requesting data and actions on the server. In fact, that was the approach I took in my PyCon talk. For the Support project, we decided to use JSON-RPC instead because it’s a little bit cleaner. In Dojo, making a JSON-RPC request is just like making a function call that returns a Deferred. So, the syntax for using our server-side API was very straightforward. Even better, parameters and return values automatically came across as the correct types (strings, numbers, arrays, etc). The server-side is also simplified, because it does not need to do much in the way of URL dispatch (JSON-RPC POSTs to a single URL) and the server doesn’t need to worry about converting incoming values from strings.

wsgi_jsonrpc is quite easy to work with. We subclassed it to handle our authentication easily and added a neat bit where making a GET request to the JSON-RPC URL would return the service description. That little change made it easy to wire up Dojo on the client:

dojo.xhrGet({
	url: "/jsonrpc",
	handleAs: 'json',
	load: function(response){
		spsupport.service = new dojo.rpc.JsonService({
			serviceType: 'JSON-RPC',
			serviceURL: "/jsonrpc",
			timeout: 6000,
			methods : response['procs']
		});		 
	},
	sync:true
});

This small snippet will synchronously load the RPC service description. It works synchronously because the UI can’t do much until it can make RPC calls for data. Then, it pulls the function names out of the response to create the JsonService. From that point onward, we just make calls to spsupport.service.function_name whenever we need to call the server.

Each available RPC call is simply a Python function in a module that has some extra metadata attached to it. For example, the request_details function is used to look up a support request by ID and return the detailed information about the request:

@auth
@params(dict(name='id', type='str'))
@returns('obj')
def request_details(user, id):
    """Returns the detailed information for a request.
 
    Params:
 
    * id: the request ID
 
    Returns:
 
    * object with the detailed information
    """

The @returns decorator is used to mark the function as one that should be available via JSON-RPC, and to also make note of the return type. The @params decorator, combined with the return type listed in @returns, are used when generating the service description for the client. @auth tells our JSONRPCDispatcher subclass that this function requires authentication. Whenever the @auth decorator is present, the first parameter passed to the function is always the user object. The function itself can then perform additional checks. For example, request_details makes sure that the request is from the same organization as the user.

This design makes it very easy for us to write automated tests for the server side code. I’m personally a fan of test driven development in general, and in the next section we’ll see why automated tests are particularly important for this kind of application.

All Out in the Open

When you provide a rich user interface, particularly using Open Web technologies, you cannot count on security through obscurity. You have to assume that people will study your code and learn about all of your “hidden” URLs that make up sensitive APIs. You would never guess that Gmail doesn’t have a public API, given the number of add-ons people have made for it.

Never trust the client code and requests coming from the client. When creating new, authenticated APIs on the server, the first unit test I write is one that ensures that unauthorized users are given the boot. When working with WSGI and WebOb, writing tests that can run without a server is quite easy:

def test_bad_credentials():
    req = Request.blank('/lookup')
    req.headers['Authorization'] = 'Basic HiThere=='
    res = req.get_response(requests.lookup_customer)
    print res
    assert res.status == '401 Unauthorized'

webob.Request.blank gives you a new Request object that is properly populated to look like a real request. You can then make changes from there to set up your test conditions. In the example above, I’m passing along bad authentication information. At the end, I assert that the result of sending bad authentication information is the expected 401 response.

As easy as testing is using WebOb, unit testing the JSON-RPC calls is even more straightforward. We just call the function directly as we would any Python function that we are unit testing. This kind of application setup makes server-side testing a breeze.

Of course, there’s a lot more work required to ensure that you’re handling data securely than just authenticating the users. We confirm that the user is authorized to access the data they are trying to access (with unit tests, of course). Using SQLAlchemy, we are not vulnerable to SQL injection attacks. We also run all of the requests over SSL to make sure that customer data is not grabbed off of possibly insecure networks.

HelpSpot: the Extra Layer in our Stack

Once a user is logged in, they’re taken to the /dashboard/ page where they can review their requests and account information and create new requests. The support dashboard is built around everything I’ve discussed so far. Each “page” in the dashboard is a separate module loaded via the loadContent call, and those modules make JSON-RPC requests to retrieve and update data on the server.

When you first log in, you can see the recent support request activity.

All of the details of your current support plan are available on one screen.

The server-side software that we wrote is responsible for keeping track of support plans and gathering up support requests for a given organization so that they can be displayed together. The support requests themselves with their complete histories are all tracked by HelpSpot. Within SitePen, we use HelpSpot’s user interface to update requests, and HelpSpot manages all email interaction. This saved us a good deal of implementation work.

HelpSpot is written in PHP, so we can’t directly call its functions from our Python-based server. One reason we chose HelpSpot is that it offers a solid web API of its own. We make simple HTTP requests to HelpSpot and it returns JSON formatted data. All of those requests are between our support application and HelpSpot. There are some instances where we needed to look up or update data in bulk, and HelpSpot did not have APIs specifically for that. Luckily, HelpSpot’s database schema is nicely designed and easy to understand, so there are instances where we also collect data directly from HelpSpot’s database.

Putting it all together

Bringing new developers up to speed on our support project is simple, because we use zc.buildout. zc.buildout creates a sandbox on the developer’s system with all of the pieces they need to work on the project.

Once we’re ready to deploy, we use a Paver pavement file to describe how to package up the software. Our pavement runs the Dojo build system to combine and shrink the JavaScript files and then bundles everything up into an egg file. The pavement also includes a task that will upload the egg to the server. Using zc.buildout also works great at deployment time, because we just have to run “bin/buildout” in the server’s deployment directory.

Creating the Desired User Experience

I think that the tools and techniques we used in building our support application were nifty and different from how most people are building webapps today. Our approach was all driven by a desire to provide a great user experience that meets our four original goals:

1. Collect up support requests from our customers
2. Act on them
3. Keep the customer informed
4. Follow along with the terms of our support contracts

The right tools helped us to reach these goals without a giant development budget.

Next month, I’ll be writing about the processes and tools we use to manage the support service within SitePen and ensure that we’re always on top of our customers’ needs.

The new windowName module (dojox.io.windowName) now includes support for resource authorization as Neil Roberts described in his article on xauth. Now the windowName module can be used with a window.name enabled resources for simple (direct) access as well resources that require an authorization step.

To request a protected access resource, you still use the send method, but indicate that the resource may need authorization by including an authElement property that refers to a DOM element which can be used for displaying authorization. To access a protected window.name resource, first create an element that will be used for authorization:

<div id="authTarget" style="width:400px; height:200px"></div>

Now we can send the request for the resource:

var authElement = dojo.byId("authTarget");
dojox.io.windowName.send("GET",{
  url: "http://persevere.sitepen.com/Customer/4",
  authElement: authElement // this will cause send to use the authorization protocol
});

Here is a demonstration of using the windowName module to connect to a remote authorized resource (click the “get data with authorization” button within the demo).

From the Server

If you provide access to a protected resource for a cross-domain site, you should generally require authorization from the user, such access should not happen without the user’s consent. A request for protected access resource with the windowName module is indicated by a request with the URL query parameter windowname=auth. The client should (and Dojo’s windowName does) set this query parameter. This windowname=auth parameter, inserted in the URL by the client, can be used by the server so that it knows that it can authorize the user before setting the window.name property and returning a response to the requesting page. If a request for a protected resource without this parameter is received, the server can simply return a response indicating that this was an unauthorized access attempt (401 status with an appropriate message).

When the server receives a request with the windowname=auth parameter, it can return a page that will authenticate the user if necessary, and ask the user if they authorize this site to access the resource. Once authorization has been confirmed, the data can be provided to the calling site by first getting the current value of window.name (this is the return URL), setting the window.name property to the data to be returned to the calling site, and then changing location of the return URL (original value of window.name). For example:

<html>
  <body>
    <script type="text/javascript">
    function authorized(){
      // this the function that is called once authorization has been done,
      // you can use any type of authorization
      var loc = window.name;
      window.name = "My Data";
      location = loc;
    };
    </script>
    <button onclick="authorized()">Authorize this resource</button>
  </body>
</html>

Note: make sure the resource’s content type is text/html so the browser properly interprets the page.

Improving the User Interface

If the target resource has already been authorized (perhaps previously in the session), the requested resource may return data immediately upon loading. In these situations, you may not want any visible user interface until it has been determined that the resource needs authorization interaction from the user. The dojox.io.windowName.send() method provides a callback argument so that you can be notified when authorization is actually needed. Let’s walk through the request and authorization process:

• First the client mashup application makes a request for a resource through the windowName module and this is sent to the server.
• The response is received from the server. If the resource is already authorized for this web site and user, than the window.name property can be set immediately and the client application can proceed quietly without bothering the user at all, and the user should not be shown any authorization user interface elements.
• If the response requires authorization (or authentication), then the authorization user interface elements should be displayed for the user. Note that the user interface for authorization does not need to be displayed until authorization has began.
• Once authorization is complete, the authorization user interface can be hidden, the user can return to focusing on the client application.

By only displaying authorization user interface as needed, we can create a more minimal compact user interface. To do this, we will start with the authorization element being hidden:

<div id="authTarget" style="display:none; width:400px; height:200px">
</div>

And we can use the callback to display the authorization area when appropriate:

var authElement = dojo.byId("authTarget");
var deferred = dojox.io.windowName.send("GET",{
    url: "http://persevere.sitepen.com/Customer/4",
    onAuthLoad: function(){
       //authorization page has loaded, make the auth UI visible now
       authElement.style.display = "block";
    },
    // this will cause send() to use the authorization protocol
    authElement: authElement
});
deferred.addCallback(function(result){
    authElement.style.display = "none"; // hide UI again now that we are done
    ... process the result ...
}

Conclusion

Using this window.name authorization protocol leverages the native sandboxing and inter-frame messaging capabilities of existing browsers to provide a significantly more secure and convenient mechanism for accessing protected cross-domain resources. This approach is more convenient because web services can utilize cookies so that authentication does not have to be repeated, and authorization can take place in page without requiring page redirections. The user can simply click a button to authorize a request. This approach is more secure because users do not have to provide their credentials to third party mashup sites giving them full access to their information. Web services can provide granular authorization levels for resource access. Far from a hack, this techniques utilizes the intentional security mechanisms in the browser to provide proper security boundaries between domains so that data can be safely shared without vulnerability to exploits. For resources that require authorization, this technique is actually simpler and more well integrated with the browser than the new cross-site XHR capabilities that will require custom user interaction or redirection schemes for secure authorization (although these new capabilities are still extremely valuable, especially for public resources).

The xauth library provides simple access to the window.name authorization protocol and can be used with any JavaScript library. Dojo’s windowName module provides tight integration with Dojo’s IO system. These client libraries further increase the accessibility of this new technique. As web services adopt this approach, we can easily utilize cross-domain data interaction with true security barriers between the client and server. Consumers are protected from malicious scripts from the web service, and the web service can protect itself with proper authorization mechanisms.

A new top-level package was recently added to the Dojo Toolkit called Deft — an acronym for Dojo Experimental Flex Technology. The Deft package was created and is maintained by SitePen’s Tom Trenka, taking advantage of Adobe’s new MPL licensing, and the corresponding APIs of the Flash Player. Most articles focus on Adobe’s Flex Builder, which isn’t open source or free. The majority of articles about Flex and the Flex Builder also put an emphasis on components developed using a combination of ActionScript and XML-based description files, known as MXML.

Instead of taking this approach, Deft focuses on ActionScript components created in support of the various projects within the Dojo Toolkit (mostly for DojoX). Deft source files are well organized based in part on the organization of other Dojo Toolkit projects, as well as the package structure required by the Flex compiler. Most Flex applications are based on the Flex AS3 Application class, which forces you to write at least one “controlling” MXML file in order compile your code. Instead of this, Deft components inherit primarily from the Sprite class — which allows you to write pure ActionScript code.

Compiling components within Deft

To compile code within Deft, we use the command line Flex compiler called “mxmlc” — which is available with the Flex Open Source SDK. You can use this command to compile both full Flex applications (i.e. with an MXML definition file) and Deft components.

First, grab the latest Dojo nightly build that contains Deft, or download it with Subversion.

Then download the Flex SDK. I recommend placing it in an easily accessible location, as we’ll be using some absolute paths. It doesn’t require installation, so it’s ready to go after it’s unpackaged.

Hello World

Now let’s start with some very minimalist AS3 code that we can compile:

package {
	public class HelloWorld {
		public function HelloWorld():void {
 
		}
	}
}

This will be a top-level package (on the same level that our compiler will be) so we just use “package” without giving it a name (more about packages later). The next line is our class declaration, which is the same name as our file. And next is our constructor.

Of course, this will compile, but would be quite useless. Here’s a little bit of code that will add display capabilities, and add a text field to the stage:

package {
	import flash.display.*;
	import flash.text.*;
	public class HelloWorld extends Sprite {
		public function HelloWorld():void {
			dbgText();
		}
		private function dbgText():void{
			var myText:TextField = new TextField();
			myText.text = "Hello World";
			myText.background = true;
			this.addChild(myText);
		}
	}
}

This will give you a basic text display object on the Stage. If you’re used to AS2, some of the new concepts can be tricky. In AS3, there’s no _root or _level0, and not a very obvious way to attach display items. Here we are importing the flash.display package, which includes the Sprite class that we’re extending. The Sprite class can be thought of as a stripped down version of MovieClip. Since a Sprite is a child of the Stage, it contains methods for adding display elements - in this case, our TextField, which gets added with addChild().

Now let’s open Terminal and compile the code. I’m going to be using Unix conventions here, but for Windows users, it’s not much different. The basic parameters for our compiler are: mxmlc, file-input, file-output.

Now it Gets Tricky

NOTE: The path to my dojotoolkit directory is /Users/mike/Sites/Tests/. I’m going to represent this path with a tilde (~) to help make things clear. But I recommend use all absolute paths initially.

Package names, the directory structure, and the current “location” of compiler must all be in sync, or you’ll get the inevitable error:

Error: A file found in a source-path must have the same package structure
  '(folder name)', as the definition's package, '(package name)'.

Our HelloWorld.as file is going here:

~/deft/trunk/deft/HelloWorld.as

…and we will navigate our command line to the same location. Compiling from here should work. But in my case I got the dreaded error shown above. I fixed it by adding another parameter to the command:

-source-path= ~ /deft/trunk/deft

This clearly defines the root of the package. Finally we define the output file, which will also be in the same location:

-output= ~ /deft/trunk/deft/HelloWorld.swf

The following is my Terminal command for compiling. If you add the mxmlc to your system path, you can use mxmlc instead of the entire path. NOTE: This is formatted for readability. I used line breaks instead of spaces for clarity.

mike$	/Users/mike/Documents/FlexDocs/flex_sdk_3/bin/mxmlc 
-source-path= ~ /deft/trunk/
HelloWorld.as 
-output= ~ /deft/trunk/HelloWorld.swf

If successful, the last line will print HelloWorld.swf. If that’s not there, you’ll be shown an error. The most common error at this stage of setup is about the path conforming to the class name - but other common errors involve naming conventions and incorrect syntax.

Shell Script

That was a heck of a command to be typing. The simplest thing to do is put this in a shell script. This way, more complicated scripts can be written, but more importantly, this long command can be saved to disk. Here are the steps for anyone not familiar with shell scripts:

1. Copy and paste the previous command into a text file.
2. Save the text file in the same path location as the HelloWorld.as file, and name it HelloWorld.sh (in Windows it would be .BAT). Make sure it was saved with that extension, and not something like “HelloWorld.sh.txt”.
3. You may have to change the permissions to make it executable. Go back to your Terminal (if it’s still open, we should still be in the right directory) and enter: chmod 777 HelloWorld.sh

Shell scripts are protected from external attacks by requiring you to use a period slash before your script name. So to execute the script, type:

./HelloWorld.sh

Now it’s easier to maintain that long command. You can of course add more options to it, or loop through and compile several files, or any other shell script wizardry you can dream up.

Hello Real World Example

Now let’s look at one of the latest projects in Deft, the Uploader. The fully qualified class name is deft.form.Uploader. Uploaderis the class name, and deft.form is the package to which it belongs. It’s very similar to Dojo’s package system - the compiler reads the import of a deft.form file, and expects to look in a deft folder, and then a form folder. In our HelloWorld example, there was no package name - so the compiler expected to find the file in the same folder where it resided.

deft.form.Uploader is the main file and imports two other custom files, deft.form.UploadList and deft.common.debugging.Tracer. Tracer is a multi-output debugging tool, which can parse and log objects and messages to the Output panel in the Flash IDE, the Flash Player Debugger, and/or the Firebug console.

You’ll find that the shell script, “uploader.sh” is located in the trunk, the same level as the deft directory. You’ll see how the source-path option points to the location of the shell script, and the path to Uploader.as is continued from there. Finally, the output points to a location where the SWF will be used in dojox.form, in the resources folder.

You won’t have to go through this build process to use dojox.form.FileInputFlash. All you have to do is get the latest Dojo nightly build or wait for the 1.2 release, and the SWF will be in the resources/ folder. The Deft files are there in the event that a developer needs to make custom modifications for a project, or permanent modifications, and optionally contribute them back to Dojo.

Those are the basics for SWF compilation. The Deft project is still in its infancy, but there are currently a few goodies in it including the multi-image uploader, pre-alpha quality support for dojox.gfx. Future plans include support for audio and video. Hopefully Adobe will continue its current path towards being open source friendly, helping Deft flourish.

The upcoming Dojo 1.2 release features a new comprehensive framework for building secure mashups with the dojox.secure project. This new project includes all the components necessary for safely loading untrusted code, advertisements, content, and widgets from other domains, validating that it is safe to execute, and providing a sandboxed environment and controlled subset of the DOM to interact with. Using traditional means, loading scripts from other domains is a great security vulnerability, every script has full access to the JavaScript environment and DOM of the source page. The alternate approach has been to use iframes, which greatly limits UI integration and programmatic interaction. With dojox.secure, untrusted scripts and widgets can be safely loaded directly into a web page with fine grained sandboxing of their capabilities, while still allowing them to access a controlled subset of the DOM and JavaScript environment.

Cross-Site Loading

The first step in using dojox.secure for loading third-party content is to define a transport mechanism for loading the resources from the target domain. The new XHR Plugin Registry is designed to provide a simple mechanism for defining what cross-domain loading mechanisms are available with the target server, and Dojo supports a comprehensive set of cross-domain loading tools. Most servers currently do not yet support any cross-domain loading mechanism, but Dojo includes transports for those servers lacking cross-domain support. The window.name transport can be used for servers that support the window.name protocol, and native cross-site XHR can be used when browsers add support (this should be available in next major release of the all major browsers). To define that resources should be loaded from siteA.com with the window.name protocol you can register a server:

dojox.io.xhrWindowNamePlugin("http://siteA.com");

However, since most servers do not support these protocols yet, using a simple proxy will often need to be used as the default loading mechansim for dojox.secure. To use a proxy, you simply need to create a very simple request handler that can take a URL, retrieve the resource and return it. You can then register the proxy as the cross-site resource loader:

dojox.io.xhrPlugins.addProxy("/proxy?url=");

Since the window.name transport can be used on all major web browsers, once a server supports the window.name protocol, it is no longer necessary to use a proxy. Note, that you should not attempt to use JSONP for cross-site loading for dojox.secure, because while it works, it is inherently insecure.

Sandboxing the Widget

Once you define the resource loading, creating a sandbox for a script or an HTML snippet is very simple. You choose a DOM element that would be the parent for the widget and load the script using dojox.secure sandbox API.

<div id="sandbox">
  This is the DOM area that the widget can access
</div>

And then create a sandbox:

dojo.require("dojox.secure.sandbox");
var mySandbox = dojox.secure.sandbox(dojo.byId("sandbox"));

Once a sandbox instance is created, you can load and execute JavaScript files and HTML snippets into the sandbox:

mySandbox.loadJS("http://siteA.com/widget.js");

The loadJS function will load the file from the provided URL, validate that it is safe JavaScript, and execute it within the sandbox. The executed script will be limited to only accessing the sandboxed JavaScript environment and the sandbox element and any children nodes or HTML of that element.

You can also load HTML snippet based widgets into the sandbox:

mySandbox.loadHTML("http://siteA.com/widget.html");

The loadHTML function will load the file from the given URL and insert the HTML from the file into the target sandbox element. All scripts within the HTML will also be properly sandboxed as well.

Because the sandbox is actually in the same JavaScript environment, interaction is very flexible. If a script returns a valid JavaScript value when evaluated, this value is returned from loadJS and data and messages can easily be passed back and forth to a sandboxed script through that value/object.

var widgetInstance = mySandbox.loadJS("http://siteA.com/widget.js");
// then call some method on the widget object
widgetInstance.sendGreeting("hello");
// then register to listen for some event
dojo.connect(widgetInstance,"onSomeEvent", function(val){ ... });

Of course, the parent environment can still freely interact with the sandboxed element and any other DOM nodes that the sandboxed content creates.

Building Widgets for dojox.secure

dojox.secure attempts to provide as many of the standard native capabilities of the browser environment and JavaScript language as possible. However, in order to maintain safe, secure, lightweight, high-performance design and avoid massive compilation techniques, there are a number of features that are not available within the dojox.secure sandbox environment. Many of these restrictions are mitigated by a rich set of library functions, but we will first look at the restrictions. dojox.secure is based on the ADsafe dialect of JavaScript, with some minor differences, so the following language features are disabled:

• Use of eval, with, ==, !=, and the subscript operator [] are not allowed.
• The this keyword can not used outside of a class definition
• Global variables from the parent environment are not accessible
• These properties may not be used: apply, arguments, call, callee, caller, constructor, eval, prototype,
  this, unwatch, valueOf, watch, and anything beginning or ending with __.

dojox.secure also prevents the following DOM features:

• Relational traversal though parentNode, firstSibling, nextSibling, parentElement, etc. properties are not allowed. You can still use DOM methods, innerHTML, and style attributes
• CSS does not allow expression, behavior, javascript:, binding, or @import
• link tags are not allowed
• Inline JavaScript within event handler attributes is not allowed.

dojox.secure uses a combination of JavaScript validation and a DOM facade to enforce these constraints. Attempts to use any of the disabled features will fail (usually with an exception, although sometimes silently).

dojox.secure Environment Capabilites

dojox.secure does provide a number of features to not only mitigate the security restrictions, but provide a comprehensive set of features for building widgets. A sandboxed script can access the sandboxed element with the element variable. Node creation and searching can be done with the provided sandboxed document variable. Most of the top level JavaScript functions are available like encodeURIComponent, setTimeout, etc. There are also top level functions get, set, and forEach for iterating and accessing properties by element since the [] operator is restricted, and dojox.secure provides a class constructor mechanism which supports the this keyword within methods. Most of the rich library functionality comes from corresponding functions in Dojo, that are accessible as top level functions (there is no need for namespaces because there is no access to the global object).

dojox.secure defines a subset of the Dojo base library that can be accessed from sandboxed scripts. This includes functions like query, mixin, connect, fromJson, and more. A more detailed description of the API available from within the sandbox, as well as a demo page for testing sandboxed scripts and HTML is available.

An example of a simple widget that can be created for use with dojox.secure:

<div>
	This is <strong>sandboxed</strong> HTML with a script.
	<script type="text/javascript">
		query("b").style("color","red");
		alert("done");
	</script>
</div>

These features allow widgets to be easily built and utilized by web applications from other domains. The API is intentionally designed such that other libraries could implement it as well. This means that you can write a widget for use with a dojox.secure client, but it can also be safely loaded and executed by other client implementations as well.

Conclusion

dojox.secure is a full framework for loading and executing untrusted code, advertisements, and widgets, and provides a powerful yet secure environment for widgets to interact within. Current mashup technology that utilizes direct script loading is inherently unsafe, but with dojox.secure, it is very simple to load widgets and create mashups with proper security enforced while maintaining excellent performance. dojox.secure also provides an excellent platform for building widgets, such that clients can consume your widgets without compromising security.

Several new technologies are coming out for accessing resources from other sites. IE8 will include the XDomainRequest and the W3C is finishing the access control specification for enabling cross-site access for XMLHttpRequest, which will probably be implemented in next major release of all browsers, including Firefox. However, it will certainly take a long time for these new technologies to become pervasive enough to use them reliably and exclusively. Is there any way to leverage this technology immediately when these browsers are released? Yes! With Dojo’s new XHR plugin system, you can start making cross-site requests right away, using the new technologies when available, and a proxy server as a fallback, all while using the familiar dojo.xhr* methods. Your source code can still use this simple API while the registry can handle choosing the appropriate underlying transports for the situation.

Using the XHR Plugin Registry

Suppose we need to securely access a web service from othersite.com. This server can easily support the W3C Access Control specification by including this header:

Access-Control: allow <*>

Now we can use cross-site XHR and XDomainRequest handling for XHR calls to this destination by simply calling:

dojox.io.xhrPlugins.addCrossSiteXhr("http://othersite.com/");

Now on IE8, if you call:

dojo.xhrGet({url:"http://othersite.com/data"})

This request can be handled directly in the browser with XDomainRequest. Similarly in other browsers that implement cross-site capability in the XmlHttpRequest API, this will result in an XHR call.

You can also use the window.name module as an XHR plugin as well. There is a specific module to make this easy to add, so you can simply call:

dojo.require("dojox.io.xhrWindowNamePlugin");
dojox.io.xhrWindowNamePlugin("http://othersite.com/");

Proxy

These plugins will utilize the browser’s cross-site request capability when available, however, you still need a fallback for older browsers. One of the most common techniques for doing this is with a proxy service. To create a proxy service, you simply need to create a resource (servlet, php file, etc.) that can take a parameter and then make a request from the target site and return the results. Once you have created a proxy service, we can add it as an XHR plugin:

dojox.io.xhrPlugins.addProxy("/proxy?url=");

This handles the remaining use cases. Any request to a foreign URL that can’t be handled by the browser will be sent to the local /proxy URL with the url parameter set to the foreign URL. Now if we call:

var deferred = dojo.xhrGet({url:"http://othersite.com/data"});
or
var deferred = dojo.xhr("GET",{url:"http://othersite.com/data"});

This will be handled directly from the browser (which is faster and more efficient) when possible, and will go to through proxy when the browser can’t handle it. Local XHR calls will still be handled by the normal XMLHttpRequest object. If we call:

var deferred = dojo.xhrGet({url:"/myLocalData"})

There will be no difference in behavior, the registry automatically chooses the original XHR handler for this request. Other handlers could be added to the registry as well. For certain applications and servers, it may be possible to use a JSONP (which is unfortunately not secure) handler or iframe proxy as a substitute cross-site XHR handler.

Security Notes

When using a proxy, there may be certain headers from the original request that can be passed on, but generally you should not pass on cookies, as they may have sensitive information from your site. You should also use proper filtering mechanisms, as blindly routing HTML or scripts can lead to cross-site scripting security holes.

HTTP Adapters

Some applications may rely on certain HTTP level meta-data like headers, methods, and status codes. However, not all transports necessarily have access to the HTTP layer in order to define or access this information. Rather than requiring application logic to be modified to transfer meta-data in alternate way, you can define a convention for converting HTTP meta-data to something that is accessible to the underlying transport at the XHR API level, in order to retain this information. Both addCrossSiteXhr and xhrWindowNamePlugin have an optional second argument that can take a function. This function will be called with the default XHR handler as an argument, and should return a function that will be called for each XHR call. You can build a custom HTTP adapter, or the xhrPlugins module provides a default converter that can easily be used:

dojox.io.xhrWindowNamePlugin("http://othersite.com/",dojox.io.xhrPlugins.fullHttpAdapter);

This adapter will automatically convert the HTTP method to a parameter named http-method in the query string of the request URL. It will also add any defined headers to the query string with an http- prefix. Consequently calling:

dojo.xhr("PUT",{url:"http://othersite.com/page",headers:{"Range":"0-20 bytes"}});

Would result in a request URL:

http://othersite.com/page?http-method=PUT&http-Range=0-20%20bytes

You can even build your own HTTP adapter for custom conversions.

Conclusion

The XHR plugin registry allows you to explicitly define and use different transports that are used in different situations based on browser and server support, while still allowing the call-site syntax to remain a simple call to a dojo.xhr function. All this is done through explicit techniques to avoid magic and surprises. This allows for a clean separation of request triggering and transport definition, allowing transports to evolve for cross-domain loading while not affecting the application logic that relies on requests and can remain agnostic to the underlying transport mechanism. The XHR plugin registry will be available when Dojo 1.2 is released, and is available in the current Dojo nightly builds.

Google and Yahoo have JavaScript APIs that let you perform searches. Wikipedia has a JavaScript API that lets you grab data from its pages. These APIs can be accessed cross-domain with a transport method known as JSONP. JSONP works by allowing you add a script tag to your page which points to a URL on their server. The server outputs JavaScript that will call a method (defined as part of the query string in the URL), passing it JSON-formatted data.

You’ll notice that these services are read-only. I don’t currently know of any cross-domain JavaScript APIs that allow you to write data in any meaningful way. An example of this sort of data would be a way, through JavaScript, to update your status on a social networking web site.

Is this lack of functionality simply because developers haven’t taken the time to implement it? To some extent, yes. Almost all of the sites where this would be useful haven’t implemented a JavaScript API at all. At the same time, this has been a pretty tough feature to implement in a way that both protects the user while still making it a relatively painless process.

Right now, several sites have APIs that work through HTTP using HTTP Basic Authentication (e.g. Twitter). If a site were to try to utilize these HTTP APIs through JavaScript, they would have to use their server as a proxy. And in so doing, you would have to trust the server with your username and password.

Why not just check to see if they’re logged in?

This may be better explained through an example. You work for Gaskets, Inc., and your supplier has just sent you an email announcing their new JavaScript API. They explain that in addition to searching their inventory, the API will allow customers to view their order history. You only need to be logged in to your suppliers site, and the API will work from any other site.

Gaskets, Inc. decides that they don’t currently have the resources to use this API. But meanwhile, over at Evil Gizmos, Ltd., a clever (evil) engineer has decided that the API could prove very valuable to the company.

You see, every time someone visits the home page of Evil Gizmos, Ltd., some JavaScript on the page calls the getOrderHistory API function on the supplier’s site. During your day working at Gaskets, Inc., you browse over to Evil Gizmos, Ltd. to see what they’re up to. When you visit, their site pulls down the complete order history of Gaskets, Inc. and saves it on their server.

How did this happen? Well, your supplier was so excited about providing this new functionality, that they ignored the warnings from their engineers about security. The evil engineer at Evil Gizmo’s Ltd. noticed that, in the email, the only mention of authentication was that you were logged in to their site.

JSONP works by adding a script tag to a page to load data from an external site. Adding a script tag is simply another method of making an HTTP request. When an HTTP request is made by the browser, no matter what site it originates from, all your cookies related to that domain are sent to that site, as part of the official HTTP specifications. If you were logged in to that site, your cookies are what authenticate you and restore your session. No matter what site adds this script tag, the server will always think that you’re logged in. A malicious callback function can accept this data, and use Ajax to save it back to their site.

Just being logged in isn’t enough

First of all, if all you are verifying when sending sensitive data through JSONP is that a user has logged in, stop right now. Does this mean that you’ve just lost the functionality you were hoping to provide? Not totally, the simplest fix is to add a configuration screen to your site that lets a user specify what sites they trust. Then, when you receive an API request, after verifying that a user is logged in, you can compare the referrer specified in the HTTP request against the user’s list of authorized sites.

While this patches the security problem, it requires extra work on the user’s part. Now, in order for someone to use this API, a user must be logged in to the API site first, as well as granting the third-party site API permission.

What should it look like?

Everything should originate from the site you visit, and you shouldn’t be required to leave the page in order to perform your authentication or authorization.

Nowhere should the site you’re visiting be able to access your username and password. And if you arrived at the site already authenticated, you should explicitly authorize the site to have access to that site’s API.

Is this possible?

Really only one possibility exists that fits our requirements. The iframe tag will allow us to nest a page from any URL in a way that obeys same origin policies.

An iframe presents several important properties: It allows you to nest a third-party site in the page, it allows multi-page interaction with the third-party site, and it has security restrictions that prevent the parent and child frames from accessing any data in the other. But most importantly, a single property exists that can transport data between documents: window.name.

Let’s quickly go over the mechanisms that make the exchange of data possible. First, in order for the parent frame to read the window.name property of the child frame, it needs to contain a document in the same domain. During authentication, while the iframe is pointed at a different domain, we have no way of reading the window.name property. Nor can we determine when a user is done authenticating. It’s up to the authenticating server to decide how many steps the user takes in order to log in. For example, if the authenticating server allows a username and password to be entered in the iframe, the user might mistype their password before getting it correct. So we’ll need to make sure that the server has a way to return the iframe to a state in which window.name can be read when the authentication and authorization phase is complete.

We start out by creating an iframe containing an HTML file in the same domain. This file sets window.name to a location that the authenticating server can use to redirect to after authentication and authorization. Once this is done, the iframe gets redirected to the third-party authentication page. On this page, the server can do anything it wants to confirm authorization, but it will ultimately output a script that grabs the redirect location from window.name, rewrites window.name with some information we’ll get to in a second, and then, using window.location, redirects back to the original site. After redirection, using this new value in window.name, authorization can be confirmed, and using the JSONP API in an authenticated way is now possible.

Throughout the whole process, everything you did inside of the iframe was invisible to the originating site.

The library: xauth

In addition to support within the forthcoming Dojo 1.2 release, I’ve written a tiny library that provides a lightweight API and the local file that should be in the iframe when the page loads.

<div id="wrapper">
	<iframe src="js/xauth/blank.html"></iframe>
</div>

<script type="text/javascript" charset="utf-8" src="js/xauth/xauth.js"></script>
<script type="text/javascript" charset="utf-8">
	window.onload = function(){
		xauth.init({
			node: document.getElementById("wrapper"),
			url: "http://gaskets.inc/api/approval.php"
		}).addCallback(function(status, token, node){
			alert([status, token, node]);
			node.style.display = "none";
		});
	};
</script>

Breaking it down

We have a wrapper node, which would typically contain an item on the sidebar of your page with a header that makes it clear what the iframe contains (eg. “Authorize Gasket Supplier”). Because we probably want to destroy this node once the user is authorized, it makes sense that we would pass this “wrapper” node to the init function. The function is smart enough to look for the first iframe in the DOM beneath this passed node, and work with that.

Along with that node, we need to specify where to send the user to for authorization.

We’re only adding a single callback here via addCallback, though many can be added. An integer indicating status, a token, and the wrapper node are passed to the function. The status and token will make sense once we cover the server side of things.

The server side of things

I won’t tell you how to write your server-side code (though the Google Code project has some examples. But the important stuff looks like this:

<?php if($authorized): ?>
<html>
<script>
var redirect = window.name;
window.name = "<?php print $token; ?>";
window.location = redirect + "#xauth=1";
</script>
</html>
<?php endif; ?>

Adding the hash xauth=1 is a way to pass status to the callbacks. I like to use 1 to indicate success and 0 to indicate failure, but the codes are up to you and you’re free to give them any meaning that you want.

You should always change the value of window.name, and whatever value you set it to will be passed as the second parameter to the callback. This is useful for providing a token to the user. The idea is that, although you can save a list of trusted sites for a given user, if you would prefer the authorization to take place for every page rather than every site, you can simply create a unique token (bound to the user’s session) that you expect to be passed in any future API calls. But this is completely optional.

Displaying the login form

Though you are free to place a login form inside of the iframe, this might freak some users out. Users that are particularly security conscious, not being able to easily see the URL of the iframe, will likely give up, leave, and not use your service. An easy solution to this is to present a link that will pop up a small login form on your site that automatically closes on successful login, and a “Continue” button that will finish the process up in the iframe.

It’s tempting to worry that using a login form inside of an iframe makes your users more susceptible to phishing attacks. Ultimately, any site could put any form inside of an iframe claiming it’s your login form, with or without your approval. You are slightly better off by not having your login form inside of an iframe because your users will become accustomed to you never having a login form embedded in another page, but there’s really no way to “fix” another site claiming that a user can log in through a form on their page.

“I don’t think I can trust this newfangled technology”

Well, old-timer, none of this is new. Although it might be better said that all the security policies that browser enforces remain in effect using this technique. In fact, the only data that gets exchanged between the two sites involved in the transaction is the data that each site explicitly assigns to what is technically nothing more than a shared variable.

• Your cookies cannot be read or modified by the third-party site
• Your HTML is only available to your pages
• You will always get a reliable HTTP referrer value
• Your server handles all authentication, period. You can theoretically tell the third-party site that the user is authenticated even if they aren’t. Simply telling them they have authentication doesn’t mean it’s true

The “trick” to this technique is simply utilizing window.name to tell the server where the iframe should be returned to once the authorization is complete, and to use it again to tell the client whether the user approved or denied the authorization request. Everything that matters as far as security goes is up to your server, and there are plenty of options to choose from.

Summarizing server options

• User does not have a session (is not logged in) with the API site:
  • A form is presented for the user to enter their username and password:
    • On successful login, window.name gets set, iframe is returned to originating site.
    • Optionally, the HTTP referrer might be saved for later automatic authorization.
  • A link is presented along with a “Continue” button:
    • When the user clicks the link, a new window pops up with a login screen.
    • On successful login, window.close(); returns the user to the previous window.
    • Pressing the continue button confirms a valid session window.name gets set, iframe is returned to originating site.
    • Optionally, the HTTP referrer might be saved for later automatic authorization.
• User has a session (is logged in) with the API site:
  • User has a list of authenticated sites:
    • If the HTTP referrer is in the list, window.name gets set, iframe is returned to originating site.
    • If the originating referrer is not in the list, authorization is requested. Depending on whether “Yes” or “No” is chosen, window.name will likely start with a different status code. In either case, iframe is returned to originating site.
  • If the user does not have a list of authenticated sites, a random token is generated, associated with the user’s session, and placed into window.name after the status code. The iframe is returned to the originating site and future API calls expect to see this token present.

In action

Check out our fictional third-party site that uses Zone Products’s API, and follow the login instructions.

After you’re logged in, visit Sketchy Site, Inc.to see that just because you’ve logged in, a site still needs your explicit approval for you to continue.

The Persevere server is an open source JSON application and storage server. Persevere pairs well with Dojo; the Dojo Data paradigm has a strong correlation to Persevere’s data storage structure. Virtually everything you can create or action you can perform with Dojo Data can be persisted in Persevere’s data storage including subobjects, arrays, circular references, and functions. Combining Persevere with Dojo allows you to rapidly develop simple database applications with create, read, update, and delete (CRUD) capabilities with minimal effort.

Starting Persevere

To get Persevere running, first download the latest version of Persevere. Next, unzip the Persevere zip file into the directory you want to use for Persevere. Finally, go to the directory where you unzipped Persevere and run:

java -jar start.jar

Persevere should start up as long as Java is installed on your system. You can now access Persevere in your web browser by visiting http://localhost:8080/.

Creating a Persevere Class

The basic container for storage in Persevere is a class; analogous to a table in relational databases. A Persevere class holds object instances, and class style structures can be defined for the object instances including methods and type definitions for the object properties. To create a Persevere class, first open the object browser at http://localhost:8080/browser.html?id=root. The object browser allows you to browse and navigate the objects in Persevere. To create a class, click the button (make sure no object is selected). A dialog will ask you for the name of your class and what class to extend. We will call the class “Product” and we will accept the default base class of “Object”. You now have data storage structure which can hold Product object instances.

Connecting with Dojo

Persevere is compliant with the HTTP/REST protocol, so the JsonRestStore is very effective with Persevere. However, dojox.data.PersevereStore is an extension of JsonRestStore specifically designed to simplify connecting to Persevere and take advantage of extra features of Persevere. This architecture is designed for a data store instance per server table. With the PersevereStore we can easily get a set of data stores corresponding to the server tables/classes:

dojo.require("dojox.data.PersevereStore");
var deferred = dojox.data.PersevereStore.getStores();

This deferred object will provide an object map of the stores when finished. We can get the data store for our Product table:

deferred.addCallback(function(stores){
	productStore = stores.Product;
});

The data store can be used with various widgets that support data stores. For example to use this table with the Dojo Grid:

grid.setStore(productStore)

or you can set the productStore as the store in the constructor. For example:

grid = new dojox.grid.Grid({
	store: productStore,
	query: "",
}, dojo.byId("gridId"));

With the ability to define any type of object structure, a Persevere data store can be used with virtually any widget. With the referencing capabilities it can be used hierarchically, so it is also well suited for use with the Tree widget.

Create, Read, Update, and Delete

Via Dojo, interacting with Persevere can be done directly with the Dojo Data API. To create our first object instance in the Product table:

paintballGun = productStore.newItem({name:"Paintball Gun",price:129.99});
productStore.save();

We can easily read properties using directy property access or the Dojo Data API:

paintballGun.price -> 129.99
productStore.getValue(paintballGun,"price") -> 129.99

We can update properties:

productStore.setValue(paintballGun,"price",119.99);
productStore.save();

Note that we do not need to define any type of schema, columns, or structure for a table ahead of time, we can simply dynamically create properties on objects that we create.

And delete the objects as well:

productStore.deleteItem(paintballGun);

One of the powerful aspects of Persevere is that you can dynamically persist a large variety of data structures. For example, we could save a sub object with manufacturer information:

productStore.setValue(paintballGun,"manufacturer",{
	name:"HotShots",
	rating:4,
	started: new Date(Date.parse("Jul 09 2002")),
	topProduct:paintballGun
});

In this example, we are saving an object as a value, and even including Dates, and a circular reference (back to the product object). Virtually anything you can create with JavaScript can be persisted to Persevere.

Querying

An essential aspect of database interaction is querying. Here we can use the standard query-object convention used by Dojo Data to search for objects. A query object is an object with name-values corresponding to the search filters to apply (and wildcards are supported). For example to find all the objects with a name that starts with “Paintball”:

productStore.fetch({
	query: {name: "Paintball*"},
	onComplete: function(results){
		results[0] -> paintballGun
	}
});

We can also utilize sorting and paging as well. To sort by price (lowest to highest) and return items 20-29:

productStore.fetch({
	query: {name: "*"},
	sort: [{attribute:"price",descending:false}],
	start: 20,
	count: 10,
	onComplete: function(results){
		...
	}
});

Persevere also supports JSONPath/JSONQuery queries, which have a much greater level of expressibility and can be used for more sophisticated queries. For example to search for all the items with a price less than $100 or a rating greater than 3:

productStore.fetch({
	query: "[?price<100 | manufacturer.rating > 3]",
	onComplete: function(results){
		...
	}
});

Live Data with Comet

Dojo and Persevere support HTTP Channels which allows for addition of live data updates through Comet notifications. Adding Comet capability is simple, just add the HttpChannels module after the PersevereStore has been loaded:

dojo.require("dojox.data.PersevereStore");
dojo.require("dojox.cometd.HttpChannels");

This is all that is necessary. Dojo will automatically subscribe to all data that is accessed, notifications will be delivered to the client, and the notifications will result in cached data updates and Dojo Data API notifications (which will update the user interface on notification aware widgets like the Grid). No additional coding is necessary to utilize live data updates.

Setting up Security

Out of the box, security is disabled in Persevere to make it easier to begin development. However, security is a key feature of Persevere, allowing Persevere to be safely used directly on the web, accessible from your JavaScript environment. Prior to deployment you should of course enable security, and it is also enabled once the first user account is created. To create a user account, go to the object browser (http://localhost:8080/browser.html?id=root), click on the sign in button , and choose to create a new user. This first user is given the administrative privileges for the system. All data is accessible to this user, but is read only by default for all other users and public access. Additional users may be created by the same process (or programmatically), but subsequent users must be granted access to data.

To enable higher level access for the public or other created users, you can select an object or table and click on the grant access button in the object browser. You can then enter “public” for public access, or a user name to enable write access to the data.

Conclusion

You can see these simple data interaction techniques in sample code in the customers example included in the Persevere download (which can be viewed locally at http://localhost:8080/examples/customer.html) There are numerous other important capabilities of Persevere including server side JavaScript execution through JSON-RPC, schema definition, prototype objects, cross-site referencing, accessing existing SQL databases, and more that are described in the Persevere documentation and we will explore in future tutorials. You should now have enough information to get a quick start to building database applications with Persevere using the oft-used create, read, update, delete, and querying operations, and easily plugging into Dojo widgets, with support for live data updates. You can start building applications almost instantly; no need to create schemas or table definitions ahead of time, simply start creating and building dynamic persistent objects in your data stores, and connect them to widgets for display. You can build your entire application, client and server, in JavaScript.

The window.name transport is a new technique for secure cross-domain browser based data transfer, and can be utilized for creating secure mashups with untrusted sources. window.name is implemented in Dojo in the new dojox.io.windowName module, and it is very easy to make web services available through the window.name protocol. window.name works by loading a cross-domain HTML file in an iframe. The HTML file then sets its window.name to the string content that should be delivered to the requester. The requester can then retrieve the window.name value as the response. The requested resource never has access to the requester’s environment (JavaScript variables, cookies, and DOM).

Dojo API

To use the window.name transport, you can use dojox.io.windowName’s single function, send, with an API very similar to dojo.xhr:

dojox.io.windowName.send(method, args);

The method parameter can be GET or POST. The args parameter is an object that provides the target URL and other information per the Dojo ioArgs API. When you call dojox.io.windowName.send, it will send the specified request and return a dojo.Deferred object, which you can listen to for the response. For example:

var deferred = dojox.io.windowName.send("GET", {url:"http://somesite.com/resource"});
deferred.addCallback(function(result){
  alert("The request returned " + result);
});

Making Web Services Available with window.name

In order to implement window.name with web services (REST or RPC), the server should simply look for requests that include a parameter windowname. If the windowname parameter is included, the server should respond to the request with an HTML document that sets its window.name to the string that is to be delivered to the client. For example, if a client makes a window.name request like:

http://othersite.com/greeting?windowname=true

And if the server wants to respond to the client with Hello, it should return an html page:

<html>
<script type="text/javascript">
window.name="Hello";
</script>
</html>

The value returned to the client will be Hello. One can easily transfer JSON data as well:

<html>
<script type="text/javascript">
window.name='{"foo":"bar"}';
</script>
</html>

The client will than receive the JSON data as a string which can then be parsed with a JSON parser like dojo.fromJson. On the client side, it is highly recommended you use a JSON or secure JavaScript validator if you want to prevent arbitrary code execution and unrestrained access to your environment from the target web service data. To securely parse the JSON, you can test the JSON with the dojox.secure.capability validator prior to calling fromJson:

var deferred = dojox.io.windowName.send("GET", {url:"http://somesite.com/resource"});
deferred.addCallback(function(result){
  // capability.validate will throw an error 
  // if there is unsafe script code in the JSON
  dojox.secure.capability.validate(result,[],{});
  console.log("received object", dojo.fromJson(result));
});

Writing extensive multi-line JSON objects in a quoted string can be rather difficult and error-prone if you are manually creating resources. You can use this template HTML to easily create JSON data that will be delivered as a JSON string without having to manually escape the JSON as a string:

<html>
<script type="\'text/javascript\'">
  window.name = document.getElementsByTagName("script")[0]
    .innerHTML.match(/temp\s*=([\w\W]*)/)[1];
  temp= {foo:"bar", // put json data here
 
 
 
 	baz:"foo"}
</script>
</html>

Likewise, if you want to deliver HTML/XML data, here is a template for doing so without manually putting all the data in a string:

<html>
<body>
<p id="content">
some <strong>html/xml-style</strong>data
</p>
</body>
<script type="\'text/javascript\'">
 window.name = document.getElementById("content").innerHTML;
</script>
</html>

This module has been tested on Firefox 2 and 3, Safari 3, IE 6 and 7, and Opera 9. You can see a simple test/demo page that loads data using the window.name protocol. By default, this demo loads data from our Persevere server, which now supports this protocol.

Benefits

This technique has several advantages over other cross-domain transports:

• It is secure, JSONP is not. That is, it is as secure as other frame based secure transports like fragment identifier messaging (FIM), and Subspace. (I)Frames also have their own security issues because frames can change other frames locations, but that is quite a different security exploit, and generally far less serious.
• It is much faster than FIM, because it doesn’t have to deal with small packet size of a fragment identifier, and it doesn’t have as many “machine gun” sound effects on IE. It is also faster than Subspace. Subspace requires two iframes and two local HTML files to be loaded to do a request. window.name only requires one iframe and one local file.
• It is simpler and more secure than Subspace and FIM. FIM is somewhat complicated, and Subspace is very complicated. Subspace also has a number of extra restrictions and setup requirements, like declaring all of the target hosts in advance and having DNS entries for a number of different particular hosts. window.name is very simple and easy to use.
• It does not require any plugins (like Flash) or alternate technologies (like Java).

How does it work?

Of course you can use the new windowName module without understanding how it works, but since this is a tool for protecting against miscreant web services, you may wish to understand how it provides protection. name is a property of the global/window object in the browser environment, and the value of the name property remains the same as new pages are navigated for the frame. By loading a resource in an iframe where the target page will set the name property for its frame, this name property value can be retrieved to access the information sent by the web service. The name property is only accessible for frames that are in the same domain. This means that Dojo must navigate the frame back to the origin domain after the remote web service page has been loaded, in order to access the name property. This same-origin policy also protects against other frames from accessing the name property while Dojo is retrieving the value. Once the name property is retrieved, the frame is destroyed.

At the top level, the name property is not secure, any information put in the name property is available for all subsequent pages. However, the windowName module always loads resources in an iframe. The iframe is destroyed once the data is retrieved, or when you navigate to a new page at the top level, so other pages never have access to the window.name property.

The principle vector of attack is for other frames to attempt to access the loading frame and navigate that frame to their own domain in order to access the name property (using the same technique that Dojo does). However, navigating frames that are not child or parent frames is not permitted in most browsers, and therefore the third party frames are blocked from this action by the browser. Only the main frame that is loading the resources can access this information. Unfortunately, Firefox 2 does not block this action. Consequently, the windowName module uses a set of three nested frames, where 1st frame blocks all frame traversal to the 2nd frame using dynamically installed getters that return null. This means that third party frames can never traverse the frames to get a reference to the 2nd or 3rd frames, and consequently can never induce navigation of the target frame (the 3rd frame) in order to access the name property. Same-origin security prevents the third party frame from removing the installed getter that protects access to the inner frames as well. These measures protect against attacks and ensure that data can be delivered securely.

The idea for the window.name transport is based Thomas Franke’s library for doing session variables with window.name, but this obviously has a completely different goal.

Conclusion

The next generation of browsers will most likely include native capabilities for securely accessing resources from other sites, however it is extremely valuable to have a safe, efficient means for loading data from other domains with current browsers, in order to build secure mashups. The window.name transport provides an efficient secure mechanism for loading data and can play an important role as a foundation for client-side mashups for current and legacy browser technology. The new Dojo windowName module is an easy to use tool for leveraging this transport, and handles the cross-browser issues and protects against the different attack vectors so you can safely utilize this protocol. The Dojo windowName module is available in nightly Dojo builds, and will be available with Dojo 1.2.

Prior to the popularity of the web, client/server applications often involved the creation of native applications which were deployed to clients. In this model, developers had a great deal of freedom in determining which parts of the entire client/server application would be in the client and which in the server. Consequently, very mature models for client/server development emerged, and often well designed optimal distribution of processing and logic could be achieved. When the web took off, the client was no longer a viable application platform, it was really more of a document viewer. Consequently the user interface logic existed almost entirely on the server. However, the web has matured substantially and has proven itself to be a reasonable application platform. We can once again start utilizing more efficient and well-structured client/server model design. There are certainly still technical issues, but we are in a position to better to build true client/server applications now.

The client/server model can be categorized into three parts:

• User Interface
• Business or Application Logic
• Data Management

Traditional web application development has distributed the implementation of the user interface across the network, with much of the user interface logic and code executed on the server (thin client, fat server). This has several key problems:

• Poor distribution of processing - With a large number of clients, doing all the processing on the server is inefficient.
• High user response latency - Traditional web applications are not responsive enough. High quality user interaction is very sensitive to latency, and very fast response is essential.
• Difficult programming model - Programming a user interface across client/server is simply difficult. When every interaction with a user must involves a request/response, user interface design with this model is complicated and error prone. The vast number of web frameworks for simplifying web development testifies to this inherent difficulty. Some have mitigated this difficulty to some degree.
• Increased vector of attack - Unorganized mingling of user interface code with business code can increase security risks. If access rules are distributed across user interface code, as user interface code grows and evolves, new vectors of attack emerge. With mixed code, new user interface features can easily create new security holes.
• Heavy state management on the servers - When client user interface state information is maintained by the server, this requires a significant increase in resource utilization as server side sessions must be maintained with potentially large object structures within them. Usually these resources can’t be released until a session times out, which is often 30 minutes after a user actually leaves the web site. This can reduce performance and scalability.
• Offline Difficulties - Adding offline capabilities to a web application can be a tremendous project when user interface code is predominantly on the server. The user interface code must be ported to run on the client in offline situations.
• Reduced opportunity for interoperability - When client/server communication is composed of transferring internal parts of the user interface to the browser, it can be very difficult to understand this communication and utilize it for other applications.

With the massive move of development to the web, developers have frequently complained of the idiosyncrasies of different browsers and demanded more standards compliance. However, the new major shift in development is towards interconnectivity of different services and mashups. We will once again feel the pain of programming against differing APIs. However, we won’t have browser vendors to point our fingers at. This will be the fault of web developers for creating web applications with proprietary communication techniques.

Much of the Ajax movement has been related to the move of user interface code to the client. The maturing of the browser platform and the availability of HTTP client capabilities in the XMLHttpRequest object, have allowed much more comprehensive client side user interface implementations. However, with these new found capabilities, it is important to understand how to build client/server applications.

So how do you decide what code should run on the client and what should the run on the server? I have mentioned the problems with user interface code running on the server. Conversely, running business logic and/or data management on the client is simply not acceptable for security reasons. Therefore, quite simply, user interface code is best run on the browser, and application/business logic and data management is best run on the server side. We can take a valuable lesson from object oriented programming to guide this model. Good OO design involves creating objects that encapsulate most of their behavior and have a minimal surface area. It should be intuitive and easy to interact with a well designed object interface. Likewise, client and server interaction should be built on a well-designed interface. Designing for a modular reusable remote interface is often called service oriented architecture (SOA); data is communicated with a defined API, rather than incoherent chunks of user interface. A high quality client/server implementation should have a simple interface between the client and server. The client side user interface should encapsulate as much of the presentation and user interaction code as possible, and the server side code should encapsulate the security, behavior rules, and data interaction. Web applications should be a cleanly divided into two basic elements, the user interface and the web service, with a strong emphasis on minimal surface area between them.

An excellent litmus test for a good client/server model is how easy is it to create a new user interface for the same application. A well designed client/server model should have clearly defined web services such that a new user interface could easily be designed without having to modify server side application logic. A new client could easily connect to the web services and utilize them. Communication should be primarily composed of data, not portions of user interface. The advantages of a clean client/server model where user interface logic and code is delegated to the browser:

• Scalability - It is quite easy to observe the significant scalability advantage of client side processing. The more clients that use an application, the more client machines that are available, whereas the server processing capabilities remain constant (until you buy more servers).
• Immediate user response - Client side code can immediately react to user input, rather than waiting for network transfers.
• Organized programming model - The user interface is properly segmented from application business logic. Such a model provides a cleaner approach to security. When all requests go through user interface code, data can flow through various interfaces before security checks take place. This can make security analysis more complicated, with complex flows to analyze. On the other hand, with a clean web service interface, there is well-defined gateway for security to work on and security analysis is more straightforward, holes can be quickly found and corrected.
• Client side state management - Maintaining transient session state information on the client reduces the memory load on the server. This also allows clients to leverage more RESTful interaction which can further improve scalability and caching opportunities.
• Offline applications - If much of the code for an application is already built to run on the client, creating an offline version of the application will almost certainly be easier.
• Interoperability - By using structured data with minimal APIs for interaction, it is much easier to connect additional consumers and producers to interact with existing systems.

Difficulties with the Client Server Model on the Web

There are certainly difficulties with applying the client/server model to the web. Accessibility and search engine optimization can certainly be particular challenges with the web services approach. Handling these issues may suggest a hybrid approach to web applications, some user interface generation may be done on the server to create search engine accessible pages. However, having a central architectural approach based around a client/server model, with extensions for handling search engines may be a more solid and future oriented technique for many complex web applications.

Our Efforts to Facilitate the Client Server Model

SitePen is certainly not alone in working to facilitate client/server architecture. However, since I am familiar with the projects we help create, I did want to mention our approaches to the client/server model:

DWR - From inception, DWR has provided an excellent framework for building client side user interfaces that can easily connect with server side business logic. DWR was years ahead of its time in establishing a framework that encouraged good client/server modeling. DWR has a solid structure for interacting with Java business logic objects. DWR has continued to progress, providing means for bi-directional Comet based communication (Reverse Ajax), and is adding more interoperability capabilities as well.

Dojo Toolkit - It should be obvious that building a good client-side user interface can benefit from a good toolkit, and Dojo has long provided just that. However, Dojo is more than just a library and set of widgets. Dojo provides real tools for building client/server applications. Dojo RPC can provides tools for connecting to web services, and can even auto-generate services based on SMD definitions. Dojo Data provides a powerful API for interacting with a data model. Dojo has lead the way with Comet technology, creating standards around browser based two-way communication. Recently we have built the JsonRestStore which allows one to connect to a REST web service and interact with it using the Dojo Data read and write API. This greatly simplifies the construction of user interfaces by simplifying the user interface-business logic interaction, and encouraging standards-based communication that can easily be used by others. Furthermore, Dojo provides comprehensive tools for robust data-driven applications; even templating can be done on the client instead of the server with Dojo’s DTL support.

The benefits of using standards-based client/server communication has facilitated integration with server frameworks like Zend, jabsorb, Persevere, and interoperability with other frameworks will be coming soon.

Cometd - Cometd provides real-time duplex communication between clients and servers. However, the distinguishing characteristic of the Cometd project is the focus on not only achieving Comet-style duplex communication, but doing so with an interoperable standard protocol, Bayeux. Comet uses a quintessential client/server approach. Any Cometd (Bayeux implementing) server can interact with any Cometd/Bayeux client. One can easily connect various different client implementations to a single server, by using the Bayeux standard.

Persevere - Persevere is a recently launched project, built with this service oriented client/server approach. Persevere is a web object database and application server with RESTful HTTP/JSON interfaces, allowing applications to quickly be built with a database backend that can be directly and securely accessed through Ajax. Persevere is focused on provided a comprehensive set of web services interaction capabilities through standard interoperable communication. Data can be accessed and modified with basic RESTful JSON interaction, clients can invoke methods on the server with simple JSON-RPC, and data can be queried with JSONQuery/JSONPath. With Dojo’s new REST data store, and SMD driven RPC services, Dojo clients can seamlessly build applications and interact with Persevere services using the Dojo APIs. Complex application logic can be added to the persisted objects in Persevere to facilitate building service oriented applications with a straightforward interface to the user interface code on the browser. Persevere is integrated with Rhino, so model and application logic can be written in JavaScript, providing a consistent language and environment for distributing client and server roles.

For more information about any of these open source projects, visit the SitePen Labs

Summary

As the web platform matures, as applications evolve to use more interactive and rich interfaces, and as web services increasingly interact, architecting web applications with an intelligent client/server model will become increasingly important. A properly designed client/server model will provide a foundation for modular, adaptable, and interoperable applications equipped for future growth.

A new data querying tool for has been added to Dojo 1.2. JSONQuery is a new module intended to succeed and improve upon the JSONPath module introduced in Dojo 1.1. JSONQuery provides a comprehensive set of data querying tools including filtering, recursive search, sorting, mapping, range selection, and flexible expressions with wildcard string comparisons and various operators.

JSONQuery provides safe evaluation with language agnostic expressions that prevents arbitrary code execution. It also uses intuitive result-based evaluation that allows successive query operations. Furthermore, the new JSONQuery module provides significant performance improvements, with 20-100x faster execution with the common filter operation on large arrays than the JSONPath module. JSONQuery generally supersets the functionality of JSONPath and provides syntax that matches and behaves like JavaScript where the syntax intersects for maximum ease of use.

Usage API

A JSONQuery can be executed with the following call:

results = dojox.json.query(query,object);

Where query is the JSONPath query to execute and object is the root object or array to query. You can also create a “compiled” evaluation function that can be reused for multiple evaluations by only passing a query string:

evaluator = dojox.json.query(query);

The evaluator function can be then be performed on a query on data by calling it with the data as a parameter:

results = evaluator(object);

In situations where a single query may be executed multiple times, doing a single parse/compilation with dojox.json.query(query) and reusing the returned evaluation function will provide better performance. It is worth noting that dojox.json.query(query,object) is functionally equivalent to dojox.json.query(query)(object).

Query Syntax

JSONQuery evaluations begin with the provided object, which can referenced within queries with $. From the starting object, various operators can be successively applied, each operating on the result of the last operation. You may explicitly begin your query with $, but this is implicitly auto-inserted, so it is not necessary. This allows you to start queries with operators. JSONQuery uses syntax that is similar to JavaScript (with a number of extra operators), so a simple query could be performed like:

data = {foo:"bar"};
results = dojox.json.query("$.foo",data);
results -> "bar"

Query Capabilities

JSONQuery provides all the functionality of JSONPath, and additional query capabilities. By analogy, JSONQuery is to JSONPath, as XQuery is to XPath. In particular, JSONQuery provides the expressive equivalence of XQuery FLOWR expressions for mapping, sorting, and filtering object data. Several of the fundamental operators of JSONQuery follow the same syntax as JavaScript, as well behaving exactly as with JavaScript evaluation:

• .property - This will return the provided property of the object
• [expression] - This returns the property name/index defined by the evaluation of the provided expression

For example, the following JSONQuery will find the foo property of the 2nd item in the provided array (the $ is omitted):

[1].foo

The following operators are new in JSONQuery:

• [?expression] - This will perform a filter operation on an array, returning all the items in an array that match the provided expression. This operator does not need to be in brackets, you can simply use ?expression, but since it does not have any containment, no operators can be used afterward when used without brackets. The following JSONQuery will find all the array items that have a price less than 15:

  [?price < 15]

  And to add a condition for the rating property to be greater than 3 (and omit the brackets):

  ?price < 15 & rating > 3

• [/expression], [\expression], [/expression, /expression] - This performs a sort operation on an array, with sort based on the provide expression. Multiple comma delimited sort expressions can be provided for multiple sort orders (first being highest priority). / indicates ascending order and \ indicates descending order. For example to sort an array by lastName first and then firstName as the second priority:

  [/lastName,/firstName]

• [=expression] - This performs a map operation on an array, creating a new array with each item being the evaluation of the expression for each item in the source array. For example, to create a list of the price value from an array of objects, we could use the query:

  [=price]

  You can also use object literals and and create a new array of objects that with a name and price properties generated from the source array:

  [={price:price,name:firstName + " " + lastName}]

• expr = expr - Performs a comparison (like JavaScript’s ==). When comparing to a string, t