Blog

Sencha Touch O’Reilly Conference App done with WordPress

All Hail the GPL. All Hail Open Source.

I’ve been coding for a long time now, but it wasn’t until I started getting into WordPress plugin development in early 2010 that I began to really appreciate the possibility of making a living creating “Free” software. The trick is that “Free” refers not to the price, but to the code. Releasing code open source gives developers permission to use, learn from, extend and alter the code. This fact came in very handy when, in the tenth and a half hour (or would that be the ten and a halfth?) I stumbled across the O’Reilly Conference App (webkit browsers only), which is included as an example in the open source and quite lovely Sencha Touch.

The Mariposa Folk Festival is a long-time client and I’ve had the opportunity to develop some great software because of the relationship. They have been one of the main driving forces behind the development of Top Quark’s FestivalApp, which has metamorphosized into The Conference Plugin – a WordPress plugin to capture and use information about a Conference, Festival or any other multi-day, multi-scheduled event. This year we all wanted to develop a iPhone app for the festival. Back in January, I had a clear vision of how this would be possible using the data driven by The Conference Plugin and we milestoned it for the beginning of June, letting other more urgent development take precedence. However, personal life played me this spring to the tune of a first house, a second trimester, a busy touring schedule and a new business all in the span of the 4 months leading up to the week before the festival (the afore-mentioned 10.5th hour).

Late one night, a week before the festival, I was surfing around and stumbled onto the Sencha Touch Demo Page. Therein, I saw the O’Reilly Conference App, a Sencha Touch app that did 85% of the things I needed the Mariposa App to do. It was like Dues Ex Machina Sencha. A gift from the open source gods. I do love Sencha Touch, but coming from jQuery land, I still find their API more than a little confusing. Still, in my opinion, it beats out jQTouch for making a more native feeling and powerful app and here I had a chunk of example code that did most of the front-end confusing UI stuff.

The challenge, to which I was able to code a beta before the sun rose and Version 1.0 while that weekend while on tour in Kenora, ON, was to construct a WordPress plugin that rendered the Sencha Touch App.

Here’s how I did it…

Bypassing WordPress Template

I started out my WordPress plugin, ceremoniously dubbed “The Conference App” by copying the entire O’Reilly App demo into a subdirectory of the plugin called the-app and changing the html and js files into html.php and js.php files so I could scriptify them. Though The Conference App plugin is built for WordPress, it needs to render out a Sencha Touch App. Such an app needs to be rendered as it’s own HTML page, from the <!doctype> to the </html>. It order to do that, I needed to take over a permalink. In the settings page of The Conference App you can specify the app’s permalink (i.e. http://mysite.com/my-app). Then, using just three WordPress hooks, the plugin grabs control of this permalink.  (Note: the code included here is from a Beta version of the software.  It’s changed slightly in Version 1.0, but you’ll get the point).

Code Snippet
/** 
 * - adds to the WordPress set of Rewrite rules
 */
add_filter('option_rewrite_rules','conf_app_rewrite_rules');
function conf_app_rewrite_rules($rules){
	global $conf_app_options; 
	$options = & $conf_app_options;
	$conf_app_rules[$options['conf_app_path'].'/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on'; // the app page
	$conf_app_rules[$options['conf_app_path'].'/proposals/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on&'.CONF_APP_DATA_VAR.'=proposals'; // the sessions page
	$conf_app_rules[$options['conf_app_path'].'/speakers/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on&'.CONF_APP_DATA_VAR.'=speakers'; // the speakers page
	$conf_app_rules[$options['conf_app_path'].'/manifest/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on&'.CONF_APP_MANIFEST_VAR.'=on'; // the cache manifest
	$conf_app_rules[$options['conf_app_path'].'/([^/]*)/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on&'.CONF_APP_YEAR_VAR.'=$matches[2]'; // the app page for a specific festival
	$conf_app_rules[$options['conf_app_path'].'/([^/]*)/proposals/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on&'.CONF_APP_DATA_VAR.'=proposals&'.CONF_APP_YEAR_VAR.'=$matches[1]'; // the proposals page for a specific festival
	$conf_app_rules[$options['conf_app_path'].'/([^/]*)/speakers/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on&'.CONF_APP_DATA_VAR.'=speakers&'.CONF_APP_YEAR_VAR.'=$matches[1]'; // the speakers page for a specific festival
	$conf_app_rules[$options['conf_app_path'].'/([^/]*)/manifest/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on&'.CONF_APP_MANIFEST_VAR.'=on&'.CONF_APP_YEAR_VAR.'=$matches[1]'; // the cache manifest for a specific festival

	// I want the CONF_APP rules to appear at the beginning - thereby taking precedence over other rules
	$rules = $conf_app_rules + $rules;

	return $rules;
}

/** 
 * - registers the valid query vars within WordPress
 */
add_filter('query_vars','conf_app_query_vars');
function conf_app_query_vars($query_vars){
	$query_vars[] = CONF_APP_QUERY_VAR;
	$query_vars[] = CONF_APP_DATA_VAR;
	$query_vars[] = CONF_APP_YEAR_VAR;
	$query_vars[] = CONF_APP_MANIFEST_VAR;
	return $query_vars;
}

/** 
 * - performs action if requesting the app
 */
add_action( 'pre_get_posts', 'conf_app_template_redirect' );
function conf_app_template_redirect(){
	// If a plugin download has been requested
	if (get_query_var(CONF_APP_QUERY_VAR)) {
		// Instantiate the containers
		conf_app_load_topquark();
		global $topquark_objects;
		$tq = & $topquark_objects;

		if (!is_a($tq->Festival,'Festival')){
			// No Festival Found, no point in continuing
			return;
		}

		switch(true){
		case get_query_var(CONF_APP_DATA_VAR) != '': $include = 'the-data/index.php'; break;
		case get_query_var(CONF_APP_MANIFEST_VAR) != '': $include = 'the-manifest/index.php'; break;
		default: $include = 'the-app/index.php'; break;
		}

		do_action('conf_app_template_redirect',$include);
	}
}

/**
 * - includes a PHP file and exits
 */
add_action('conf_app_template_redirect','conf_app_include_and_exit');
function conf_app_include_and_exit($include){
	global $topquark_objects;
	$tq = & $topquark_objects;
	include($include);
	exit();
}

This actually grabs control of four permalinks, each of which serves a different purpose. (Note: each of the permalinks below can also be called with a year parameter (i.e. http://mysite.com/my-app/2011) to render with data from that particular festival year).

  • http://mysite.com/my-app – renders the app itself
  • http://mysite.com/my-app/speakers – returns JSONP data for all of the speakers
  • http://mysite.com/my-app/proposals – returns JSONP data for all of the events within the schedule
  • http://mysite.com/my-app/manifest – returns a cache-manifest file that enables the app to go offline

These permalinks lead directly to the appropriate view script within The Conference App plugin, each of which knows how to render its own particular output.  Using this method, I’ve bypassed any WordPress templating for a set of URLs on a path specified on the Settings page.

WordPress Plugin Files Rendering Sencha Touch JavaScript

Of course, we love WordPress and don’t want to bypass it completely.  We want to make use of its hooks to be able to render a customized app based off of settings page for The Conference App and the data within the The Conference Plugin (all of the speaker and schedule information). I very purposefully wanted The Conference App plugin to be built in such a way as to allow other plugins to add pages, change vocabulary and add any other customizations they want. So, I sent just about everything that I render through apply_filters.

Sencha Touch data is handled by Stores and Models. Models model the data and Stores store it. How intuitive. I defined the Models and Stores within the main plugin file and then called them via apply_filters calls within the app JavaScript files (which are, as mentioned, actually PHP files that render JavaScript).

Code Snippet
add_filter('the_conference_app_models','the_conference_app_models');
function the_conference_app_models($Models){
	$Models['Proposal'] = array();
	$Models['Proposal']['fields'] = array('id', 'title', 'url', 'description', 'day', 'time', 'end_time', 'pretty_time', 'date', 'topics', 'room', 'proposal_type', 'speakers');

	$Models['Speaker'] = array();
	$Models['Speaker']['fields'] = array('id', 'first_name', 'last_name', 'name', 'position', 'affiliation', 'bio', 'twitter', 'url', 'photo','website', 'pretty_website', 'proposals');

	return apply_filters('the_conference_app_the_models',$Models);
}

add_filter('the_conference_app_stores','the_conference_app_stores');
function the_conference_app_stores($Stores){
	global $conf_app_options; 
	$options = & $conf_app_options;
	$conf_app_rules[$options['conf_app_path'].'/?$'] = 'index.php?'.CONF_APP_QUERY_VAR.'=on'; // the app page
	$mothership = get_bloginfo('url').'/'.$options['conf_app_path'].'/'.CONF_YEAR.'/';

	$Stores['SpeakerStore'] = array();
	$Stores['SpeakerStore']['model'] = 'Speaker';
	$Stores['SpeakerStore']['sorters'] = 'last_name';
	$Stores['SpeakerStore']['getGroupString'] = do_not_escape('function(r){if (r.get(\'last_name\') == \'\') return \'-\'; else return r.get(\'last_name\')[0];}');
	$Stores['SpeakerStore']['offline_enable'] = true;
	$Stores['SpeakerStore']['proxy'] = array(
		'type' => 'scripttag',
		'url' => $mothership.'speakers',
		'reader' => array('type' => 'json', 'root' => 'speakers')
	);

	$Stores['AboutListStore'] = array();
	$Stores['AboutListStore']['fields'] = array('name', 'card');

	$Stores['SessionListStore'] = array();
	$Stores['SessionListStore']['model'] = 'Proposal';
	$Stores['SessionListStore']['sorters'] = 'time';
	$Stores['SessionListStore']['getGroupString'] = do_not_escape('function(r){return r.get(\'pretty_time\');}');
	$Stores['SessionListStore']['offline_enable'] = true;
	$Stores['SessionListStore']['proxy'] = array(
		'type' => 'scripttag',
		'url' => $mothership.'proposals',
		'reader' => array('type' => 'json', 'root' => 'proposals')
	);

	return apply_filters('the_conference_app_the_stores',$Stores);
}

Then, my Stores.js.php and Models.js.php files, which get included in the main app JavaScript file, render out looking like this (again, this code is from a Beta version, but it’s enough to get the point:

Code Snippet
Ext.regModel('Proposal', {
	fields: ["id","title","url","description","day","time","end_time","pretty_time","date","topics","room","proposal_type","speakers"]
});
Ext.regModel('Speaker', {
	fields: ["id","first_name","last_name","name","position","affiliation","bio","twitter","url","photo","website","pretty_website","proposals"]
});

oreilly.SpeakerStore = new Ext.data.Store({
	model: "Speaker"
	,sorters: "last_name"
	,getGroupString: function(r){if (r.get('last_name') == '') return '-'; else return r.get('last_name')[0];}
	,proxy: {"type":"scripttag","url":"http:\/\/www.mariposafolk.com\/app\/2011\/speakers","reader":{"type":"json","root":"speakers"}}
	,isLoaded: false
});
oreilly.SpeakerStore.on('load',function(){this.isLoaded = true;});
oreilly.OfflineSpeakerStore = new Ext.data.Store({
	model: "Speaker"
	,sorters: "last_name"
	,getGroupString: function(r){if (r.get('last_name') == '') return '-'; else return r.get('last_name')[0];}
	,proxy: {"type":"localstorage","id":"Speaker"}
	,isLoaded: false
});
oreilly.OfflineSpeakerStore.on('load',function(){this.isLoaded = true;});
getSpeakerStore = function(){return oreilly.OfflineSpeakerStore; }
oreilly.SessionListStore = new Ext.data.Store({
	model: "Proposal"
	,sorters: "time"
	,getGroupString: function(r){return r.get('pretty_time');}
	,proxy: {"type":"scripttag","url":"http:\/\/www.mariposafolk.com\/app\/2011\/proposals","reader":{"type":"json","root":"proposals"}}
	,isLoaded: false
});
oreilly.SessionListStore.on('load',function(){this.isLoaded = true;});
oreilly.OfflineSessionListStore = new Ext.data.Store({
	model: "Proposal"
	,sorters: "time"
	,getGroupString: function(r){return r.get('pretty_time');}
	,proxy: {"type":"localstorage","id":"Proposal"}
	,isLoaded: false
});
oreilly.OfflineSessionListStore.on('load',function(){this.isLoaded = true;});
getSessionListStore = function(){return oreilly.OfflineSessionListStore; }

Notice that there are two stores each for the Speakers and Sessions. The first is an online store that talks via a <script> tag to the mothership and the second is an offline version that talks via proxy to LOCALSTORAGE, that wonderful HTML5 addition to the web developer’s arsenal. Elsewhere in the code, it’s set up such that when the Online version loads, it automatically populates the Offline version.

This is just an example of what I ended up doing all over the place in The Conference App – using WordPress aware PHP files to render Sencha Touch JavaScript. This way, I control the Sencha Touch app with WordPress plugins.

JSONP / Sencha Touch / WordPress

JSONP is a method that can be used to do cross-domain data calls. I wanted The Conference App to use JSONP (as opposed to regular ol’ AJAX, which is limited to calling the same domain that the page is loaded from) with the forethought that the app could be turned into an honest to goodness App-Store app, which requires such a cross-domain protocol.

JSONP works differently than AJAX in that it loads a script via a <script type="text/javascript"> tag. The JavaScript that gets rendered can’t simply be JSON data, otherwise the page would do nothing with it, and in fact would spit out a javascript syntax error. To get around this, you wrap the JSON data in a function call. Sencha Touch tells the script the name of the function to call by passing a GET variable called callback (i.e. http://www.mariposafolk.com/app/speakers/?callback=Ext.util.JSONP.callback). The data script builds an associative array of the data and then renders it (json_encoded) wrapped in this callback.

The Sencha side of things is actually already done, with a single line of JavaScript, rendered by a single variable within a WordPress Plugin.

	// This is the PHP variable, which later passes through the filter the_conference_app_the_stores
$Stores['SpeakerStore']['proxy'] = array(
	'type' => 'scripttag',
	'url' => $mothership.'speakers',
	'reader' => array('type' => 'json', 'root' => 'speakers')
);
	// This is the Sencha code that gets rendered
oreilly.SpeakerStore = new Ext.data.Store({
	model: "Speaker"
	,sorters: "last_name"
	,getGroupString: function(r){if (r.get('last_name') == '') return '-'; else return r.get('last_name')[0];}
	,proxy: {"type":"scripttag","url":"http:\/\/www.mariposafolk.com\/app\/2011\/speakers","reader":{"type":"json","root":"speakers"}}
	,isLoaded: false
});

You can see in the Sencha code that the JSONP call calls `http://www.mariposafolk.com/app/2011/speakers` (you can go there now and see what gets spit out). It expects back a JSON Array where the `root` is `speakers` (i.e. `{“speakers”:[{"id":322...},...]}`).

Back in the WordPress PHP script world, using the mechanism described above for bypassing the WordPress template, I redirect the `/speakers` permalink to a script that knows how to build the above JSON. Where it becomes JSONP is that Sencha will actually call the above URL with a `$_GET` variable called `callback`. Callback is a callable JavaScript function that knows how to process the JSON and populate the `Store`. The PHP needs to actually render the script wrapping the JSON in a function call to `$_GET['callback']`. This is what the code to get the speaker list looks like in the WordPress Plugin.

(BTW, you’ll notice that even though the plugin is called The Conference App, the object references are for things like `$tq->Festival` and `FestivalApp`. This is all for historical reasons dating back to when the software was being developed for a Festival).

usePackage('FestivalApp');

	switch(get_query_var(CONF_APP_DATA_VAR)){
	case 'speakers':
		$Lineup = $tq->Festival->getLineup();
		$output['speakers'] = array();
		
		// Getting published to respect the two-stage publishing structure
		$FestivalArtists = $FestivalAppPackage->getPublished('FestivalArtists',$tq->Festival->getParameter('FestivalYear'));
		
		foreach ($FestivalArtists as $ArtistID => $Artist){
			$artist = array();
			$artist['id'] = intval($ArtistID);
			$artist['url'] = $Artist['ArtistWebsite'];
			$artist['name'] = $Artist['ArtistFullName'];
			$artist['first_name'] = $Artist['ArtistFirstName'];
			$artist['last_name'] = $Artist['ArtistLastName'];
			$artist['bio'] = $Artist['ArtistDescription'];
			$artist['website'] = $Artist['ArtistWebsite'];
			$artist['pretty_website'] = preg_replace('/\/.*/','',str_replace('http://','',$Artist['ArtistWebsite']));
			// Find the thumbnail (@TODO, make this a simpler call)
	        $AllMedia = $MediaContainer->getAssociatedMedia(new Artist($ArtistID));
	        if (!$AllMedia) $AllMedia = array();

	        foreach ($AllMedia as $Media){
	            switch($Media->getParameter('MediaType')){
	            case MEDIA_TYPE_GALLERYIMAGE:
	                    $Image = $GalleryImageContainer->getGalleryImage($Media->getParameter('MediaLocation'));
	                    $Gallery = $GalleryContainer->getGallery($Image->getParameter('GalleryID'));
	                    if ($Image){
							$artist['photo'] = get_bloginfo('wpurl').'/'.GALLERY_IMAGE_DIR.$Gallery->getDirectoryName().rawurlencode($Image->getParameter("GalleryImageThumb"));
	                    }
	                    break;
				}
	        }
			$artist['proposals'] = array();
			foreach ($Artist['Shows'] as $ShowID){
				$artist['proposals'][] = array('id' => intval($ShowID));
			}

			foreach($artist as $key => $value){
				if ($value === null){
					$artist[$key] = '';
				}
			}
			$output['speakers'][] = apply_filters('the_conference_app_data_speaker',$artist,$Artist);
		}
		break;
	}

	header("Content-type: text/javascript");
	ob_start();
	if(isset($_GET['callback'])){
		echo $_GET['callback']."(";
	}
	echo json_encode($output);
	if(isset($_GET['callback'])){
		echo ")";
	}
	echo ";";
	exit();
?>

To see what gets rendered, visit `http://www.mariposafolk.com/app/2011/speakers?callback=foo`. Back in Sencha world, `foo` (function names have been changed to protect the anonymous), is a callable function that will then populate the Speaker Store. The code gets included via a `<script>` tag, so `foo({…})` actually gets called.

And that’s how to get the Speaker and Session generated from the WordPress plugin The Conference App into the Sencha Touch App.

Taking it Offline

The piece of the puzzle that wasn’t provided for at all in the O’Reilly Conference App demo code and that I developed myself was the code necessary for enabling the App to work in offline (i.e. Airplane) mode. This is achieved using features of HTML5, namely `LocalStorage` (to store the Speaker and Session data) and Cache-Manifest (to store the images and JavaScript files).

The Cache-Manifest file can be seen at `http://www.mariposafolk.com/app/2011/manifest`. It’s really just a list of files, which gets generated by the WordPress plugin and passed through filters. It specifies all of the images for the speakers and the necessary Javascript files required by the App to run. Once they’re loaded the first time, all of these files get cached on the device. The cache-manifest file then says to load the cached versions. Turns out testing this can be a bit chafing. I haven’t quite mastered how to elegantly handle server updates to these files. The code as it stands just Brute Force loads everything again. It seems there should be a way to load only what’s actually changed, like updated data. As I figure out these bits, I’ll come back and write more blog posts explaining any lessons I learned along the way.

The LocalStorage trick in Sencha Touch is theoretically very simple. However, I ran into a performance problem that I had to workaround by calling `.suspendEvents()` on the OfflineSpeakerStore before writing it out. See the commented code below.

Again, I generate the following Sencha Javascript code within my WordPress PHP plugin scripts. Some of the code you’ve seen already in the above examples, but here it is all together. Basically I create two Stores within Sencha, each of which uses the same Model (see `Ext.regModel(‘Speaker’, {…})`), but uses a different `proxy`. The `proxy` of the Online store (loads from the Mothership) is `scripttag` (discussed above). The `proxy` for the Offline store (stored on the device) is `localstorage`. The Offline store gets populated automatically when the Online store loads. When the app loads on the device, it checks to see if the Speaker store is stored locally already, and if it is, then loads it. Otherwise, it loads the Online store.

loadStores = function(forceOnline){
	if (forceOnline === undefined){
		forceOnline = false;
	}
	
	// Load the oreilly.SpeakerStore Store
	oreilly.SpeakerStore.addListener('load',function(){
		// This line empties the Speakers stored in LocalStorage, if they exist
	    oreilly.OfflineSpeakerStore.proxy.clear();
	
		// this turned out to be very important for performance
		// When I didn't supend events, the action of writing the data
		// into LocalStorage took forever.  Turns out too many events 
		// were being fired.  This fixed the problem
		oreilly.OfflineSpeakerStore.suspendEvents(false);
		
		// For each record loaded from JSONP, add it to the LocalStorage store
	    this.each(function (record) {
	        oreilly.OfflineSpeakerStore.add(record.data);
	    });
		
		// Ooops...  Can't run sync with events suspended.  Have to run it manually
		// I view this as a bug in Sencha Touch.  The code below is copied directly from
		// sencha-touch-debug.js
		var me        = oreilly.OfflineSpeakerStore,
		    options   = {},
		    toUpdate  = me.getUpdatedRecords(),
		    toDestroy = me.getRemovedRecords(),
		    toCreate  = me.getNewRecords();

		options.create = toCreate;
		options.update = toUpdate;
		options.destroy = toDestroy;
		me.proxy.batch(options, me.getBatchListeners()); // Actually does the writing to LocalStorage
		
		// Okay to run events again
		oreilly.OfflineSpeakerStore.resumeEvents();
		
		oreilly.OfflineSpeakerStore.fireEvent('load',oreilly.OfflineSpeakerStore);
	});
	if (!forceOnline && localStorage.getItem('Speaker')){
		console.log('loading offline SpeakerStore');
		oreilly.OfflineSpeakerStore.load();
	}
	else{
		console.log('loading online SpeakerStore');
		if (oreilly.SpeakerStore.getCount()){
			console.log('emptying store');
			oreilly.SpeakerStore.suspendEvents(false);
			oreilly.SpeakerStore.remove(oreilly.SpeakerStore.getRange());
			oreilly.SpeakerStore.resumeEvents();
		}
		oreilly.SpeakerStore.load(); // Will also load the Offline version
	}
}

Ext.regModel('Speaker', {
	fields: ["id","first_name","last_name","name","position","affiliation","bio","twitter","url","photo","website","pretty_website","proposals"]
});
oreilly.SpeakerStore = new Ext.data.Store({
	model: "Speaker"
	,sorters: "last_name"
	,getGroupString: function(r){if (r.get('last_name') == '') return '-'; else return r.get('last_name')[0];}
	,proxy: {"type":"scripttag","url":"http:\/\/www.mariposafolk.com\/app\/2011\/speakers","reader":{"type":"json","root":"speakers"}}
	,isLoadedEh: false
});
oreilly.OfflineSpeakerStore = new Ext.data.Store({
	model: "Speaker"
	,sorters: "last_name"
	,getGroupString: function(r){if (r.get('last_name') == '') return '-'; else return r.get('last_name')[0];}
	,proxy: {"type":"localstorage","id":"Speaker"}
	,isLoadedEh: false
});

Conclusion

Using the mechanisms described above and staying up all night one night, I was able to build a Sencha Touch app rendered by a WordPress plugin that worked offline.
This code is out of context, but if you understand PHP and JavaScript, then you can certainly learn from it, extend it, use it and mash it up into whatever project your working on. The above code is all released under GPL. The Conference App is also GPL. It’s also very reasonably priced, provides context for all of the above and helps me feed my new baby. C’mon. You know you want it.

On the weekend of the Mariposa Folk Festival, we successfully ran version 0.99 of The Conference App. It got well used by many people (accurate stats are forthcoming in a later version). You can see Version 1.0.0 of The Conference App plugin rendering out the 2011 Boston WordCamp schedule. Visit topquark.com/wordcamp/app/2011-boston on an iPhone, Android, Torch or Webkit browser (Safari, Chrome).

Leave a comment if you have any questions about anything you read here.

0


About the Author

Top Quark is Trevor Mills is Top Quark. He is they as you are me and we are all together. He holds an Engineering degree, specializing in high energy physics, he plays bass in the Juno award winning band Digging Roots and he lovely loves his family. Top Quark’s suite of WordPress plugins are just what you’re looking for.

Add a Comment