Creating Computed Macros in ember.js

Apr 27th, 2013

One of the more recent pull requests that has made it into ember.js contains a whole raft of new computed macros for you to take advantage of. But what is a computed macro or for that matter what is a macro?

Wikipedia describes a macro as this:

A rule or pattern that specifies how a certain input sequence (often a sequence of characters) should be mapped to a replacement input sequence (also often a sequence of characters) according to a defined procedure

The attr_accessor class macro in ruby is a perfect example of such a macro. The attr_accessor class macro takes a symbol (input) and generates two new methods (replacement) for get and set operations.

As the title of this post suggests, it is possible to write your own computed property macros in ember.js.

Below are some examples of the computed macros that come out of the box with ember:

On line 3 of the above gist we are creating a short hand alias to another property using Ember.computed.alias.
On line 4, Ember.computed.equal is used to check that a property equals a static value.
On line 5, Ember.computed.not is used to check the inverse of another property.
On line 6, Ember.computed.bool is used to check that a condition is truthy, in this case that a property path actually returns a value.

It is worth remembering that computed properties update themselves when any of the dependent keys defined as arguments in the property declaration change or the result of the computed property remains cached otherwise.

Creating your own computed macro

Let me first illustrate a simple scenario, below is a gist of the ember-data domain model that I will use to illustrate this example:

The above gist contains a Contact model definition that has two hasMany relationships with phoneNumbers and emailAddresses. The PhoneNumber and EmailAddress models are identical and in any given collection of either, there can be only one item of the collection that has isPrimary set to true.

Let us say that we wanted to create a computed property for each collection that selected the isPrimary value and observed any changes to which item had isPrimary set to true.

A very bad way of doing this might be this might be this:

On lines 10 and 16 of the above gist, the same functionality is repeated twice. You can see a working example of this on this jsFiddle.

So how can we DRY this up? By creating a computed macro of course. Below is a listing of creating a primary macro that will transform the instruction set into basically the code above:

We pass the following instructions into the macro:

primaryEmail: App.computed.primary('emailAddresses'),
primaryPhoneNumber: App.computed.primary('phoneNumbers'),

The macro will effectively create the two duplicated computed properties that were originally created. All the macro really does is create the following dependent key:

var dependentKey = hasMany + ".@each.isPrimary";

This dependent key and a function are then passed passed as arguments to Ember.computed that does the clever stuff.

You can find a working example at this jsFiddle.

I need to restrain myself from seeing computed macros as my current hammer with everything looking like a nail but I am sure you must agree that they are insanely easy to create and insanely useful.

Ember.js - Calling a Handlebars Helper From Another Handlebars Helper

Apr 7th, 2013

If you ever have wondered how to call a handlebars helper from another handlebars helper but were unsure how to, then fret no more.

In case you do not know, handlebars is the templating language of choice for the ember.js client side mv* framework. One of the really great things about handlebars is that it is logic free and you cannot litter your templates with gargantuan expressions. We have all been guilty of this with rails erb, haml, jsp, asp, asp.net or whatever and let ye without guilt cast the first stone. Handlebars is the perfect antidote to this simple but very complex erb templating code below:

I came to ember from backbone and I found this logicless approach quite limiting at first but I have grown to love it. It really does make your templates a hell of a lot cleaner and even if you do get tempted to stray from the chosen path, there is not much you can do apart from create a handlebars helper.

If you are familiar with ember then you will have used handlebars helpers extensively and perhaps even without knowing it, for example you have probably used the view helper to insert new instances of a particular view into a template like this:

You might also have used the #if, #unless or #each block helpers. You can also create your own helpers that are evaluated in the context of a handlebars template. In the example I am going to show below, I want to call a helper and pass in a model that contains an image property, the helper will contain all the logic for generating the correct img element for a particular model instance. This logic will also generate a default img if the image property of this model instance has not been defined. Below is a simplified version of how I would call the helper from a template:

This exerciseImage helper is being called in an each block that is iterating over the model of an ArrayController. The ArrayController is made up of exercise model instances and the this on line 3 of the above gist is the current exercise model instance of the iteration loop.

The logic I want to capture is this

If the model has an image property then I want to display it.
If the model has no image property then I want to display the relevant image for that exercise group (e.g. abs, arms, legs, back).
A group is a belongsTo property of the exercise.

I already have a helper that takes a group model and displays the correct image and I want to display the group image if no image exists for that exercise model. In other words, I want to call the existing helper from the new helper. The group image handlebars helper looks like this:

On line 1 we are registering the helper as a bound helper with the registerBoundHelper method. This means that changes to the bound model will invoke the helper again and the relevant part of the template will get re-rendered again. I think you will agree that this is pretty damn powerful.
On line 3, we simply pull the name field of the model and use it to generate the correct image src on line 7.

Below is the handlebars helper for an exercise model instance that will make use of the helper listed above. If the exercise model instance that is the context when this helper is called has no image property, i.e. the user has not supplied one, I want to default to the exercise’s group image. Below is the code that enables this:

On line 4, I test whether a valid image property was retrieved from the model.
On line 5, I create a new set of arguments from the original set that ignores the first argument. The first argument would be the exercise model instance and we do not want to pass that to the groupIcon helper.
On line 6, I push to the front of the new arguments array, the string path to the group model from the current context which is the exercise model instance.
The reason I am pushing the string path as the first argument is because we are simulating how this helper would be called when directly calling it from a handlebars template like this:

The runtime is expecting a path to some property of the context which in this case is group property of the exercise model instance.
On line 7, the javascript function object’s apply method is used:

return Ember.Handlebars.helpers.groupIcon.apply(this, args);

apply is used to ensure that the method call is made with the correct context and with the newly constructed set of arguments that we created with the string path to the group as the first element.

You can also use this technique to apply a layer of indirection to an existing handlebars helper which can be useful to intercept the call and apply some decoration or whatever.

If you have anything to say about this then please leave a comment below.

The Ember Controller - Same Same, but Different - Part 1 - the ObjectController

Mar 24th, 2013

I have travelled around the beautiful country of Vietnam twice and they have a great saying:

Same, same but different

I think this saying is descended from the ember controller. The ember controller is similar in many respects to a controller object that you might find in a server side framework such as rails but at the same time it is quite different.

In my last post I made this statement:

a model can notify the view of any changes via the observer pattern.

As it turns out, the above statement is not entirely true or at least, I purposely omitted some facts. A model does indeed notify the view of any changes but it does so by way of a middle man who sits in between the view and the model. This middleman is the ember.js controller and templates are always bound to a controller of some description. The ember api docs portray this shadowy middleman as this:

In Ember.js, controllers allow you to decorate your models with display logic. In general, your models will have properties that are saved to the server, while controllers will have properties that your app does not need to save to the server.

As always the best way to illustrate this is with some code. Let us look at an example of what would happen if we did not have a controller.

Let us say we had an ember-data Employee model class like this:

The above model has 3 simple fields of firstName, surname and age and a computed property of fullName.

Now let us say that we are working on an application that has one employee and we can change the status of this employee to retired or we can reinstate the retired employee. A very specific and limited application you might say. We might have a view like this to set an employees status to retired:

You will have to forgive the crude html but it is a jsfiddle that you can see in it’s entirety here. You might also have a view that allows you to make that employee active again which would look like this:

Now what if we wanted to capture the fact that we had selected the contact by checking the checkbox? Part of the reason for using a client MV* framework is that we want to deal with nice abstractions to capture what is going on, we don’t want to be mucking about the DOM to check if a checkbox is selected and then tie this back to our beautiful model. We want to use our rich abstraction to capture this check, so we might add an isChecked property to our model as I have done on line 8 of the gist below.

We could then set up a two way binding (refer to my last post) between the isChecked property of the model and the checked attribute of the checkbox like this:

On line 3 of the above gist there is a checkedBinding=”isChecked” declaration that will take care of changing the model’s isChecked property without any DOM manipulation.

Now we can capture whether isChecked is true for this property on the model. All is great, slap yourself on the back, you are a client MVC wizard. But hold on, do not gather your wife and kids around you and tell them of your gallant exploits just yet.

Using the gist below, take the following steps:

make sure you scroll the iframe down to show the 2 links
Click the active link
Check the checkbox
Click home
Click retired
Recoil in horror as the checkbox is already checked. You did not check this checkbox, why is it checked???
Scream that you want to use jQuery and jQuery only as this is now getting complicated.

This is of course because the model instance is shared throughout the whole application and isChecked is a two way binding. When we set isChecked on the model, we are setting it throughout the application. There is of course an answer to this.

Enter the ObjectController

If you cast your mind or eye back to the beginning of this post, I quoted the following from the docs

In Ember.js, controllers allow you to decorate your models with display logic.

Controllers in ember act as proxies for their underlying model. We can then decorate the model with display logic. In this less than perfectly crafted example, the isChecked property is exactly the type of display logic we want to decorate our model with. You can think of the isChecked property as an example of application state that should only exist for the lifetime of the current view and not be persisted along with the model’s long lived properties.

As stated above, controllers in ember are proxies for their underlying template. We can access the model’s properties through the controller because the controller just acts as a pass-through for the model properties. Ember has a number of different flavours of controller, there is the ArrayProxy when your model is a list of models or in this case, there is the ObjectController when you are dealing with a single object. Templates are always connected to controllers and not models.

We will now use the ObjectController to solve this troubling situation. Currently our router looks like this:

We have one resource which is the Employee resource that we are dealing with and two routes that correspond to the two views with the checkboxes, retired and active. Ember is simply brimming with conventions and each url or state change of the application involves a number of collaborating objects that follow the convention xxxRoute, xxxController, xxxView or they will be generated for you to eliminate a lot of the boilerplate code that used to exist. When a url changes in the application or you enter the application at a particular url, ember tries to match the url fragment with a route handler. Ember will then try and find a controller object and a view that match the naming convention or they will be generated for you. As we have not defined an ActiveController or a RetiredController, ember will generate these for us.

The route handler is the place where you set the underlying model for a controller as the route handler’s job is to translate a specific url into a model object. There are a number of hooks that serve this purpose and both routes in the above gist are using the model hook to specify which model is associated with each url. In both cases, we are hardcoding which model is going to be associated with each url. We are now going to change things by specifying a controller for each of these routes that will override the automagically generated controllers. Following the convention of xxxController, we create the following two controllers:

Our work here is done, take the same actions as before on the jsfiddle below:

Our work here is almost done, we can decorate the model with application state that is not persistent outside of the controller’s scope.

Conclusion

This is a fairly stupid example but I hope this clarifies why we need a controller and what its job is. The bottom line is this, templates bind to controllers and controllers are proxies to their underlying model objects. This allows us to pin application state that is contextual to the current view on the current controller and not data that will be saved with the model.

In reality this example would be a list of employees and you would probably attach the isChecked property to an itemController which can be bound to each item on a list.

You can now call your wife and children to gather around to tell them of your brave exploits. They will be amazed.

If you find these posts useful, then let me know what else you would like me to cover.

Ember.js - Client Side MVC Is Not Server Side MVC, Erase Your Brain

Mar 23rd, 2013

I’m writing this in the week ending 23 March 2013. A week that future generations will refer to as hate on ember week. A lot of people are complaining that ember has a barrier to entry and that barrier is the complexity or the amount of new information that the debutant has to take on board. So I thought I’d pen a couple of posts that might help set the ground work for understanding ember.

I think it is very important to grasp this concept before moving on to any of the other concepts in ember such as the router, controller, ItemController etc., etc..

Ember has a multitude of new concepts to take on board and trying to grasp them all at once can be very overwhelming.

Client Side MVC is completely different than Server Side MVC

I am a front end only developer by circumstance and partially because I enjoy coding on the client more. It was not a choice I made. I just worked on progressively more javascript heavy apps.

I have worked with a number of server side mvc frameworks, and as it turns out, these server side incarnations are not actually true mvc in the same sense that the classic smalltalk mvc was. These server side MVC frameworks are better classified as a model2 architecture. Let us look at the server first.

A typical http request/response of a server side mvc framework is this:

There is a single point of entry into the system. All requests are going to come into the system by the browser hitting a route.
The web server process determines which route it belongs to and dispatches that request to the corresponding controller action.
The controller will then retrieve the appropriate model before handing the model or some wrapped viewmodel to the view to do the rendering.
Bottom line is, each GET request usually results in a big blob of something being returned to the browser. That something could be html, json, xml, text or whatever.

On the server, the collaborating MVC objects only exist for the length the http request. Http is a stateless protocol and this constrains the real mvc pattern that was popularised by smalltalk. The views on the server can only receive a request, and dump out some data.

On the client side, there are no such constraints as the objects can live as long as the browser session. What finally triggered off all the right mental associations about client side mvc for me is the following statement:

a model can notify the view of any changes via the observer pattern.

This is why comparing ember to rails is not entirely accurate, ember is a spin off from sproutcore which had a stated aim of bringing OSX’s cocoa to the browser.

The best way to flesh this out is with a practical code example. The gist below contains about the least amount of code I could write to illustrate how a view is updated with changes to the model and vice versa. You can view this code in the following jsfiddle.

First up we have the code that will create a very simple object that we can use to show how these updates are reflected.

On lines 3 to 6 of the above gist, we are creating a very simple object that represents a person.
This Person object has firstName and surname properties.
This Person object is the M for Model of the MVC acronym.
We are creating an ember object and not just a plain old js hash because it needs to be an ember object in order to observe property changes.
On lines 4 and 5 we are passing in a hash of initial values.

Next we have the markup which is written in an ember flavour of the very powerful client side templating language handlebars:

You can think of the template as the V for view of the MVC acronym.
On line 1 we have a handlebars expression. A handlebars expression is enclosed in double curly braces.
The expression on line 1 points to the Person object’s first name property that we created in the previous gist.
On line 2 we are creating a view with the handlebars view helper. We can use the view helper to insert subviews into other views and so aid composability. The view helper takes an object or you can think of it as a path to an object as an argument.
On line 2 we are telling the view helper to render a subview of type Ember.TextField which is one of the out of the box view objects that comes with ember. You might not be surprised to learn that the Ember.TextField renders an input field of type text.

Now we come to the meat and potatoes of the piece thus far and that is the rather peculiar looking valueBinding expression on line 2 of the above gist. I don’t think I am exaggerating by saying:

Do not progress any further with ember until you have fully grasped the concept of bindings. It is the first key checkpoint on your road to Damascus.

Ember has a multitude of conventions and this is the first one. Any ember object property that has the case sensitive suffix Binding attached to it will be treated as a special property by the framework. The valueBinding expression below conforms to this rule. We are setting up a two way communication between two objects with the following expression:

view Ember.TextField valueBinding="App.Person.firstName"

The left side of the valueBinding expression points to a value of the containing object that is the name minus the Binding part. An input has a value attribute and Ember.TextField is an abstraction of the input element, so it also has a value property. The right side of the expression can be thought of as a path to one of our objects. The Person object was created as App.Person and this object has a firstName property.

In short we are wiring together the value of the firstName property of the Person object to the value property of the input. You can confirm this by typing any text into input field of this jsfiddle below:

It is worth noting that the Ember Handlebars templates are binding aware and they pick up these changes automagically without any additional code. Once I learned about bindings, I knew ember was something I should take notice of.

So here we have it, the model is updated from any changes in the view via the observer pattern with no additional code. No hacky pulling the value out of the DOM and updating the model. Leave all that behind you, it does not belong here. The opposite is true also, if the model was updated, the input’s value attribute would be updated.

Computed Properties

If we wanted to display my full name or a combination of the firstName and surname properties of our Person object then we could update the handlebars to look like this.

I could do that but then there would be no point in writing this section. Ember has a special mechanism called computed properties that allow you to create functions that behave like normal properties. Before going any further, let us update the code to create a computed property that is a combination of the firstName and surname properties of the Person object. Below is the updated code:

We are doing things a wee bit differently here than before:

On line 3 we are using Object.extend rather than Object.create. This is the preferred route. You can think of the objects you create with Object.extend as the classes from which you create instances of these classes from.
On line 11 we are creating an instance of the Person class and assigning it to an App.person variable which we will reference in our handlebars.
Lines 7 -9 define the computed property. This fullName computed property simply concatenates the firstName and surname properties into a single output.
The property expression on line 9 that is tagged onto the end of the function definition struck me as very odd the first time I came across it. The property definition on line 9 takes a comma delimited list of string arguments that equate to ember paths. These paths can be thought of pointers to ember object properties.
When we add this property syntax onto the end of a function and supply a list of string arguments, we are telling the ember runtime to observe changes in the properties that these paths point to. In this instance we are stating that whenever anything changes in the firstName property or the surname property then this computed property needs to recalculate and output its result.

We need to update our handlebars template to use the computed property:

On line 1 of the above gist, we are pointing to the fullName function that we created on the App.Person class. You can see that we can refer to this function as a normal object property like we did with the firstName and surname properties because the function was extended with the property definition.

Changing either of the text in the inputs causes the computed property fullName to recalculate.

You can verify this below or at this jsfiddle.

Conclusion

Ember is brimming with a multitude of abstractions that might be different than anything you have come across in the past. I believe it is better to try and take these concepts in one at a time rather than dive in and get blown away by the routing, controllers, itemControllers and the rest. I believe it is important to grasp these core concepts first before progressing. You will use them more than anything else. Bindings and computed properties are powerful and much more powerful than the examples I have used but we need to get the basics right first.

Ember.js Routing - the Final Cut

Mar 3rd, 2013

This is my third post about the now infamous ember.js router. My initial reaction to the new router was mixed as I quite liked the old router but over time, I have grown to like it. One thing that is undeniable about the new router is that it is much easier on the wrist and fingers as it requires writing considerably less code. That said, the new router is heavily convention based and there were initial periods of bewiderment as I looked at an errorless console and a white screen as my routes were not found. These periods have receded over the passage of time but there are some traps and pitfalls that catch the unaware. These things generally only shake out during real world use. First of all let me set the scene with the model of the sample app that I will use to illustrate the concepts.

Model

I have a simple application that I have been using for my posts that simply allows a user to create a bank of gym exercises that somebody would use while exercising in the gym. The end goal is to create exercise programs from this bank. There is an exercise model which describes the individual exercise and a group model that is used to categorise the exercise by abs, arms, back, legs etc. Both models are listed below for completeness:

Behold The Router

I am going to refrain from contrasting the new router with the old because this version seems final and as before the router of an ember application is really the focal point of the app. The router orchestrates state changes in the application via url changes from user interaction. The router and the various associated states are tasked with displaying templates, loading data, and otherwise setting up application state. Ember handles this by matching urls or url segments to routes and in order to make this happen, the router has a map function where you can map url segments to routes.

Below are the map of relevant urls in this sample app:

A user will enter an ember site at a specific url or they will interact with a view which will raise an event and cause the url to change. Entering via a url or the application changing the url will invoke the router which will try and translate the change in url by matching the url with one of the route handlers that you specify. We have specified in our router that the root url will be handled by a route named home:

WZ.Router.map  ->
  @route 'home', path: "/"

Ember routes are also charged with hooking up the correct model, the correct controller and the correct view which will point to a handlebars template. These will all follow the convention of prefixing the particular object with the string argument that is passed to the route method which in this case is home. The ember runtime will look for a HomeController, a HomeView or if it cannot find a view, it will look for a home.hbs file.

Now we come to the confusing point, you don’t actually have to create any of these objects and in this example, I have not specified a HomeRoute, HomeController or HomeView. When you navigate to the ’/’ root route, Ember will look for a HomeRoute and if it does not find it, it will automatically generate a HomeRoute. The same can be said for HomeController and HomeView. We can verify this by adding the following expressions to our handlebars template:

Now if we refresh the page, we can actually see what controller and view are backing this route. Ember has automagically generated these objects for us. The goal of this generation is to eliminate the needless creation of these objects if they are not needed. We can of course override this behaviour for more complex scenarios. If I actually needed a controller/view to back this route I could specify the HomeController and HomeView routes that adhere to the naming convention like this:

If we now refresh the page we can see that our objects are hooked up and ready for action: The above is actually a great way of debugging that your views are actually hooked up as I was scratching my head on a number of occasions as I came to terms with the various naming conventions.

Application Objects

When your application boots ember will look for an ApplicationRoute, ApplicationController and an application.handlebars file with these objects being generated if they are not specified. My application.handlebars file looks like this:

Anybody familiar with the old router will be more than familiar with the term outlet. An outlet is a placeholder in the template were other templates or content can be injected into. In the above example there is one named outlet nav and a default unnamed outlet. I think I am right in saying (and manybe somebody can correct me) that you do need to create the application.handlebars file and it must contain at least one outlet. In our HomeRoute example, the contents of the home template will be injected into the default unnamed outlet which you can see on line 2 of the above gist. But what about the nav outlet which will render our twitter bootstrap nav bar: In order to achieve this, I have created an ApplicationRoute that will be executed by the ember runtime when the app first boots and stop the auto generation by the runtime of an ApplicationRoute:

The ApplicaitonRoute is the correct place for this as the nav will appear on every page. An ember route has various hooks that you can use to set up other objects or to render additional content. The renderTemplate hook can be used to render a template that differs from the one picked up by convention and it can also be used to render content into other outlets as in this example . The current controller and model instances get passed as arguments to this method.

On line 5 of the above gist we are using the render method to specify:

Which template we want to render, which in this case is located in our directories folder at ‘nav/nav’
Which parent template the new content will be injected into which in this case is the application template. This is specified by the into property of the options hash.
Which named outlet of the parent template we want to render the content into which in this case is specified by the outlet property of the options hash and we set that property to nav.

One gotcha I had was that I had to call an additional render on line 3 or I got an error. This appeared to be because the main outlet had not been rendered or was undefined. This was only the case for the ApplicationRoute.

Resources

If we cast our mind back to our map of urls and routes that we specified:

We can see on line 5 that we are declaring a resource mapping, anybody from a rails background will be familiar with this concept but for those who are not, a resource can be thought of a thing or one of the nouns that make up your application. This usually equates to one of the models and this is no exception as an exercise is a pivotal thing in our application. On lines 4 to 6 are the nested routes that belong to the resource. These can be thought of the verbs or the actions that you will carry out on the thing or resource. We are stating that we will have an index route which will list all exercises and two additional routes for creating new exercises and editing existing exercises. I am not going to get into the restful side of what a resource is because I don’t believe it is relevant to client side mvc.

Index Route

We have declared that we will have an index route for the exercise resource that will list all exercises: In order to achieve this I have created the following 2 routes:

Why 2 routes and not just an index route? I have created an additional ExercisesRoute that is picked up by convention from the declared exercises resource of the router’s map in order to use one of the ember route’s hooks which is namely setupController. I mentioned at the beginning of the post, that each exercise belongs to a group and I want to have all these groups loaded up and ready for use in the ember data persistence store. I don’t know where or which url the user will use to enter the application but I know I need all the groups loaded and in the store. The ExercisesRoute is ideal for this because it will be executed before any of the child index, new or edit routes that are nested in the resource declaration. I could have used the ApplicationRoute for this bootstrapping but I wanted to confine it to the exercises part of the site.

The setupController hook is normally used to set up the current controller with the current model so in this case it would be setting the model on the ExercisesController but we are going to gate crash this party as we do not need to specify an ExercisesController and set its model. On line 3, we are using the controllerFor convenience to access the groupsController and set its model. We set its model to all the groups that exist and will be returned from an async call to the server.

On lines 5 - 7 of the above gist, an ExercisesIndexRoute is declared with a name that sticks strictly to the naming convention law of the land or it will not be picked up by the runtime. The index route of the exercises resource is nested and the ExercisesIndexRoute is named and cased to reflect this:

Exercises(resource)Index(route)Route

We will also need to name our controller and views ExercisesIndexController and ExercisesIndexView if we want the runtime to pick them up and not auto generate them.

Model Hook

On lines 6 - 7 we use another important hook of the router, namely the model hook:

model: ->
  WZ.Exercise.find()

The model hook is a hook you would use to set which model is associated with that url. A url of /exercises should be a listing of all the exercises currently in the database. In order to achieve this, the model hook is used to query the back end store of all the exercises. The ember runtime will use this hook to setup the controller with the model in the setupController hook unless you override it and do something else.

Templates

As mentioned earlier, I have an ExercisesRoute which equates to a resource in the routers map. Under this route are the nested actions, index, new and edit. If we wanted to ensure that all these nested routes are enclosed in the same html elements like below, were the same header will appear in each nested route: To enable this, the exercises.handlebars file which is picked up by convention looks like this:

With this in place, all the nested routes will have their content rendered into the above outlet with no extra coding on your part.

Conclusion

I have grown to like the new router although initially it was annoying to leave the old router which I liked. The naming conventions are a double edged sword, on the one hand they drastically cut down on needless ember class declarations but I suspect it might put a few new people off as it can be difficult to diagnose why your screen is suddenly white and there are no errors in the console. My advice is to stick with it as ember is a work of art, something quite beautiful and most needed in the javascript world. On a selfish note, I hope to base the next phase of my career on ember.js work and I hope it has the same growth as rails once did. This of course is mere speculation on my part, we’ll just have to see.

If you disagree with any of this or can point out any mistakes on my part then please leave a comment below.

I hope ember.js does not confuse you!

Ember.js Routing - the Director’s Cut

Aug 20th, 2012

WARN: This article is now out of date. Check this post for a more up to date article on ember routing.

I’ve had a number of requests to update this post which discussed a solution to the lack of routing at that time of writing in ember. This solution used an addon called the ember-routemanager from Gordon Hempton.

A lot in Ember has changed since that post and Ember now has its own full blown routing solution that is not a million miles away from the route manager I blogged about. I believe Gordon Hempton who created the original ember-routemanager is now an ember core member which might explain the stark similarities.

Enough of the old and on with the New

The new ember router has been on the master branch for a while now and it is an emerging pattern that is prone to change and indeed has changed quite a bit in its short existence. The ember routing solution is one of the main reasons I am drawn to ember and it differs greatly from anything else out there in the javascript mv* space.

The Ember router extends the very elegant ember statemanager:

Ember.Router = Ember.StateManager.extend({
});

The basic premise of the ember routing solution is that you describe your application as a hierarchical tree of objects - one object per conceptual state (Ember.Route extends Ember.State):

WZ.Router = Em.Router.extend enableLogging: true location: 'hash' root: Em.Route.extend index: Ember.Route.extend route: '/' redirectsTo: 'home' home: Em.Route.extend route: '/home' connectOutlets: (router, event) -> router.get('applicationController').connectOutlet 'nav', 'navbar' router.get('applicationController').connectOutlet 'home' router.get('homeController').connectOutlet 'bottombar', 'bottombar'

On line 1 we create a subclass of the Ember.Router and assign the reference to a property named Router of the Ember application object which in this example is named WZ. Naming the property Router is a convention you must adhere to.
Line 2 tells the Ember.Router to log state transitions to the console.
Line 3 sets the location property of the Router to hash which means that url fragment identifiers like #/post/1 will be parsed for matches in the routing hierarchy. You can also specify a value of history which will use the browser’s pushstate api if one exists.
On line 4, a root route is created. As the name root suggests, all other routes (or states as I still think of them) will be either direct children of the root route or grandchildren of the root route. The root route acts as the container for the set of routable states but is not routable itself.
On line 5 and line 8 are two such child routes or states of the root route named index and home respectively.
Leaf routes in the routing hierarchy can have a route property describing the URL pattern you would like to detect.
On line 6, the index route has a route property with a value of ’/’ which is what the url will be when the application first loads.
On line 9, the home route has a route property of /home.
On line 10 is the connectOutlets method that you can override in each child route and provide a mechanism for rendering content onto the page as the url changes. More on this later.

When an Ember application loads, Ember will parse the URL and attempt to find an Ember.Route within the Router’s child route hierarchy that matches the url. Loading the page at ‘/’ which is what the url is when the application is first initialised, will first of all transition the router to the first route named root and then to the subsate or child route where the router can find a match on the url. In this case, the router will find a match on the route property of the index route and will transition to that route. The index route in the above example simply redirects to the home route via the redirectsTo directive on line 7 of the above gist.

Loading the page at the url #/home will transition the router to a substate or route at a path or place in the hierarchy of root.home. This path syntax is useful for testing and what it translates to is that we are currently at the home route which is a child of the root route.

Below is a test that verifies a root url of ‘/’ will transition to the home route:

describe 'router', -> describe 'transitions', -> beforeEach -> WZ.initialize() it 'should transition to home from root', -> WZ.router.route '/' expect(WZ.router.get('currentState.path')).toEqual 'root.home'

On line 7, the route method of the router is called and a url fragment is passed as an argument for the router to try and match on.
Line 8 asserts that we have transitioned to the expected home route which is a direct descendant of the root route or state. We verify this by checking the currentState.path of the router which uses the dot syntax to signify where we are in the router hierarchy.

Nested Routes

As you would expect, nested routes correspond to fragments of the url. Below is a direct descendant of the root route named vault which has a path of root.vault:

In the above example, the vault route has a route of /vault, as well as a child state of new which in turn has a child state named step1. If a url of /vault/new/step1 is requested, all three of these routes will be composed together and all 3 states or routes will be executed in sequence. Each connectOutlets method on each state or route will be executed, giving you a chance to change what is displayed on the page as the url changes and the application state changes.

Below is a test that verifies a url of /vault/new/step1 transitions to the correct state:

it 'should transition to step1', -> WZ.router.route 'vault/new/step1' expect(WZ.router.get('currentState.path')).toEqual 'root.vault.new.step1'

State transitions and View Changes

As each url change transitions the router or statemachine from state to state or route to route, so you would expect what is rendered onto the screen to change also. Each nested route can take responsibility for what is rendered onto the screen:

Below is the vault route and its child states

vault: Ember.Route.extend route: '/vault' connectOutlets: (router, event) -> router.get('applicationController').connectOutlet 'exercises' index: Ember.Route.extend route: '/' connectOutlets: (router, event) -> exercises = WZ.store.find(WZ.Exercise) router.get('exercisesController').connectOutlet 'main', 'exercisesHome', exercises new: Ember.Route.extend route: '/new' connectOutlets: (router, event) -> router.get('exercisesController').connectOutlet 'main', 'exercisesNew' step1: Ember.Route.extend route: '/step1' connectOutlets: (router, event) -> exercise = WZ.Exercise.createRecord() exercise.setProperties {name: 'Barbell bicep curl', description: 'Barbell bicep curl'} router.get('exercisesNewStep1Controller').set 'groups', WZ.store.find(WZ.Group) router.get('exercisesNewController').connectOutlet 'step', 'exercisesNewStep1', exercise

And below is a screen grab that outlines which parts of the page are rendered by which route whenever a page is rendered at the following url #/vault/new/step1 If the url changes to #/vault/new/step2 then only the bottom segment of the page will be changed when the step2 route is transitioned to.

Outlets

So how is this beautiful tapestry of nested views stitched together? How do router transitions marry themselves to view changes? Well, the observant amongst you will have noticed a method named connectOutlets that appears in all of the leaf routes (non-root route) of the router.

In order to illustrate how this works, I am going to first refresh our memory of what the router itself looks like:

As explained earlier, the router will parse the url and try and find a match on the route property of one of the router’s child routes or states. When the application first loads or a url of ‘/’ is requested and a route will be found on the index route which redirects to the home route.

On line 10 of the above, we come to the now infamous connectOutlets method (lines 10 - 13 of the above gist).

Ember provides some nice conventions that are tied to routing. The connectOutlets method will be called when a state or route has been entered and gives the developer an opportunity to reflect the change in application state by injecting content into the place holder of a client side template which is named an outlet.

connectOutlets works by assigning Ember controller/view pairs, that follow the convention xxxxController and xxxxView and assigning them to named placeholders called outlets that are found in handlebars templates.

When you create and initialize a new Ember application object (more on this later), you must have a controller named ApplicationController and a view named ApplicationView or Ember will throw an exception. These two objects will form our first controller/view pair that conforms to the xxxxController and xxxxView convention. What is rendered from the xxxxView will be placed in the outlet or handlebars placeholder.

Below is my ApplicationController from this sample app:

WZ.ApplicationController = Em.Controller.extend()

And below is my ApplicationView from this sample app:

WZ.ApplicationView = Em.View.extend
  templateName: 'app/templates/application'

With these in place, Ember will render the template that is specified at the templateName property in the view above. Below is the template that can be found at the path of the templateName property above:

Pretty sparse, huh? All that the above template contains are two outlets, one outlet named nav and one default outlet. Outlets can be thought of as place holders that you can inject content into. The connectOutlets method on each child route is the place to connect up these outlets with corresponding Ember view and controller pairs.

Below is a refresh of the connectOutlets method for the /home route:

home: Em.Route.extend route: '/home' connectOutlets: (router, event) -> router.get('applicationController').connectOutlet 'nav', 'navbar' router.get('applicationController').connectOutlet 'home' router.get('homeController').connectOutlet 'bottombar', 'bottombar'

The first outlet we connect is the named nav outlet (line 4 of the above gist). The first thing to note on line 4 is that we are accessing the controller via the router with the following syntax:

router.get('applicationController').connectOutlet 'nav', 'navbar'

Ember has made the wise choice to move away from accessing objects with the path syntax that was popularised in earlier ember applications when you might have accessed the application controller like this:

WZ.controllers.applicationController

And instead objects ending in Controller are detected and are assigned as properties of the router when an Ember application is initialized like below:

window.WZ = Ember.Application.create()
WZ.initialize()

This means we can pull the controller instances from the router and call the controller’s connectOutlet method to inject content into the outlet placeholders that are specified in the handlebars template. connectOutlet creates a new instance of the provided view class, wires it up with its associated controller, and assigns the new view instance to a property on the current controller.

What this translates to, is that if we call connectOutlet like this

router.get('applicationController').connectOutlet 'nav', 'navbar'

We are telling ember to connect the outlet that is named nav and that we want to create a new instance of a view called NavbarView and wire it up with a controller named NavbarController.

The following line from the above gist calls the controller’s connectOutlet method and only passes in one argument:

router.get('applicationController').connectOutlet 'home'

This tells ember to connect the outlet with no name with a view called HomeView and wire it up with a controller called HomeController.

The HomeView view that was instanciated from the connectOutlet above method points to a handlebars template that contains a named outlet that we can hook up with the following line:

router.get('homeController').connectOutlet 'bottombar', 'bottombar'

The above line grabs a reference to the homeController object via the router, and then call connectOutlet which will connect an outlet named bottombar with a controller/view pair named BottombarView and BottombarController. And so we could go on into finer more maintainable detail

And this is what the page looks like in the browser and which section belongs to which outlet. I hope you can see that this is a very elegant solution to weaving a rich and maintainable UI from your handlebars templates.

One last thing to mention is that you can supply a content for the controller by supplying a final argument for the connectOutlet method like this:

index: Ember.Route.extend route: '/' connectOutlets: (router, event) -> exercises = WZ.store.find(WZ.Exercise) router.get('exercisesController').connectOutlet 'main', 'exercisesHome', exercises

On line 4 of the above gist, we are retrieving an array of objects from the remote store and passing that into the connectOutlet method as the last argument. The content will be assigned to the controller.

Conclusion

I am blown away by the elegance of this routing solution and it is the best out there bar none in the javascript mv* space.

I besiege you to get the latest ember from github and take the plunge, you will not regret it.

All the code that is used in these blog posts can be found at this github repo.

Please feel free to add any comments below.

Iridium - a Toolchain for Javascript Development

Aug 10th, 2012

I want to give a shout out to a very new and very interesting project called iridium from Adam Hawkins who blogs at BroadcastingAdam. Iridium aims to provide some much needed tooling to the new front end javascript/coffeescript project type that is popularised by the javascript mv* space.

With more and more of the application code finding its way onto the client in the form of javascript or coffeescript, there is a necessity to make a clear separation between between the front end client code and the backend server code. With Gems like the rails-api gem stripping away all the unnecessary cruft that is not needed when serving only json from a rails application, wouldn’t it be nice if you could spin up a new javascript/coffeescript project with just a few keystrokes?

Problem 1 - Lack of Front End Project Guidance

Gone are the days (I wish) when client side code and server side code lived side by side in a one size fits all monolithic project structure. Good practice dictates that you should segregate these two distinct layers into two (or more) orthogonal projects that can evolve at their own pace.

On the server side, I can quickly create a new rails project by issuing a command like

rails new project_name

Rails brought conformity to the mvc pattern by creating a guaranteed project structure for every rails project. Wouldn’t it be nice if we could bring some order to the new breed of client side only project?

The Solution

Iridium aims to add some much needed tooling to the front end project space and sets out with the following goals:-

CLI driven interactions
Expose as little Ruby as possible
Focus on JS/CSS/HTML
Make JS testable

First Steps

The first thing to do is to ensure that you have casperjs installed. Casperjs is what iridium uses as its integration test framework. Full installation details can be found here. I personally used homebrew to install casperjs.

Next, you need to create a Gemfile with the following entries:

source :rubygems gem 'iridium', :github => 'radiumsoftware/iridium' gem "rake-pipeline", :github => "livingsocial/rake-pipeline" gem "rake-pipeline-web-filters", :github => "wycats/rake-pipeline-web-filters"

I created an instance of the above Gemfile in my ~/projects directory so that I can easily create front end projects from that location.

A quick bundle install will install the specified gems and then you can start using iridium.

Iridium solves the first problem by allowing me to spin up a new client side project with most of the initial decision making made for me.

I can now create a new front end project named sample by simply entering the following command while in the project directory.

bundle exec iridium app sample

Note that I am using bundle exec, do not pass go if you are not running iridium in the context of the bundle specified in the Gemfile.

The above command will create the following output:

create sample
create app/images
create app/javascripts/app.coffee
create app/javascripts/boot.coffee
create app/javascripts/controllers
create app/javascripts/models
create app/javascripts/templates
create app/javascripts/views
create app/public
create app/stylesheets/app.scss
create app/vendor/javascripts
create app/vendor/javascripts/handlebars.js
create app/vendor/javascripts/jquery.js
create app/vendor/javascripts/minispade.js
create app/vendor/stylesheets
create site
create test
create test/controllers
create test/helper.coffee
create test/integration/navigation_test.coffee
create test/models
create test/support/qunit.js
create test/support/sinon.js
create test/templates
create test/unit/truth_test.coffee
create test/views
create application.rb
create readme.md
create .gitignore

This will create the following project structure: We now have a guaranteed folder structure for every client project we create. Iridium also provides some smart defaults that will provide some much needed guidance for anybody new to this space:

We will be using SASS as the css framework of choice and an app.scss file is created in app/stylesheets/app.scss.
There is a clear separation between unit and integration tests.
We will be using QUnit for unit tests.
We will be using Casperjs for integration tests. Integration tests are often overlooked in the front end world.
sinon.js is injected into the test environment and is used to mock or stub out any async calls when unit testing.
The Rake-pipeline is used for javascript loading and for packaging assets for deployment.

The Case for the Rake-Pipeline

The rake-pipeline is a better fit for single page applications than an asynchronous module loader like requirejs because it bundles all your dependencies and individual files into a single script. The rake-pipeline does all this concatenation and minification transparently behind the scenes without you having to do anything at all. Using the rails asset pipeline eases rolling deployment for you because you still get your scripts served as individual files when in development mode but in production mode, scripts are served minified and concatenated.

I prefer the rake-pipeline over requirejs because requirejs is great in development and satisfies the requirement of being able to break the functionality up into smaller files but and I must stress the word but, it is at this time of writing, a major pain in the rear to compile requirejs out for a production build. You are going to have to use something like the r.js compiler to bundle all your dependencies into a single script.

With a combination of the Rake-pipeline, @wycat’s rake-pipeline-web-filters and minispade we can achieve the following goals:

Break it up: While it’s common to write smaller libraries as one big file, I like the mental separation provided by dividing the source into multiple files.
Inline dependency management: I’d rather define each file’s dependencies right in the code, rather than define the concatenation order in some sort of makefile.
Keep files separate while debugging: When we’re setting breakpoints and stepping through code in the debugger we want to see our individual source files, not have to wade through one big file.
A transparent solution: We need the production library to be free from dependency management code.

With the above combination, I can develop my code in separate files and then I can prefix my code with minispade require statements like below:

require 'app/router'
require 'app/controllers/application'
require 'app/views/application'

When you create an iridium project, the rake-pipeline is already hooked up and a copy of the latest minispade.js can be found in app/vendor/minispade.js.

The above is valid coffeescript and I can simply point to one file with all my requires.

app.coffeee and boot.coffee

A nice conventions is created for you when you create a new iridium project that really helps when it comes to testing. A file named app.coffee is created for you where you can put all your minispade require statements as was outlined above. This file is hooked up for you in both the web application itself and the test environment. No initialisation code goes into this file, just the minispade require statements. A separate file named boot.coffee is created to house any application start up code. The boot.coffee file is not referenced by the test environment, paving the way for you to unit test separate bits of the application rather than having to boot the whole thing.

An example of the app.coffee file that is referenced by both the testing environment and the actual website itself could look like this:

require 'app/router'
require 'app/controllers/application
require 'app/views/application'

And an example of boot.coffee could look like this, with the code below initialising an emberjs applicaiton:

require 'application/app'
Application.initialize()

Index.html

index.html is an annoying part of web development. You cannot start or serve your application without an HTML page to load your JS. Iridium has a built in index.html which loads your assets and dependencies. This will work for most simple applications. You can override this by providing your own index.html in public.

Once you have generated your application, you can run the built in development server like so:

bundle exec iridium server

Which will give you the following output:

$ cd sample
$ bundle exec iridium server
>> Thin web server (v1.4.1 codename Chromeo)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:9292, CTRL+C to stop

Problem 2 - Testing the client code

The rise of the client side MV* framework and the proliferation of code on the client has made the call to test, a more important one than ever. The tide does appear to be turning on testing client code with the rise of important testing frameworks like Jasmine and QUnit but adoption of this practice could be better. Part of the difficulty is getting set up. Why should I have to concern myself with wiring everything together every time I start a new project? Anybody who initially wrestled with Jasmine and the rails asset pipeline will know exactly what I mean when by wrestling.

Anybody from a rails background will be comfortable with running particular flavours of test suites (unit/integration/functional etc.) via rake’s uniform command line interface. Iridium gives you the same capabilities and makes the clear distinction between unit tests and integration tests which is not always the case in the javascript world.

The Solution - The Iridium Test Runner

When you create a new project with iridium, your project is ready to test from both the unit test side and the integration test side. No set up code is required. There is a test/unit folder and a test/integration folder which should need no explanation as to their presence.

QUnit is the default test suite for writing unit tests and casperjs the framework of choice for integration tests.

What is very nice and is the ability to run both the unit tests and the integration tests a single unified test suite.

When the following command is entered:

bundle exec iridium test

The above will run both the unit tests and the integrations tests and present the results as a unified test suite.

It is also possible to be more granular and run, for example only unit tests:

bundle exec iridium test test/unit/* --debug

Note that I am passing in the –debug switch to enable console messages to be printed to standard out.

I can also run tests against individual files.

bundle exec iridium test test/unit/truth_test.coffee  --debug

Sinon.js gets included in test/support as a nice opinionated default to assist with mocking out any async ajax calls.

Conclusion

The front end project space has been crying out for a project like iridium. Rails provides uniformity and opinionated guidance every time you create a project. We need the same in the front end project space.

Go check the project out on github and spread the word. There are many features I have not mentioned in this post and many that will be added in the future.

Ember.js - Routing With States

Apr 22nd, 2012

WARN: This article is now out of date. Check this post for a more up to date article on ember routing.

When I first started investigating Ember.js I wrote a post about an Ember addon called the SC.StateChart that is really part of the old Sproutcore framework. In that post, I wrote that I liked the enterState and exitState handlers of the StateChart and how they would be ideal for switching between views. As of Ember 0.9.6, the SC namespace is no longer supported and I decided to refactor/remove anything that was Sproutcore related. It was during this refactoring that I found a very similar object called the Ember.StateManager that already exists in the Ember core code. It turns out that the Ember StateManager is a very elegant solution to transitioning between views and attaching and removing views from the DOM.

The Ember StateManager is part of Ember’s implementation of a finite state machine. Simply put a state machine can be in only one of a finite number of states. The machine is only in one state at a time; the state at any current time is called the current state. It can change from one state to another when initiated by a triggering event or condition, this is called a transition. The classic example is a state machine that represents a lightbulb transitioning from the off state to the on state.

Let us flesh this out with some code. Below is a StateManager that I am currently working on:

WZ.ContentRouteManager = Em.RouteManager.extend rootView: WZ.layout.get 'contentArea' initialState: 'main.home' enableLogging: false main: Ember.ViewState.create route: 'home' view: Ember.View.create templateName: 'app/templates/home/main' home: Ember.ViewState.create view: Ember.View.create templateName: 'app/templates/home/home' vault: Ember.ViewState.create route: 'vault' view: Ember.View.create templateName: 'app/templates/vault/main' index: Ember.ViewState.create route: 'index' view: Ember.View.create templateName: 'app/templates/vault/home'

On line 1, I am defining a definition from which a StateManager instance can be created from. I am actually extending another definition called the RouteManager that extends StateManager. The RouteManager is part of an excellent addon called the ember-routemanager that ties the StateManager and a routing implementation together. More on this later.
On line 3, the initialState property is set to instruct the StateManager which state to transition to when an instance of the StateManager is created. I am using the Ember path dot syntax to say that the initial state will be a state named home (line 11) that is a child of a parent state named main (line 6).
On line 6 is the definition of the previously mentioned parent main state. A StateManager is composed of child state objects. In this example, the StateManager is composed of Ember.ViewState objects. Ember.ViewState objects can contain Ember.View objects (as you would expect).
On line 15, is another state named vault that has a child state named index (line 20).

An Ember.ViewState object extends Ember.State and as the name suggests, Ember.View objects are associated with the Ember.ViewState . All child state objects in the above example are ViewState state objects and each view has a templateName property that is a relative path to a handlebars template.

Anybody from a .NET background will wince at the name ViewState but I can assure you, there is no connection whatsoever.

When a StateManger is composed of instances of ViewState objects, the StateManager will interact with Ember’s view system and manage which views are added and removed from the DOM based on the StateManager’s current state. One way of transitioning between states is to invoke the goToState method:

@routeManager = WZ.ContentRouteManager.create()
@routeManager.start()

@routeManager.goToState 'vault.index'

The above example will transition to the index state which is a child of the vault state. Using the Ember path syntax allows you to transition to a child state in one statement, no matter how deep it is. When you transition from one ViewState to another, the view of the currentState is destroyed and removed from the DOM as you exit the state and the view of the ViewState you are transitioning to is created and attached to the DOM as you enter the new state.

The StateManager can also receive and route action messages to its states via the send message, which you can read about in the Ember docs.

I find the StateManager to be a very elegant solution that is bereft of tedious boilerplate code. This is in stark contrast to Backbone.js where the user is left with both the architectural decision and the execution of how this should take place. I mentioned one such solution in my Backbone.js - Lessons learned post.

My first attempts at Backbone were littered with a lot of repetitive code like the following:

render: => @$el.html(@template()) @collection.each (item, counter) => @renderView($('tbody:first', @$el), 'append', new ReportBusinessUnitView(model: item, vent: @vent)) @

In Backbone, I think there is too much boilerplate code required to glue everything together. I have started using Derick Bailey’s excellent backbone.marionette recently with Backbone and you really need to use this if you are using Backbone regularly.

Nested ViewStates

When ViewState objects are nested like in the example below:

WZ.ContentRouteManager = Em.RouteManager.extend rootView: WZ.layout.get 'contentArea' initialState: 'main.home' enableLogging: false main: Ember.ViewState.create route: 'home' view: Ember.View.create templateName: 'app/templates/home/main' home: Ember.ViewState.create view: Ember.View.create templateName: 'app/templates/home/home'

Both the StateManager’s currentState and any child states of the currentState will draw into the StateManager’s rootView property (line 2). In the above example, the RouteMangaer will initially transition to the path defined in the initialState property which in this case is the main.home path which points to the home state on line 11. This means that the handlebars template declared with the templateName property in both the main state’s child view object will be compiled and attached to the DOM and the handlebars template defined in the home state’s child view object will be compiled and attached to the DOM.

Below is how the main state’s view and the home state’s view look when they are rendered onto the browser: You can get quite inventive and compose reusable partial views as and when you need them. I am setting the rootView property which means I can just layer views onto the body element of the html document but you can also specify a rootElement property:

rootElement: '#some-other-element'

Routing

This brings us nicely on to routing, as I mentioned earlier, I am using the excellent ember-routemanager addon that ties the StateManager to a routing implementation. This addon allows you to create RouteManager instances that derive from the now infamous Ember StateManager. Below is a reminder of how I am extending the RouteManager:

It is worth noting that I am not creating an instance of the RouteManager but merely extending it which will allow me to create an instance on application start up or create instances in my jasmine specs.

What you should note from the above gist is that I am defining a route property on each child state object (lines 7, 12, 16 and 21). This allows me to use a combination of the route property and the nesting of the child states to define client routes that will transition to the required states and attach and remove the views from the DOM.

Below is how I create an instance of my derived RouteManager:

@routeManager = WZ.ContentRouteManager.create()
@routeManager.start()

The start method will start the RouteManger listening for location changes. This allows me to define routes either as the now familiar hash links:

<!--Transition to the vault.new state -->
<li><a href="#vault/new">New</a></li>

Or I can programatically set the location property which is useful for testing:

@routeManager.set 'location', 'vault/index'

All that you would expect from a routing frameworks seems covered, below is an example of a route with a dynamic parameter:

route: 'vault/:exerciseid'

Setting the browser location to #vault/5 would map to the vault state with an exerciseid of 5. Wildcards and regular expressions are also covered:

route: /(\d{4})-(\d{2})-(\d{2})/

Conclusion

I got quite excited as I explored this approach (I should get out more). This is a real productivity boom that could be further enhanced by having the route inferred using convention over configuration by dynamically creating the route with respect to the state’s nesting. The RouteManager/StateManager/StateMachine is a very elegant solution for transitioning between views that exists in many other smart client UI frameworks. Being a web developer, I just have not come across it before.

Testing the RouteManager

I ran into a few problems when testing the RouteManager. Below are some initial specs that now all pass:

describe 'route managers', -> @routeManager describe 'Content Route Manager', -> beforeEach -> Ember.run => @routeManager = WZ.ContentRouteManager.create() @routeManager.start() afterEach -> return if !@routeManager @routeManager.get('rootView').forEachChildView (view) -> view.remove() @routeManager.destroy() it 'should render main section', -> expect(@routeManager.getPath 'currentState.name').toEqual 'home' it 'should transition to vault index', -> Ember.run => @routeManager.set 'location', 'vault/index' expect(@routeManager.getPath 'currentState.name').toEqual 'index'

One of the problems I had was clearing the rootView property and disposing of the RouteManager after each spec. This is solved in the afterEach method on lines 10 to 16 of the above gist where I am removing each view from the DOM and destroying the StateManger.

Please feel free to add any comments below.

Ruby, RabbitMQ RPC and WebSockets

Apr 10th, 2012

Scenario

The rails project that I am currently working on presented an interesting problem which I am guessing is a reasonably common one. The problem is how to have a long running process run both independently of the browser and display results in the browser as and when they happen. This is my solution to the problem but I would love to hear how others have dealt with the same scenario.

The focal point of the application in question initiates a long running process that crawls a web site that is specified by user input (see image below) and screen scrapes each page in the site with the intent of collecting any data that meets a certain search criteria. This long running process could last many minutes and it is safe to say that each process will outlive the lifetime of a standard page request in terms of time. The long running process will persist any results that are found to a backend data store. The first requirement is that the job completes if the user closes down their browser. The next requirement is that if the user is connected to the browser, I want to visually display any results that are found by adding data rows to a data table as and when they happen and not have to wait until the process has executed to render all the results at once. Below is a screenshot of how the user inputs their search terms and how the data is displayed. While working on the window’s platform, I have often used Microsoft’s MSMQ for reliable messaging so I just needed to do some research into what was available on non-windows platforms. The message queue handler will be charged with doing the actual work of crawling around the website and persisting any results to the backend data store. This will fulfil my requirement of having the job complete if the user closes down the browser.

That just left displaying the results of the screen scraping in the UI as and when they are found. I could have used polling to periodically check for results on the backend store but this did not seem in keeping with the times and this also appeared to be a great excuse to play about with websockets. I just needed someway of having the queue handler pass the results back to the websocket layer as and when they are found.

Solution

The creation of a queue handler and a websocket layer are easy problems to solve separately, the hard part was finding a means for the two distinct boundaries to communicate together.

The websocket layer seemed very well covered on the ruby and rails platform and the EventMachine based, async EM-Websocket Ruby websocket server was an obvious choice.

When it came to queuing, I first looked at the popular resque library as my background job processor or queue handler but I could not find anyway for the queue handler to speak to the websocket layer. I then looked into RabbitMQ which has many more features that are a better fit for my needs. As there is a gem for everything in Ruby, I quickly tracked down the AMQP gem which is a feature rich well maintained, fast, asynchronous RabbitMQ ruby client.

AMQP stands for Advanced Message Queueing Protocol and is an open standard for passing business messages between applications or organisations. You can think of it as an interface describing a common set of functionality that all message queuing systems must provide if they implement the AMQP interface.

RabbitMQ RPC

After a bit of digging about, I came across a messaging pattern that was a good fit for my problem which is the RPC pattern. We all know about RPC (Remote Procedure Calls) in general computing terms but how does this translate to the message queuing world?

In the context of my application, the RPC pattern manifests itself like this:

The user will submit a url to perform the search that will be sent to the websocket layer via the following clientside coffeescript code:

Lead.Controllers.ParsingController = Ember.Object.extend start_parsing: (url_search) -> socket = new Lead.WebSocket("ws://#{document.domain}:61615") @set 'socket', socket socket.onopen = (evt) -> $('#leads table').show 'slow' json = JSON.stringify url_search socket.send json socket.onmessage = (evt) -> Lead.get('leads_controller').addLead evt.data

The above code contains an Ember.js controller with a start_parsing method/action on line 2. When the user enters a url, this method is called and an Ember url_search model object is created containing the url of the site to be scraped.

On line 3, a new websocket connection is attempted.
The onopen event fires when a successful connection has been negotiated. Line 7 provides an event handler for the onopen event that transforms the url_search object into a JSON string and sends it to the websocket server.
On line 12, an onmessage handler is provided for the websocket layer to transmit any results that are found while screenscraping the site.

Below is the EM-WebSocket server that the Ember controller listed above will communicate with:

@sockets = [] EventMachine.run do connection = AMQP.connect(:host => "localhost", :user => "guest", :password => "guest", :vhost => "/", :logging => true, :port => 5672 ) EventMachine::WebSocket.start(:host => '127.0.0.1', :port => 61615) do |ws| socket_detail = {:socket => ws} ws.onopen do @sockets << socket_detail puts " SOCKET INFO #{socket_detail.inspect}" end ws.onmessage do |message| puts "received message : #{message}" publish(message, "leads", ws) end

On line 2, we start the EventMachine event loop by calling the run method. The event loop can be thought of an endless loop that can be stopped and started by designated events or the expiration of a timer. Everything from line 2 onwards is the block that is passed to the run method and is executed during the lifetime of the event loop.
On line 3, the connection details for the rabbitmq server are specified (more on this later).
On line 11 the WebSocket server is started via the start method.
Line 14 provides an event handler for the server side websocket onopen event.
Line 19 is the server side onmessage event handler which is triggered when the Ember controller of the previous gist, sends a message containing the JSON of the url search request.
Line 22 calls the yet to be exposed publish method that will send a message request to our rabbitmq server instance.

The RabbitMQ RPC Client

So far, we have initiated a new websocket connection with the server and sent a JSON string that can be transformed into an object that describes what website to crawl and scrape results from.

I stated earlier that the RabbitMQ RPC pattern will allow the backend queue handler to communicate with the websocket layer. The webscocket layer can be thought of as the RabbitMQ client and the backend queue handler can be thought of as the RabbitMQ server. The websocket layer outlined above is our RabbitMQ client layer that will send a request message containing the JSON string sent from the front end Ember controller code outlined in the first gist.

In order for the RabbitMQ client to receive responses from the RabbitMQ server, we will need to specify a callback queue that the RabbitMQ server can send messages to. In the context of my application, the RabbitMQ server will send data scraped from the specified web site back to the front end code that will render a new row in a table for each result found.

In order to put this into the context of my application, below is the code that both creates a callback queue for the RabbitMQ server to communicate with and publishes the request message to the RabbitMQ server.

def publish(urlSearch, routing_key, ws = nil) corr_id = rand(10_000_000).to_s connection = AMQP.connect(@conf) callback_queue = AMQP::Channel.new(connection).queue("", :exclusive => false) callback_queue.subscribe(:ack => true) do |header, body| header.ack ws.send body if ws end callback_queue.append_callback(:declare) do AMQP::Exchange.default.publish(urlSearch, :routing_key => routing_key, :reply_to => callback_queue.name, :correlation_id => corr_id) end puts "initial message sent" end

On line 2, I am creating what is well known in messaging circles as a correlation id. The correlation id or unique identifier will be used to correlate (obviously) or associate the RPC responses with the request.
On line 4, a RabbitMQ connection is made from the configuration hash specified in the pervious gist.
On line 6, I am creating the callback queue that the RabbitMQ server will send responses to. The important thing to note here is that I am specifying an empty string as the first argument for the queue creation. If you do not specify a name then an anonymous queue is created. The RabbitMQ server will assign the queue a random name which can be used later to let the RabbitMQ server queue handler know which queue to send the RPC response messages to.
Line 8 sets up the callback queue to receive messages from the RabbitMQ server.
Lines 9 to 11 contains the code that handles any message sent from the RabbitMQ server. Line 11 sends any results back the UI code if the WebSocket connection is still open. This will in turn, trigger a client side onmessage handler:

socket.onmessage = (evt) ->
  Lead.get('leads_controller').addLead evt.data

On Line 14 the append_callback method is invoked which takes a block of code as one of its arguments. The block will be executed when the declare event of the queue creation is triggered. It is important to wait until this event has been fired because we want to ensure that the anonymous queue has been created and the name of the anonymous queue is available to send to the RabbitMQ server.
Line 15 publishes the message to the RabbitMQ server. It is important to note the :reply_to and :correlation_id arguments that help tie the RPC calls together. The RabbitMQ server will now know the details of the anonymous queue to send any responses to.

The RabbitMQ RPC Server

So far, we have sent a message from the front end code containing any arguments required in a JSON string, we have also set up a callback queue that we can send any parsing results to and finally we have published the request message and specified the name of the anonymous response queue that we can send any results of the screen scraping to.

Below is the code for the rabbitmq server in nearly all of its entirety:

module Worker module App module Cli class Development < Thor namespace "worker:dev" @conf = { :host => "localhost", :user => "guest", :password => "guest", :vhost => "/", :logging => true, :port => 5672 } desc "start_consumer", "start the test consumer" def start_consumer AMQP.start(@conf) do |connection| channel = AMQP::Channel.new(connection) requests_queue = channel.queue("leads") requests_queue.purge Signal.trap("INT") do connection.close do EM.stop{exit} end end requests_queue.subscribe(:ack => true) do |header, body| url_search = safe_json_decode(body) puts "we have a message at #{Time.now} with search terms of #{url_search.inspect}" doc = open(url_search["search_url"]) parser = Parser.new(nil, doc) parser.find_potential_leads do |lead| puts "lead = #{lead.company}" json = lead.to_json(:include => [:emails, :contacts]) AMQP::Exchange.default.publish( json, :routing_key => header.reply_to, :correlation_id => header.correlation_id ) end end puts " [x] Awaiting RPC requests" end end

Line 17 defines the Thor task that contains the RabbitMQ server code. What I like about Thor is that a Thor task is just a plain old Ruby method defined in a ruby class that inherits from the Thor superclass.
Line 19 calls the AMQP start method that opens a connection to the RabbitMQ server.
Line 22 creates the queue that will listen for request messages from the RabbitMQ Client or websocket layer.
Line 33 uses the subscribe method of the AMQP queue object that takes a block as an argument which gets passed a message each time a message is published to the queue. The block takes 2 parameters, the header which is the metadata of the message and the body of the message which will be the JSON passed from the client. The metadata header is especially important in this case because it will contain both the correlation_id and the reply_to details that were created in the RabbitMQ client layer and give us all the data we need to communicate any results back to the RabbitMQ client layer.
Line 38 creates the parser object that does the actual crawling and screenscraping.
Line 40 calls the find_potential_leads method of the parser object that takes a block and yields any leads found back to the block that will allow me to publish any leads via the reply_to queue back to the websocket layer.
Line 42 serialises the result into a JSON string ready to pass back to the websocket layer.
Line 45 publishes the json string back to the RabbitMQ client websocket layer by specifying the repy_to and correlation_id from the header of the incoming message.

And that is job done, I have communication between my websocket layer and my RabbitMQ server layer using this quite simple RPC approach.

I have created the RabbitMQ server process as a Thor task. I like the name spacing capabilities of Thor that lets me execute namespace tasks with respect to their environment like:

thor worker:dev:start_consumer

I also like that Thor tasks can easily accept parameters which is in stark contrast to how Rake handles parameters.

Conclusion

I struggled for quite a long time to find something that fitted my needs. My next steps are to refactor the client code to use SockJS for better cross browser support.

If you disagree with any of the above or can point out a better way then I would love to hear from you, please leave a comment below.

Unit Testing Ember.js, How I Learned to Stop Worrying and Love the Runloop

Apr 3rd, 2012

I am currently using Ember.js for a side project and I have ran into some interesting and unexpected behaviour while driving the development of the front end code test first. I am using the excellent javascript bdd library Jasmine as my test runner and I must also issue a warning that the code examples listed below are in coffeescript. I have also been using the excellent guard-jasmine gem to automatically run my tests when any files are modified which really is a great experience.

I had vaguely heard of the Ember runloop in a few of the articles that I had read but I had not had to deal with it directly until I started getting more and more unexpected behaviour in my tests.

This is best illustrated with some code. Below is a simple Ember controller/object that creates a view in its constructor:

Lead.Controllers.UrlSearch = Ember.Object.extend init: -> @_super() @url_search = Lead.UrlSearch.create() @view = Ember.View.create controller: @ urlSearchBinding: 'controller.url_search' templateName: 'app/templates/url_search/show' @view.appendTo $('#fieldset')

The above code creates an Ember model object in the constructor of the controller on line 5 which is then bound to the view on line 9. The view is then attached to the DOM on line 12.

Below is my original failing test that I wrote:

describe 'Controllers', -> describe 'UrlSearch', -> it 'should create search capture dialog', -> controller = Lead.Controllers.UrlSearch.create() expect($('#goButton').length).toEqual(1)

I wrongly expected that the code on line 4 would create the controller and subsequently attach the view to the DOM immediately. What was equally as perplexing was that the view appeared to be immediately appended onto the DOM when viewed in the browser.

I threw this question out to the ever expanding universe (twitter) and the ever expanding universe (twitter) responded in kind. I cannot actually remember who answered (apologies) but the response was along the lines of:

Ember defers the DOM manipulation until later via the runloop. You can make the runloop execute immediately by calling Ember.run.end()

This advice was only half right, more on that later.

So what was this crazy man’s talk of a runloop? To understand what the runloop is, it is worth considering one of Ember’s strengths or main selling points which is its bindings support. A binding simply connects the properties of two objects in such a way that when one property changes, the value of the other one changes. With Ember you would generally not change the DOM directly, you would instead make a change to the model and let the relevant bindings reflect this change. This is MVC done properly with changes to view being reflected in the model via the observer pattern. With backbone.js, you generally react to DOM events which is quite limited because it is all too easy for the model and view to become unsynchronised and tedious error prone code is required to keep the two in sync.

It turns out that the runloop is a critical piece of the Ember machinery that ensures all bindings propagate data changes to their bound listeners. In the example I outlined above, I have the following binding that binds my model to the view:

urlSearchBinding: 'controller.url_search'

Then in my handlebars template, I am binding the value of an input field to the search_url property of the model:

view Ember.TextField size="60" valueBinding="urlSearch.search_url"

The upshot of this is that when a user enters some text into the input field, the search_url property of the model will change to the text that has been entered into the input field. The reverse is also true, a change to the model will result in the input changing. We could take this further and have another bound object whose property subsequently changes when the search_url property changes of our model and so on. This is could turn into a performance and synchronisation overhead if the Ember runtime was constantly reacting to each binding as and when they change.

Behold the runloop

The runloop is charged with batching up all the binding updates into one final set that can be executed all at once at a designated point in the runloop. Typically Something triggers the start of the runloop which is more often than not, one of the many supported browser DOM events that Ember has registered itself to listen for, e.g. click, mouseup, etc., etc., etc. The runloop can also be triggered manually with code or from the expiration of a timer. If we change a property using the set method of an ember object like below:

@url_search.set('search_url', 'http://thesoftwaresimpleton.com/')

The change to the binding is not propagated immediately but instead is placed on a queue or deferred for later execution at the end of a running runloop. When a runloop is triggered from a DOM event or manually in code, the bindings are not flushed until a certain point in the execution lifecycle of the runloop. The runloop is also in charge of other things like executing expired timers.

The important thing to remember here is that Ember waits until all the bindings have been propagated or flushed before updating any views which is why my code works fine when running in the browser but not in my test. The runloop had not finished and the bindings had not been flushed so the view did not update.

It is thankfully possible to manually start a runloop with code which turned out to be the missing link in my initial tests. A runloop can be manually started at any time by placing your code between an Ember.run.begin() and an Ember.run.end() statement. Once Ember.run.end() is called, you can be assured that all your binding changes will be flushed and any timers will be expired. Only after Ember.run.end() has been called will any views be updated. With this assurance, I can now test for the existence or changed state of DOM elements in my tests. Below is how the now infamous failing test can be made to pass:

describe 'Controllers', -> describe 'UrlSearch', -> it 'should create search capture dialog', -> Ember.run -> controller = Lead.Controllers.UrlSearch.create() expect($('#goButton').length).toEqual(1)

On line 5, I am triggering the runloop via the convenience method Ember.run which accepts a function as an argument. Ember.run() will take care of wrapping the passed in function between Ember.run.begin() and Ember.run.end() statements. Needless to say, the test now passes.

The runloop is now incorporated into the majority of my tests where I ensure that the unit or critical part of the application that I want to test is running in the context of a runloop. Below is an example of a test where I test my interaction with an Ember state manager:

describe 'StateCharts', -> describe 'rootState', -> it 'should transition to parsing state', -> spyOn(Lead.parsing_controller, 'start_parsing') state_manager = Lead.StateManager.create() Ember.run -> state_manager.send('startParsing', {search_url: "http://thesoftwaresimpleton.com/"}) expect(Lead.get('parsing_controller').get('isParsing')).toBeTruthy() expect(Lead.get('parsing_controller').start_parsing).toHaveBeenCalled()

On line 8 I ensure that the send method of the state manager is wrapped in a runloop.

Conclusion

I am not sure if I would have discovered the runloop had I not been writing tests. I think the runloop is an amazingly powerful concept and is used in many other frameworks and platforms. I think one of the IPhone frameworks uses it in a similar fashion.

Before I finish, I want to point out another trapdoor that I fell down before becoming better acquainted with the now infamous runloop. My first attempt at dealing with the runloop is in the gist below where I just blindly added Ember.run.end() to my test on line 5 in order to end the runloop and flush the changes:

describe 'Controllers', -> describe 'UrlSearch', -> it 'should create search capture dialog', -> controller = Lead.Controllers.UrlSearch.create() Ember.run.end() expect($('#goButton').length).toEqual(1)

This is bad for a number of reasons, namely because other tests will now rely on Ember.run.end() being called from this test and as we all know, tests should be atomic and not rely on things being in a particular state due to the execution of another test. If I delete the test, the runloop no longer completes. What is equally as bad is that the runloop is being terminated unexpectedly for other tests. This was another blind alley I gladly wandered down before deciding to do the research and write about it in this blog.

If I have got any of the above wrong then please leave a comment below. I would love to be corrected.

← Older Blog Archives