Boken design series: Gamevil's Fantasy War

With this post I will start a series dedicated to broken design. What's broken design? It's about things that we use that are fundamentally flawed. It's about stuff that seems obvious to some people but not to product designers.
I will start the series with a post about a game for the iPhone that I recently tried. The game is called Fantasy War, produced by Gamevil, and can be found on the Gamevil website and on iTunes.
The purpose of the game is to become the leader of a fantasy realm. You can be human, orc or elf, and you progress through the game by completing some quests, battling the forces of evil in some dungeon, and waging war against other players. As you accumulate experience and resources, you can improve the infrastructure (gold mine and lumber mill), learn new skills (cloth making, leatherworking, etc), upgrade your weapons, and hire stronger and more powerful armies. The graphics resembles some 1990s arcade games, but it's still not bad for a mobile game. The rules are fairly quick to learn without explicit instructions and the level progression gradually unlocks more interesting content. The developers make money selling "runestones", which allow you to do a lot more stuff and get really powerful armies.
Sounds ok, right? Well I actually like the game. It's very easy to play and the content is not bad at all. So where's the problem? What's the broken design?
Well, the problem is that this is not an iPhone game, or at least it's not meant to be. When you install the game, you actually install a simplified internet browser with little or no content. Every time you interact with the game, you actually navigate through a web page. All commands, all icons, all images, are just hyperlinks on a web page. Hang on, so where's the problem? I use internet all the time with my iPhone, and I have a lot of games on my iPhone that need an internet connection to work. No big deal. No, indeed, it wouldn't be a big deal... if the game wasn't so connection greedy! There is no content on the iPhone. Everything needs to be downloaded from the internet, including all the images (and there are a LOT of images!). In other words, this game assumes that it is running on a permanently connected device on broadband speeds... which is exactly the opposite of what an iPhone is. My user experience with this game goes something like this:

1. start the game
2. watch the rotating "loading..." icon for a few seconds
3. watch a dark screen behind the "loading..." icon
4. watch the text labels of the home screen appear behind the "loading..." icon
5. watch the graphic background appear behind the "loading..." icon
6. watch the images loading in chunks behind the "loading..." icon
7. home screen fully loaded
8. tap on any activity
9. watch the rotating "loading..." icon for a few seconds
10. repeat from point 3

In the end, I spend more time watching the rotating "loading..." icon that actually playing the game. Add to this the fact that there is no fault tolerance built in this simple browser, and you also get to build familiarity with the "check your internet connection" error, after which you can only shut down the game and start it again.
Also add to this the fact that dungeons are time limited (you need to kill the boss within 'N' minutes, or you have to wait for an hour or more before you can try again) and you can see how you can quickly build up frustration.
The cherry on the cake is that if something goes wrong with the game you get some Korean error message popping up, which is ok if you can read and understand Korean (I can't do either).
The verdict: broken design!
This is a basic architectural requirement for anything that is supposed to run on an iPhone: it must (please note, "must", not "should") be built for a seldom connected environment with less-than-broadband speeds. If that's not the case, then don't bother.

Architectural Inertia

Inertia: indisposition to motion, exertion, or change (source: Merriam-Webster Online Dictionary)


What happens when you subscribe to some web-based service and then you forget your password? No big deal, you can usually click on a convenient hyperlink next to the logon box labelled "Forgot your password?". That usually takes you to some sort of password-retrieval process involving the service provider sending you an email with a specially crafted URL; you then open that email, click on that URL, and enter a new password in the web form that has just opened up for you. Job done.

There are many variations on the theme, some of which are more involved than others, for example asking a whole set of personal questions to confirm your identity, but the general idea is always the same, and in many cases you can also retrieve your username through a similar procedure. These are good examples of helping users help themselves when a problem arises, or automated technical support.

Web applications, however, are moving away from usernames in favor of some other identifier that is more directly linked with the user, like the user's primary email address.

A number of web application designers, therefore, thought it would be a good idea to keep the same theme for automated tech support by offering right next to the logon box a convenient hyperlink labelled... "Forgot your email address?"

:-)

The Cloakroom Pattern

I am writing this in the context of a web application.

In the "good old days", browsers were single-viewed: there was no such thing as tabbed browsing. If you wanted to browse multiple web pages at the same time, you had to open multiple browser windows. Web architectures, application servers and frameworks have evolved to satisfy this single view of the world by managing context at three different levels: application, session and request.

A request exists when the client sends a message to the server. The moment I open my browser google up some stuff, a request springs to life. This can store transient (short-lived) information, like the terms I want to search for, which are unlikely to be re-used after the request has been satisfied.
At the application level, a context exists for the entire life of the application, until the server is restarted. For example, I might need to audit certain user activities to a database, so I might want load the database connection details when my application starts up and store them in the application context.

Somewhere between the application context and the request context, a web application might also want to manage a user session. For example, when I am browsing my web mail, there must be some kind of information that persists between one request and the other, so that for example I don't need to login again if I move from one mail item to the next.

How does caching takes place and why?

Let's say I sell pizza, and that part of my web application has a search function that goes through my catalog and returns to the user all the pizza styles that match the user's choices. Let's also say that the application stores the search criteria in an object that can be cached, let's call it searchObj, so if the user refreshes the page (without changing the search criteria), the application saves time and resources by simply re-using the same data instead of making a new round trip to the database.

According to what we said above, if searchObj needs to be persisted across requests, it makes sense to cache it at the session level.

So here I am as a potential customer using this pizza web application, searching for pizza that contains ham, so I type "ham" in the input box, click the submit button and look at the resulting list. All the listed pizzas have ham in the ingredients. If I happen to refresh the browser, the application simply re-uses the same list without making a new round trip to the database.

Now let's say I open a new browser tab (not a new browser window) to display the results for a different search. This time I want to search for pizza that contains olives, so I type "olives" in the input box, click the submit button and look at the resulting list. All the listed pizzas have olives in the ingredients. Great.
Now I go back to the previous browser tab, the one with ham-based pizzas, and hit the refresh button. All the listed pizzas now have olives in the ingredients.


What happened?

It happened that searchObj was overwritten by the second search, but how?
Let's think of this scenario in a different way. Let's say I need some milk, and that I suffer from particularly bad memory, so before I forget I decide to write "Milk" on a post-it note and stick it to the door, then I go to get your jacket, car keys, etc. Now let's say my lodger, the sentient invisible pet chimp Iain, needs some fruit juice but instead of writing "Fruit Juice" alongside "Milk" on my post-it note, he decides to replace my post-it note with another one saying "Fruit Juice". Now I'm ready to go out, but of course I have forgotten what I needed to buy, so I pick up the post-it note from the door and happily go on to buy... fruit juice!

In this example, the post-it note is searchObj, Iain and I are the request-scope beans activated from two different tabs of the same browser, and the door is the session. Assume my house only has one door, the entrance/exit door (multiple tabs on the same browser share the same session).


How can we solve the problem?

In terms of "post-it note on the door", it's fairly easy: we draw two squares on the door and label them "Marco" and "Iain". Now we can use our own post-it notes, as long as we stick them in our own designated area on the door.


How does that translate into a web application?

We need to think of this type of context as sitting somewhere between the request scope and the session scope. If we think of each browser tab as a different "view" of our user session, then we can talk of view-level context and view-scoped objects. However, this is not a built-in functionality in the well-known web application frameworks or containers, so we need to simulate it, but how?

In the above example, we said the door represents the session, so we need to stick into the session some kind of container that can hold labelled compartments. How about a Map, for example a Hashtable? Yep, could do with one of those, but how do we actually generate the keys? In other words, how do we make sure that each tabbed view of the same browser unequivocally identifies itself when posting information to the session and retrieving information from the session?

I'm not sure we can handle the "unequivocally" part, but here's what I would do: I would use the coat check pattern, also referred to as the cloakroom pattern. I don't think you'll find that in reputable software engineering books, so don't bother looking.

This is a snippet from Wikipedia's definition of "Cloakroom".
"Attended cloakrooms, or coat checks, are staffed rooms where coats and bags can be stored securely. Typically, a ticket is given to the customer, with a corresponding ticket attached to the garment or item."

In particular, you'll see that tickets are generally given away in sequential order, and that you don't actually need to show personal ID documents when picking up your coat: you simply produce the ticket that was given to you when you gave them your coat. For our web application, the tickets are issued by some sort of decorator class that intercepts HTTP requests and does something like this...

1. check if there is a querystring variable called "coatNum" (or whatever you fancy)
2. if there is one, do nothing, otherwise increment an internal counter and use it to decorate the querystring by appending the "coatNum" variable name and value

For a JSF application, this might be a phase listener (maybe for the RESTORE_VIEW event?). For an IceFaces application, things have already been worked out in the form of the "rvn" querystring variable.

For added security, some might argue that the view number should not be easily guessed, so sequential numbering is actually discouraged, but remember that we are talking about different tabs or views of the same browser window. In any case, just for clarity, I will stick to a counter that gets incremented every time it's used.

There is a second part to this: once we know what view number the request wants to use, how do we use that view number to organise our view-scope objects? We said we could use a Map, but where does that live? In the real life coat check scenario, let's say at a classical concert, there can be multiple coat rooms, with each coat room used by multiple punters. This might suggest that the Map holding view-scoped objects should be application-scoped. Even though it is possible to do so, it would require additional overhead in terms of resources, because *all* view-scoped objects for *all* users in the entire application. Also, we would have to write additional code to manage object expiry and cleanup, otherwise we would see the Map growing to infinity and beyond. There are also some security and privacy concerns, since every request would have access to *all* view-scoped objects.

One solution is therefore to stick the Map in the session, or a session-bound bean. As a result, the internal counter that identifies the view number must also be session-bound, so that it starts at zero every time a new session is generated.

In summary, here is what I would do every time a new request comes in:

1. use an internal session-bound variable to generate view identifiers (e.g. a counter) and a session-bound Map to cache view-level objects
2. intercept requests and check for a query string variable that identifies the view number
3. if it's not present, then decorate the query string with a variable that identifies the view number
4. if the session-bound Map already contains an object for the given view number, then discard the object received from the request and re-use the cached object instead, otherwise take the object from the request and cache it
5. process the request and return a response to the client

Hold on a minute... what if the request actually uses more than one object?


In that case we don't simply have a session-bound Map, but rather a Map of Maps. In other words, the session-bound Map will still be keyed by view ID, but the value for a given view ID will be a Map of objects, keyed by object ID. We can therefore talk about a "view map" being a collection of "object maps".
This is the revised workflow:

1. use an internal session-bound variable to generate view identifiers (e.g. a counter) and a session-bound Map (the view map) to cache view-level object maps
2. intercept requests and check for a query string variable that identifies the view number
3. if it's not present, then decorate the query string with a variable that identifies the view number
4. for the view-level object that should be cached, find its identifier (might well be a simple obj.toString() call)
5. if the view map contains an object map for the given view ID, then retrieve it, otherwise create a new object map and put it in the view map for the given view ID
6. if the object map contains an object with the same object ID, then discard the object received from the request and re-use the cached object, otherwise take the object from the request and put it in the object map
7. process the request and send a response to the client

The mechanism can also be extended with an expiration algorithm that kicks in on steps 5 and 6, so that cached objects are refreshed from time to time if needed, but that is another matter altogether.

Architectural Agility - Part 1

Software Engineering is a wonderful world full of surprises and challenges. We learn early on in life that any activity that floods your system with adrenaline is an addictive activity.
In the life of a ten year old, this might take the form of climbing a tree for the first time, ascending to seemingly unreachable altitudes with a non-zero probability of following that up with weeks of hospitalisation.
In the life of a programmer, this might be the result of producing thousands of lines of code under deadlines that you never considered even remotely realistic.

Let's face it: the feeling of impending doom is just great.

In the life of an architect, the main perverse sense of doom and destruction originates from the fact that you are supposed to shape the system in your head, then somehow implant that picture in the heads of over 50 developers, and pray that you've been clear enough with your specifications.
The uninitiated might rightfully ask "Why? Just write the darn specs and pass them on: if they're good developers, they'll work them out..." Alternatively, "Ever heard of agile?".

Well, I don't know about the rest of the world, but in my projects I've never managed to do either. Usually, the situation involves very vague or even yet-to-be-discovered requirements, and only one or two agile developers in a team of 50-plus. So how have I managed so far? Well, it's surely a continuous learning for me, and I still discover something new almost on a daily basis, but here are some points that I have picked up along the way and found very valuable.


#1 : Learn how to produce on-the-fly specs
Why? Because on one hand I have this very hazy and almost monochromatic picture of the requirements, on the other hand I have 50 developers expecting fairly detailed specifications of what they are going to produce, and somewhere in the middle I have a bunch of reasons for being unable to cultivate an agile team.

#2 : Learn how to be ALWAYS available for the team
Why? Perhaps it's just me, but on-the-fly specs are NEVER good enough.


#3 : Learn how to monitor progress all the time
This doesn't mean to become a control freak, but rather to understand the dynamics of the project and minimise the risk of producing the wrong thing due to sub-optimal specifications. Why? Because 50 developers, over (officially) 8 working hours in a day, makes 400 hours/day of code writing and testing time. In one week, that's at least 2000 hours, or about one developer/year worth of effort. That means that failing to convey an architectural concept for the system and leaving 50 developers alone for a whole week translates into seriously refactoring one developer/year worth of code, which is not something you usually do in your spare time. It's like steering the proverbial oil tanker: one mistake in plotting the route, and it will take considerable effort trying to get it back on the right track if you don't spot the mistake right away.


#4 : Functional refactoring is inevitable
Why? Well, due to all of the above. However, I tend to view refactoring as falling into one of three categories: architectural, functional, and schematic.
Architectural refactoring is serious: that is what happens when we change parts of the architecture , for example when half way through the project we realise we need a caching framework for our web application.
Functional refactoring is somewhat less serious and could be considered a minor version of architectural refactoring: that happens when we change some of the application's behavior, for example when we move hard-coded values to some configuration file, and surely when we are bugfixing.
Schematic refactoring is standard routine: that is when the application's functionality is unaffected while changing its internal structure, for example when we abstract common functionality from several classes into one parent class, or formalise common class contracts into interfaces. I'm now learning to shape functional refactoring into an agile project in its own right, and probably write some considerations on that in another post.

M.