As soon as 3D Bookshelf is released, we're planning on running a small promotional competition on twitter. In this post, I'll briefly describe the architecture we put in place for managing the competition and providing a real-time view of the number of people who are participating.

Overview

There are many different ways to run a competition on twitter. Some companies have used hashtags to draw attention to their brand. Others use retweets to put their brand message in front of their followers' followers virally. Each method has pros and cons associated with it.

For example, the hashtags method can work well because it allows the participant to tweet what they want so long as they include the hashtag. On the other hand, it means that a valid participant may well be tweeting about the competition owners' brand in a negative way.

The approach we're favoring at the moment is to require each participant to follow us and then tweet a specific competition message.

Management

Regardless of the competition method, participants must be identified and remembered so that at the end of the competition each participant has a fair chance of being chosen as the winner (or one of the winners). For the type of competition we're favoring we have two choices. We can use the twitter search API to identify participants, or we can use the mentions timeline.

Using the twitter search API appears to be the best choice at first. It is subject to some limitations regarding the volume of results you can expect to receive. It is also subject to some date range restrictions. However, if it is polled often enough (within the rate limitations), these restrictions shouldn't be a problem.

Using the mentions timeline is slightly more complicated in that you are required to implement a little more management code, but it offers more reliability in helping enumerate valid participants.

The reason the search API is less reliable is due to a known issue with the API. Apparently, this issue does not affect the mentions timeline. As described on the twitter web site, "if you post an @reply to a specific user, these will be delivered to that user. If we're not experiencing issues, your account is public, your tweets have been posted for more than 24 hours, and you're still not listed, your tweets have most likely been filtered out of our search index for quality reasons."

Implementation

To manage the twitter competition, we wrote a simple service that polls the mentions timeline and harvests valid participants. It remembers the last maximum TweetId encountered and only requests tweets back as far as this TweetId on subsequent requests. It also honors the rate limiting guidelines.

For each valid participant, we fire the participant details to a very simple local web service for storage in a database. The 3D Bookshelf website then requests the number of participants from this web service.

We deliberately decoupled the database access from the web site and service so that both of these components can talk a simpler language to the web service. This simplified both the web site and the service and also meant we only ended up with one ORM cache running behind the web service. It also made system testing and unit testing shorter and simpler.

Strategy

Having a reliable engine in the car is only half the battle. We're learning as we go so we're not sure what to expect, but we do know that without a good map for directions we wont get very far. In a follow up post, I'll describe the strategy we're working on to push the 3D Bookshelf twitter competition as far as we can.

Check back soon for more. Or, you can sign up to my RSS feed or follow me on twitter for updates.

In an earlier post, I mentioned a behind-the-scenes screencast I was working on that shows the 3D Bookshelf prototype in action. That screencast is now complete and you can view it below.

The purpose of any prototype is to prove a concept to such a degree that further development is both possible and feasible. By starting 3D Bookshelf with this prototype we were able to assert that the iPhone & iPod touch (all generations) were capable of running the core engine at up to 60 fps and that our content requirements would be significantly reduced due to the real-time nature of the system. This in turn made estimation and planning much easier.

Shame on me, I'm still typing. Why not check out the screencast for yourself for more info!

Don't forget to subscribe to my RSS feed and follow me on Twitter for more updates!

Developing software costs money so it's important to make sure everyone on your team is moving towards a clear, shared goal to avoid wasted effort. There are many approaches to achieving your goal, and over the course of the last 20 years the software industry has seen them all.

Having spent a year researching Agile development (Scrum), followed by a year using it, I've seen the results of this approach first hand. In short, when you couple experience with a simple approach to estimation and tracking and add a capable Scrum master to manage impediments, your chances of successfully guiding a team to a goal, on schedule are very good.

You may have noticed the word 'experience' above is emphasized. This is very deliberate because this is where agile development (and Scrum) breaks down, and this is why agile can be the wrong choice when developing certain types of iPhone apps.

So what is it about certain types of iPhone apps (and a certain level of experience) that makes Agile the wrong choice?

Two things.

1. Often the goal of your team will be to add some magic to your iPhone app. This might be a 60 fps Doom renderer. Even with a well written spec, the actual final details of this magic may be unknown until well into the project. This may be because without prior experience writing an app just like the one you're working on, research is required. The results of research (and time required) are very, very difficult to estimate.

2. Some types of software development (like server side development) generally consist of well defined goals. The server receives A, stores B and returns C. An experienced developer can take this spec and provide a good task break down with estimates that will likely hold true. With most of the iPhone apps we've worked on, the same type of spec is usually present, with one added requirement. Quality. Achieving the required quality level usually means iterating over a project until a general consensus deems the app at a quality level ready for launch.

Point 2 could mean putting a beta in the hands of a sample audience and then responding to comments and requests that could dramatically increase the quality of you app. This is also very difficult to estimate and could mean a significant re-write if the quality of your app is of the utmost importance. 

Having said all of this, it is possible to use Agile to a high degree, when developing high quality iPhone apps.  For 3D Bookshelf, we identified the areas requiring research and scheduled them up front. This takes point 1 out of the equation. We accepted that we would spend a certain amount of time researching our options. This involved the production of a proof of concept prototype. We didn't know how long this would take. To keep things real, we did put a limit on the amount of time we could invest in this stage of the project. In the end, we got through it much quicker than anticipated, but we couldn't have known that in advance. With that out of the way, most of the rest of the project became very straight-forward to estimate and schedule.

Agile developers reading this are probably thinking, hold on, if you've put a time limit on the research task, surely this fits with Agile or Scrum. The answer is no. The results at the end of this stage could have an explosive effect on the rest of the project. For example, we could have tried to produce 3d Model content for the 3D Book system (and a lot of it) before we found out that it was much much simpler to produce a real-time engine to create the 3D Books. That's potentially a lot of wasted effort both in terms of management and content production.

There are other aspects to this project that were inherently suited to Agile development. These included the Twitter competition service, the web site and preparation of most of the content and tool updates.

The last few weeks were set aside for polish. That's where we are right now. We've said goodbye to schedules. That takes care of point 2. We're no longer trying to track this using Agile techniques. We know we're close to the end of the project. We know we can get to a quality level we'll be happy to release at. When will that be? We don't know yet.

Again, to be realistic we have set a limit to this. We can't throw infinite time at the project and expect to make any kind of financial return.  All I can say is we'll release between now and that cut-off date.

We've successfully applied the approach above before, to a previous iPhone project (a custom app for a high profile UK artist). Based on some initial research we produced a sprint task breakdown that came to 120 man hours. These tasks were completed in 131 man hours, only 11 hours over the estimate. Not bad considering the research involved (an advanced OpenGL renderer) and change requests. In the end we added a further 20 hours of polish at the request of our client. At all points in the project we had a good clear understanding of where we were in terms of the overall schedule.

Conclusion

Agile software development becomes quite powerful when it is coupled with experience and familiarity with the project domain. This is the guide we use for deciding when, and when not to use it.

Check back soon for more. Or, you can sign up to my RSS feed or follow me on twitter for updates!

For the last year, I've been using SCRUM at Pocket Kings here in Dublin. Although I finish up at Pocket Kings on the 28th of this month, I fully intend to continue using SCRUM for my own projects at Ideal Binary.

Version One is the SCRUM tool I've been using at Pocket Kings so that's what I'm familiar with. Once up and running it's relatively straightforward to use. It is an enterprise level tool though and if all you need is to track your own work (and not that of a team or group of teams), it may be overkill setting it up and hosting it.

My requirements are pretty simple. Essentially, I want a way to store my task estimates and also track my progress from the beginning to the end of each sprint. I also want to be able to get visual feedback (a Burndown chart) so that I can see how accurate my estimates are, and to what degree discovered tasks are entering the sprint, if at all.

Tracking discovered tasks can help identify improvements that can be made with future estimates. Also, being able to show a client discovered tasks on a Burndown graph can help them understand what can cause a project to overrun a due date, or why they may need to consider removing a feature if they want the project to be completed on time.

My team lead at Pocket Kings showed me how to get started implementing a simple system that can provide this with none other than Google Docs.

What I've implemented here is somewhat different from what I was shown, but it provides me with everything I need. Below is a snapshot of the spreadsheet with data from a hypothetical sprint. One quick point about this hypothetical data - in general I try to produce task estimates that are around one to two hours in duration. If I find I've estimated a task at more that this, I like to break it up into smaller tasks. For demo purposes, I haven't broken up the large tasks into smaller ones.

Anyway, to set the spreadsheet up, you enter a start date and an end date. Then you enter your tasks along with the task estimates. After this all you have to do is update the details of each task on a daily basis just like Version One. The formulas take care of the rest.

As you can see, below each task there are three rows. These are used to track changes in details relating to the task. These are:

• Estimated Remaining - This is the estimate for remaining work to get this task finished.
• Effort - The effort made to complete work that was planned for this task.
• Discovered Effort - The effort made to complete work that was not planned for this task.

Discovered Effort may be used to track required work that we simply didn't anticipate or it may be used to track feature creep. This happens when we or the client decide to add functionality during a sprint that wasn't planned. This isn't necessarily a bad thing, so long as it is understood that there is a cost associated with it.

Data from the rows described above are gathered from each task and totals are presented at the top of the spreadsheet. It's these totals that we use to produce the Burndown graph.

The Burndown graph for our hypothetical data is shown above. You can see we're a few days from the end of the sprint. Our Estimated Remaining work is burning down and is inline with the Target Burndown. You can also see we've hit some discovered tasks on the 4th and 7th day of the sprint. Regardless, visible effort is being made and we're burning down nicely. This sprint looks good.

The above system is relatively simple. I'll most likely end up installing Version One when my tracking requirements become more complex. The reason I'm using Google Docs and not Microsoft Excel is that I work with both Apple and PC platforms and it's nice to be able to access this stuff in an easy consistent way no matter what platform I'm using. Google Docs lacks some features present in Excel, but there are no show-stoppers here. Google Docs is also free.

I'm very impressed with what can be achieved with Google Docs.