Archives For Game Development

I was speaking about prototyping at a conference in Finland and got asked a question that left me fumbling for an answer.

“How much of my budget should I spend on prototyping?”

I’m insulated from budgets at work and don’t consider them at home. My pretty lame answer to being put-on-the-spot was that I didn’t really know and that I personally consider prototyping an incredibly important part of making anything so I would earmark a lot of budget towards it. It’s a pretty bad answer in hindsight because the most expensive parts of game development by far are content creation and marketing. Trying to place a dollar value on something like prototyping is hard as the value added is mostly a reduction in project risk. Asking “how much time and therefore money did we save by not doing something?” is equivalent to “how long is a piece of string?”

To me the value of prototyping is two-fold.

Value to the Individual

In games we’re seeking a to produce experiences that elicit emotions, the most common being enjoyment of something fun. There is no real way of knowing in advance if a game idea is fun or not or if its something you really want to make. Making videogames is an act of creation and it is a craft. Other acts of creation follow a similar pattern. Writers need to write a lot to get good at writing. Painters need to paint a lot to get good at painting. You need to knit a lot to get good at knitting. To get good at making videogames you need to make a lot of videogames. The more you explore and make things the better refined your process and skills become. You also gain a great deal of knowledge and experience which you can use as future shortcuts. Prototyping is essentially the act of making the core game experience or a feature as quickly as possible to see if it’s good or not. It follows that prototyping lots of small games, systems and other ideas is a great way to get better at making games.

Having a large portfolio of prototypes is also a great way of demonstrating a variety of skills to potential employers.

Value to the Business

As I said above the main business value of prototyping is a reduction in project risk. Early post-Waterfall methodologies noticed this for example Boehm’s Spiral Model. It’s should be pretty clear to most people that the first version of pretty much anything is terrible in at least one aspect. In the Spiral Model development doesn’t move through a series of stages ala Waterfall but in a series of iterations that result in a series of prototypes before the product is finally put into production. It recognizes that we can’t effectively reason about an idea without a concrete implementation. Project risk can be mitigated with prototyping in a few ways:

  • “Fail fast” – Today’s trendy business phrase. But as above there is real value in realizing early that you shouldn’t do something and not wasting resources chasing it in a heavyweight fashion until your realize it is unworkable or rubbish.
  • Improving communication – Ever sat in a meeting room to discuss a design document and found you spent the entire meeting squaring away different interpretations of the doc? Text is a horrible way to communicate ideas about a game which is a complex, interactive, aural and visual experience. You can see this in the drift as a younger generation rely on Let’s Play videos more than traditional journalism to make purchasing decisions. Simply watching someone play the game tells you more about it than the reviewers opinion of the experience put into print. The same is true for prototypes, they give a concrete entity for demonstration, reasoning and discussion.
  • Expose unknowns – Upfront design has long been fraught if you only have a muddy picture of your requirements. Nothing can get much muddier than games where the game at the end can often bear little resemblance to the initial idea. Prototyping at least gives you some insight into what the unknowns will be.
  • Avoiding “Sunk Cost” – Avoiding the sunk cost fallacy is at its heart not “throwing good money, after bad” or better still knowing when to kill a project despite having a large amount of money invested in it. Developing concrete prototypes makes evaluating the project much easier.

There is also less tangible value added by keeping prototyping in mind. Businesses are beginning to realize that like a server a person can’t run at a hundred percent capacity without loosing overall efficiency over time. Google made the practice famous but a whole variety of companies are letting employees take charge of a percentage of their work time. For example here at CCP on the EVE project we have twenty percent of our working time available for our own projects. It’s the lowest priority work and entirely optional but it’s there and people use it. Beyond providing a buffer for work tasks that take longer than expected it also makes sure teams aren’t burning themselves out. Encouraging prototyping in this time has a couple of benefits. Firstly the business can discover new business opportunities. For CCP that paid off in a big way with EVE: Valkyrie, even if the initial prototype hadn’t been greenlit into development the visibility raised must of been worth many times more in equivalent marketing spend than it cost the company. Granted the core team that made the initial prototype eventually put way more of their own hours into the project but it’s an interesting example of these things “going wild” and surprising the executive staff. Secondly it helps keep people motivated, no one enjoys drudge work but it often has to be done and spending a few hours each week on something you find exciting is good compensation.

Whilst I wouldn’t expect prototyping efforts to spin off new products regularly they do also give the business a bunch of products with some ground work already done they could evaluate and invest in when it comes time to do so. Hopefully this sort of culture will experience less downtime between projects and less project risk by having to start all the investigation of multiple ideas from scratch.

Finally the value added by each individual getting better at their craft is not only a tangible benefit to that individual but to the company as a whole.

How much of my budget should I spend on prototyping?

Let’s revisit the initial question. Prototyping is a tool to investigate and conquer the unknown so the budget requirement comes down to how well you really know what you want to make. I guess the answer I’m shooting for now is make it a significant part of your process, especially at project conception and during pre-production.

“Yeah, we’re doing Scrum but…”

Speak to nearly anyone in the games industry about their Agile implementation and this is what will come out of their mouths. It’s also a prevalent issue in Scrum adoption in other software creation industries. Leading to the term ‘Scrumbut’ being coined. It seems to signify, particularly in the long-term that a company has chosen to adopt Scrum, usually because they think it will fix some problem in the development process but has hit snags fitting their existing process into Scrum so they end up adapting the methodology they want to use rather than the process they want to fix. Then typically a project fails, the process is never fixed so the same problem rears its head and tears are shed. The collective take away often seems to be that Scrum didn’t work because it didn’t fix the problems. In reality it was the company or project that failed because they didn’t fix their own problems and saw/were sold Scrum as a magic bullet.

That said for any existing company and project there has to be a transition period between the previous way of working and the full adoption of a newly chosen Agile methodology. Many of the technical parts of Agile methodologies have a non-trivial amount of work to implement. For example setting up a continuous integration process and the suite of automated regression tests it runs. Further you can’t just bodge an existing set of processes into Agile methodology de jour and expect good results. Agile implementation is as much a reworking of culture as it is process. Any good Agilista will tell you we should be favoring people and interactions over processes and tools and the change in a companies working culture is probably the biggest impediment to changing process. In my experience this culture shift can’t just be a top down endeavour or a bottom up ‘revolution’ but something that is embraced at all levels and given adequate nurturing. This shift is ultimately about trust. Executives must be willing to let go of a strong command and control structure and developers ready to be open and honest about any problems they face so they can be addressed in a timely manner. The cultural shift to Agile also needs to extend to external partners/customers as working relationships will almost certainly need to change. For example you need a much more collaborative working relationship with your customer (but you know that it’s a tenant of Agile!) which for a lot of game developers means working closely with a publisher, a working relationship that doesn’t exactly have a reputation for trust in either direction! Everyone needs to have at least understood the Agile principles and what that means for their working relationship.

Some of this can result in some odd Catch-22s.

“We want to release more often but we don’t have confidence in the build, so we want to setup continuous integration, but we don’t have any regression tests, but we’re too busy working to ship the build to write them.”

Worse still there don’t appear to be ready, easy answers for how to transition any particular company or culture to any particular Agile methodology nor are there many/any games companies talking about how successful they have been. At best the advice seems to be ‘use this to fix your problems don’t expect them to magically disappear’.

In games the default methodology adopted appears to be Scrum which for a lot of reasons does not appear ideal to me for the totality of the lifecycle of a game. Games have distinct phases in their lifecycle, we don’t typically build out all the levels when the core gameplay is still being worked on extensively because it’s wasteful and we can look to Lean manufacturing to see some good discussion about why this is. Further some bits of game development like content creation which are much, much larger than in other software projects don’t really fit into the Scrum model. Games are very often boxed products or built as such and not sold as services so there needs to be a planning phase, however short and high level in order to set a clear vision, goals and build an initial product backlog. There are lots of fiddles like Sprint 0 added to Scrum to try to cope with these problems but to my mind Scrum needs to sit within a larger framework. Further Scrum doesn’t seem to scale very well beyond small teams and games projects are often huge in comparison to the ‘Scrum sweetspot’.

The really good news is that games development isn’t a unique snowflake in finding Scrum a bit lacking. In fact looking at data from the likes of the Ambysoft Surveys it’s easy to see that large teams in general are finding Agile adoption hard and that a lot of the barriers are very similar to the ones mentioned above. There are lots of other methodologies out there like Disciplined Agile Delivery that although not targeted towards quite the same projects as games try to deal with the larger complexity and team sizes found in enterprise IT projects. Whilst I don’t think they show the way out for games projects interested in Agile they definitely show the direction we should be looking towards.

As a software developer I’ve always had an affinity for the Agile Manifesto. However as someone who has spent their professional life creating games products I know that the environment stymies many of the bigger principles involved. The purpose of this post is really to state my experience before I try to go on and talk further about Agile so those words can be set in context.

Previous jobs and other group projects had no or ad-hoc project management but my introduction to Agile “in practice” was actually intuitive. It occurred whilst I was working at Realtime Worlds and had finished APB and there was suddenly a decision vacuum from our project leaders about what exactly we should be working on next. Previous suggestions were in the vein of “go wide” with game districts themed around things like a zombie apocalypse or a “no-rules” districts (which was awesome fun). The state of the project was such that many of us were deeply unhappy with the core game prior to shipping but could do nothing about it because we were in heavy bug-fix and polish mode. The project was organised in classic functional silos myself on a Gameplay Programming team with other silos representing special engineering interests and disciplines. A core group of us spread across these teams really wanted to tighten up and juice the core experience. In the decision vacuum I pitched an idea to organise a cross-functional team working iteratively to improve the core game experience. We had a set of really good ideas formed from feedback about where the main pain points were and how to tackle them. We knew we wanted to move quickly but not too far with each iteration so we avoided unnecessary work. We also knew we needed to work closely in a collaborative relationship between developers. In short we accidentally formed our own ‘agile’ team. Sadly Realtime Worlds died soon after we released our first iteration so we didn’t learn some of the lessons about project management I’ve since come to appreciate but the core of what made us successful was the core of what makes Agile tick day to day for developers. Caring about the customer, creating something simple in a cross-functional team, tight collaboration and improving on things in an iterative fashion. The work we started was continued by GamersFirst who transformed APB into APB: Reloaded a pretty successful F2P shooter.

After Realtime went bust I moved to CCP Games to work on EVE Online. EVE itself a marvelously interesting product but I was also drawn to CCP because they used Scrum and were organised in a way that I thought was closely aligned to how I wanted to work. After three years here working in myriad teams on myriad things I can state that it’s reasonably true, we may still be doing Scrumbut however people are at least receptive to working improvements and the EVE project is pushing towards different way of working and has come far. One of the nice things we have is a fortnightly Agile Community of Practice which is not a cabal of senior managers deciding working practices from on high but a collection of interested developers from the project discussing Agile and how we can improve our working practices. This has lead to some interesting developments in working practices with some teams now running on one week sprint cadences (after their Scrum master gave a great evidence filled presentation on why they should), the questioning of our own general working practices, release practices and such like. Last week I chaired a discussion where we took each Agile principle and looked at our how well we as a project meet it. It was illuminating not only because different perspectives on our situation were shared but because it made me question some of the validity of the Agile principles (or at least their wording) to game development. Working here has definitely shown me is that Scrum is probably not the answer (but it might be part of it) to large scale projects for a whole load of reasons. Both topics for future posts.

An amazing amount of Agile literature and methodologies have been generated by the general IT community but very little has been game specific. Over the course of this blog I’d like to try to put together my own thoughts on the subject into something a bit more organised that goes beyond the usual siren call to “adopt it and see magic happen”. A colleague Rob Galanakis is already blogging about this sort of thing and is part inspiration for restarting a blogging effort.

A talk I gave at Unite Nordic in 2013.

Basically an exposition of my thoughts on prototyping, how to do it right and what value it provides to game developers.