Presenting The EVE Method

This is my master thesis that I wrote together with Bo Behrmann Jensen, Frank Wisnes and Thorvald Kingbo at IO Interactive during our stay at the company from August 2006 to March 2007. In it we present our own game development method called EVE, which stands for Experimentation, Visualization and Evaluation.

>>> Download the entire master thesis here (PDF ~2 MB)

The following section is taken directly from the thesis. For the theoretical foundation that this method is based on please refer to the thesis in the link above.

The EVE Method
So far we have been looking at existing development methods in several different fields. We have discussed software engineering, innovation and game design theory in relation to game development. We also had a look at post mortem articles from game productions, and went on to identify some of the problems game productions are currently facing. In doing so, we found that the single largest source of problems is the pre-production phase and the process of verifying new ideas. This is where our method comes in. We want to give developers a set of tools to help them refine their ideas for game mechanics, a framework which at the end of the day makes the end product better and production run smoother. A small change in the beginning of the project can be all it takes, and in this chapter we will look at one way of making this change. We have called our method EVE: Experimentation, Visualization and Evaluation.

One of the main goals of our method is knowledge gathering. The more you know about an idea the better your decisions will be, and we believe in a hands on approach. In part, this knowledge gathering is done through early experimentation. The idea behind early experimentation is to look at an idea from different angles and through a refinement process of trial and error give the designers a better understanding of their own ideas. This can help reveal flaws in the design at an early stage of the process, saving considerable work compared to discovering them later on. Assuming that the initial idea is perfect is a high risk business, and experimentation helps reducing this risk. For this reason, creating a safe setting in which mistakes can be turned into cheap and valuable lessons is the first step towards implementing the EVE method.

Once that mindset is in place, it is time to start visualizing your ideas. While every idea usually starts on a piece of paper the written word will only take you so far. Nobody would even consider starting a game production without first making some concept art and the same should be true for prototyping game mechanics. That is why we suggest making lightweight prototypes as a way of giving designers an idea of how their ideas will feel when implemented in the game. A playable prototype makes it much easier to convey the idea to the rest of the development team, and the process of creating prototypes also gives the designers a better understanding of their own ideas. In addition, a playable prototype makes evaluation possible.

By evaluation we mean that constantly evaluating your own work as well as getting evaluation from others. By creating prototypes and experimenting with different solution, the designers create something tangible for them-selves and their team. Simply playing around with your own prototype can expose weaknesses in the design, and having other developers from the team try them can further help with this process. Another goal of EVE is to create a feedback loop for new ideas, where feedback is given as soon as possible so it can be taken into consideration. We have seen from post mortems that many developers went 6 months or maybe even a year into production before they had something playable up and running. We want to turn this trend completely around.

This method is designed for testing out new ideas, whether that happens in the pre-production or during the production. The thing to remember is that it is possible to evaluate elements of the game before the game is completely finished, and even before the design is finalized. The kind of prototypes we suggest can be everything from different colored dots moving around on the screen, testing A.I. behavior, to a fully functional inventory system implemented in the engine. As long as the process from idea to playable prototype is fast, the tools used to build the prototype does not matter. In pre-production the engine which will be used for the finale game might not be available or sufficiently understood, and in those cases it is better to use an existing engine to illustrate the different mechanics. Later in the process when a skeleton of the game is up and running, making something in the engine might be just as easy as using a different tool. As long as the prototype can be made in a few days, the tools used for making it are irrelevant. But there is one important point to remember; prototype code should not be reused in the final code.

If you are taking the time to make your prototype code of publishable quality, chances are you are wasting your time. This is the arena for hack solutions, shortcuts and dirty fixes. The goal here is quickly make something you can use to illustrate your game mechanics, not something you can copy-paste into the latest build. There is an additional pitfall here; the more time you spend polishing your prototype, the less likely you are to trash it if it turns out not to work. This concept scales perfectly: throwing away two days of work is easier then throwing away two weeks’ worth of work. Nobody wants to throw away half a year worth of work, and that is what our method is trying to save developers from doing.

However, some game genres will benefit more from this approach than others. For instance, strong story based genres like the adventure game will benefit less from this approach, yet it could be used to test mini-games and the usability of the interface. EVE is a method for testing new concepts and ideas for game mechanics, if a game’s main selling points are graphics and story there is less to be gained from using this approach.

In the following section we will try to elaborate and clarify what the three stages of the EVE method involves. EVE is more a mindset than a list of tools that are directly applicable in a game development. Lastly we will exemplify the method through a story which should give an idea of how to use the method in a more practical approach.

Experimentation
Failure is an option – the possibility of failure is something which every project has to deal with. Schedules, models, specifications etc. are all tools to minimize the risk of failure. However, precise as these may be it is still impossible to predict every situation that can occur in a production. The best way to prevent these situations in a production is to have tried them before doing it for real. So how do we do that? We suggest a way of working were you test the essential and cost expensive parts before you go into full production. By doing it in a much smaller scale it becomes less risky and more manageable to make errors. To cancel a project in full production can lead to economical disaster, so to testing the vital parts of a production in an environment which is both cheap and allows failure might end up saving the company a lot of money.

One thing is to be aware of failure; another thing is to embrace it. The essence of a preproduction phase is to test all the crazy ideas in an environment where failing miserably are indeed an option. To embrace failure in preproduction is to be financially responsible for the much more expensive production that follows. If you can weed out all the bad ideas by testing them in a cheaper preproduction you will save the effort and money when 50 people depends on your work in a later stage of the production.

Exploration as a design tool – it takes a thousand bad ideas to find a good one and to explore the bad ideas will help you understand the good ones. Exploration is easier said than done – it requires a lot of discipline and creativity to explore alternative solution to a problem you already solved. To explore different solutions is like searching for a proof of concept. You might find the perfect solution or you might not, but no matter the outcome the time spent should give you more information and by doing so increase your chances of making the right decision.

Set-based design is a way of thinking where the explorative element is an on-going process. You keep developing multiple prototypes of the same ideas were the one which fits the overall design best will be the one used. This way of working ensures that you always have a game composed of elements which work together as one unit. If something ceases to work in the context of the game it can simply be replaced by another element which have been tested and evaluated by means of prototyping. To try many possibilities without any prejudices is the key element when working exploratively.

Visualization
Playing is believing – one picture says more than a thousand words, which is the concept of visualization. Computer games are not something you can easily imagine or explain. You have to try it and in order to do so you have to build it or parts of it. What might seem fun on paper can turn out to be tedious when playing it. Our intention is to introduce the tangible element in game development as soon as possible through the concept of lightweight prototypes. The use of these makes it possible to clarify by means of playing if and how an idea works in relation to the overall vision. They act as a small confirmation test from which you can draw information on whether or not the idea is worth pursuing.

Lightweight prototypes does not mean first playable – the term prototype is used in many situations throughout the game industry and have different meanings. However where these often represent a bigger fraction of the finale game, a lightweight prototype is made to illustrate the smallest parts of a game – game mechanics. Prototypes such as first playable, vertical slice, alpha, etc. are all an important part of a game development, however they serve a different purpose than a lightweight prototype and when compared the amount of time and resources spent separates them clearly from lightweight prototypes.

The production timeframe of a lightweight prototype should be no more than a few days, even hours if possible, since the idea is simply to gain enough knowledge about the idea to be able to make a sound judgment regarding the future of it.

Keep it simple – lightweight prototypes are not just a way of working but also a way of thinking. It is easy to make things more complex, but the point of making a lightweight prototype is to illustrate the basic idea. The urge for doing beautiful graphics, animations, story etc. must be downscaled, since you only risk wasting valuable time at this point. That does not mean you should exclude everything that makes a prototype look great – of course not. However, looks or other features must never be prioritized over the game mechanics. If an idea cannot stand on its own without cool graphics or impressive sounds, it probably is not such a good idea anyway.

Single out the key elements – What is the core of your game? What do you want the player to experience? Why is this element important for the game? These are all questions which can help you find the core of your game – the game mechanics. Before the prototyping phase can start a clear understanding of what these are must be in place. The use of keywords, brainstorming and “the six thinking hats” method are all ways of working which can help you find and understand the game mechanics. If you end up with multiple game mechanics – which will probably happen – prioritize them after level of importance and use this as a starting point for the visualization phase.

When doing lightweight prototypes, keep the scope at a manageable size. Each game mechanic should get its own set of prototypes, and trying to prototype several mechanics at once will risk clouding the output. It is easier to test the idea if the variables are kept to a minimum. Make many small prototypes instead and test them individually. This should give a much clearer view of which ideas that could work in a final game and which ones should be scrapped, and once tested the good prototypes can always be combined later for further testing.

Abstract prototypes – a reason for using prototyping is to quickly reach a point where something playable provides a clear picture of the vision. Using a lot of time on fine tuning certain elements and details like graphics and sounds will only increase the impression slightly – a red dot which takes 1 minute to make can at this point work fine in contrary to using an animated character which took you numerous hours to make. One could argue that it is the details which differentiate great computer games from bad ones. True, however, it is testing and evaluation which is the goal of the visualization process, so save the details for the final game and instead use the time to explore other facets of the idea. Roughly said the process of making prototypes is about quantity and not quality – to explore the different possibilities by means of many prototypes instead of using time polishing a single one.

Modular prototypes – the success of a game production depends on how well you handle changes. Even small developments will face the need to change something at some point so you can either plan for it in advance or face the problems as they emerge. The use of prototyping is a perfect way to plan for changes. The idea is to think of lightweight prototypes as Legos; each prototype is a module that when, combined with others, is united to form something greater. Each game mechanic is tested through numerous prototypes where each implements a different version of the same mechanic. When combining the ideas in a final game it becomes easy to exchange one mechanic for another if you realize that it does not fit or the design has been change in some way. Modular prototyping is about having more red LEGOs ready if the one you tried first does not fit your house.

Evaluation
Play testing is a never-ending process – the first step to improve something is to understand where and why the flaws are and the only way to do this is through evaluation. The process of using lightweight prototyping is a dynamic and ongoing development of ideas which consequentially limits the possibility for formal and structured testing sessions. Instead testing must function as a natural part of the design process. Self-testing is a way to approach evaluation where playing the prototypes should help to understand what and why something is done wrong. Through this continuous testing you gain a broader understanding of the idea and the context in which it functions. To ask yourself questions concerning the idea and the relation to the rest of the game and to compare it to similar prototypes will help you see new perspectives or even whole new solutions to the problem.

However the most important way to gain an understanding of the idea is to have other people evaluate the prototype. Feedback is by far the most rewarding way to evaluate anything and neglecting it will only hurt the game. The use of confidant-testing fits the lightweight prototyping approach were colleagues or other confidants act as test persons. But it is important not to make this into a formal test session; at this point such an elaborate setup will be a waste of time. Instead, invite a colleague in for a cup of coffee while you show your work and simply make a mental note of the things he or she says and does relating to the prototype. You will go back to working on the prototype the minute your colleague leaves anyway, so there is no need to document the feedback more extensively then on a Post-it note. All information is welcome at this point, and you can always dismiss irrelevant comments later on. Remember, you are not building a game at this stage, but simply exploring ideas, so to receive all the feedback possible will only increase your chances to build a great game later on. Furthermore, the input from others helps you to see new sides and by doing so prevents your ideas to be single-minded.

Make throw-away prototypes – so finally you managed to make a really great prototype which is ready to be implemented in a final game. However tempting this might be it is important to stress that this should not be an option. Do not expect your work to be used for anything else than evaluating the concept. The prototype should be nothing more than a source to extract information from. When making prototypes there are no rules of how to make them and there should not be since the goal is to get something up and running as soon as possible. To demand that the work done can be used in a final game only limits the creative process and makes people think about is-sues such as implementation, appearance and re-usability which are not important at this stage – when doing lightweight prototypes it is always more important to focus on what you made instead of how you made it.

This way of thinking is something which needs to be adopted by the entire development team in order for lightweight prototyping to work. Appearance and eye-candy is an effective way to sell products that lacks quality them-selves and to promote this as an important feature when prototyping re-moves the focus from game mechanics and onto something which this process is not intended to be used for. To make lightweight prototypes is to accept the absence of high value concept art, setting, and sound and to focus on game mechanical issues exclusively.

Be proud of your work – in order to receive feedback you need to show your work to others. While this sounds good on paper, it is easier said than done. To show a prototype in which you have spent valuable time implementing a couple of dots that moves around does not seem as a good way to impress your boss. However, for a developing team using lightweight prototyping it is important to promote this behavior. If not, you risk that people start prioritizing visuals over game mechanics in order to “sell” their product better to colleagues. When working with lightweight prototypes it requires a show-off element among the developers for the process to work – everyone has the same goal and everyone should appreciate the value of a lightweight prototype.

Make decisions as late as possible – the information regarding game mechanics may be somewhat limited in the beginning of a production and to base the entire game design on these may turn out catastrophically. Instead of making important decisions at such an early stage of the production, the rational thing would be to wait until you know more. Evaluation of lightweight prototyping should be the key element in obtaining the knowledge needed in order for you to make a reasonable decision. Does it take one, five or ten prototypes to give me the information I seek? Well, the answer is of course not black or white, however, a rule of thumb is to keep prototyping until you simply cannot wait any longer to make a decision. The idea is to make important decisions regarding game mechanics at the last responsible moment since you might gain new information which changes the context and requires a new decision, making previous decisions a waste of time.

A day in the life of a rapid prototyper
An idle Tuesday morning, John is pouring his first cup of coffee for the day. As a designer on the company’s latest project he’s got his work cut out for him, and the expectations are high. Walking to his desk he runs the combat mechanics over in his head. They are still only a rough sketch, but the basics for the third person shooter are there.

Coffee in hand, John heads for his workstation. But before he gets there, Peter, one of the other designers, grabs his arm – his latest prototype is ready and he needs some feedback on it. Peter’s prototype is simple; a box moving through a blocky environment and a movable crosshair corresponding to the Xbox controller which has been hooked up. Pressing 1-5 on the keyboard cycles through different possible control mappings, allowing for a comparison later on once more pieces are in place. They discuss the prototype for a few minutes before John returns to his workstation.

On the wall in their office hangs Post-it noted describing different game mechanics, ranging from inventory system features and reload mechanisms to special AI behavior and suggestions for puzzles. John pulls down the note named “active reload”, reading the two line description as his computer boots up… “Make reloading more interesting by adding an element of timing”. He starts working on a small prototype, spending the first half of the day experimenting with different ways to implement this element of timing. Throughout this process all the designers are there to provide each other with feedback once something is ready. As soon as something playable is up and running, one or both of the other designers spend a few minutes trying it out and commenting on it.

Once the prototype is good enough to explain the idea, a short description of the prototype is written in the wiki along with a link to the file. Their wiki allows everyone to get a feeling for how the project is coming along. The publisher can see which direction the game is taking, and people on the development team can try the different prototypes and comment on them.

>>> Download the entire master thesis here (PDF ~2 MB)