Notes

• The final tool we used when implementing the design side of the AI was Playtesting

  For our playtests we bring in between 25 and 35 non-hardcore game players who have never played the game before

  we put them in a lab and let them play for an hour

  then we have them fill out a survey about their experience

  
  
  we've found this gives us the most objective results and lets us act on the feedback with the confidence that it is accurate

  
  
  As a brief aside:

  If you aren't doing playtests you should be

  if you are doing playtests you should be doing more playtests earlier in your development cycle

  you can't have enough playtesting

  
  
  We actually have the designers watch the playtests

  because nothing forces you to confront reality faster than watching 30 people bang their heads against a level

  until they hate the level, they hate the designers and they hate the game

  
  
  In fact, during an early playtest we had more than half of the people ask if they could stop playing and go home

  The desire to get people to stop wanting to quit was a major motivation for our early design process

• The most useful data we got from our playtests was a list of things to avoid

  At the top of that list is Subtlety

  If it isn't totally obvious, it's too subtle

  Even if you make something as obvious as you can possibly make it, half the people will miss it the first three times they see it

  In Halo the Grunts run away when an Elite is killed

  Initially nobody noticed so we had to keep adding clues to make it more obvious

  By the time we shipped we had made it so not only does _every single_ Grunt run away _every single_ time an Elite is killed

  but they all have an outrageously exaggerated panic run where they wave their hands above their heads

  they scream in terror

  and half the time one of them will say "Leader Dead, Run Away!"

  I would still estimate that less than a third of our users made the connection

• Another thing to avoid is letting your AI Look broken

  When it comes to AI most Players are looking for any excuse to break their suspension of disbelief

  they will see flaws even where there aren't any and jump all over them

  For Example: Early versions of Halo's AI had a short pause between seeing you and recognizing that you were an enemy

  this was to allow people to get the jump on the AI or retreat if they were getting in over their heads

  Playtest feedback showed that Player's assumed this Feature was a Bug and they all said it made the AI look dumb

  We ended up completely removing the perception time on all the characters

  because even though it was intentional people interpreted it as the AI being broken

  Insufficient challenge

  I talked about this earlier but it's important enough to reiterate

  Easy enemies are assumed to be dumb

  Playtests are perfect for establishing how difficult the AI should be to make it look smart

  The other useful feedback we got from our Playtests was a clear idea of what things we could refine

  We could make sure that the thought processes we were trying to communicate where in fact getting across

  We also learned which of our animations were being interpreted correctly and which ones were confusing to the Player

  One of the most important things we were able to refine because of our playtests was our engagement distance

  Nothing breaks our AI faster than engaging it from the wrong range

  Sniping makes the AI look oblivious to danger

  fighting too much from melee range makes the AI look robotic and awkward

  We were able to use playtests to tweak our weapon ranges and activation ranges

  to keep the Player at the optimal engagement range for our AI