In their latest interview from the PAX East convention of 2011, Darth Hater sat down with Bioware’s creative director, James Ohlen.
Darth Hater: “We also heard that in your dialogue system, you are tracking metrics on when people are pausing to see how long they’re taking to make decisions and things like that to really work on the dialogue. So you are getting ton of information from the testers?”
James Ohlen: “We keep on getting more and more information as the testing groups get larger. We had two groups of a thousand players on a shard and that got us a lot of information because it was a lot more people playing together. We were able to see a population that was not indicative of the final game, but it was closer to it because the populations were so much bigger than just a couple hundred. We’re able to use that to make a lot of adjustments; it also validated a lot design decisions we made months earlier. The game really gets a lot better every single month. It is pretty impressive. The developer teams are getting faster at making changes and making adjustments to feedback. Two years ago it would take so long to see a change in the game, but now you see changes every month.”
From what I remember in the past, we have never really gotten a definitive number from a developer about just how many people are involved in “game testing”. If I remember correctly the only estimations of a number came from the statement of “more than I could fit in my office” or something to that effect. (the actual quote escapes me at the moment but it was very similar to that statement)
When they tell me that they have two groups of a thousand players on a shard that gives me a better idea as to how far they are along in the numbers of people that they have allowed in at this point. I figure it is more than two thousand at this point, but at least I now have a baseline reference.
The comment that they were “on a shard” also makes me wonder if they also have the ability of running more than one shard at this time with their current server structure. I am thinking they do and that they may be currently operating several shards at this point in time. Maybe a private in house version for the developers only that showcases the latest builds, maybe a couple for outside the office testers, and etc. I may be way off base, as this is just speculation on my part.
In the end though, with this developer interaction I feel better about the testing process going on at Bioware. One of my fears from the past was that they didn’t have the amount of people testing their game that they needed to find out all that was necessary. That a small minority of people testing would sway the design decisions in directions I as a MMO veteran did not approve of for my gaming time.
With a number of at the very least two thousand testing the game, I feel much better that a wider view of MMO mechanic and game play design is being addressed.
Now if I could just get my hands on it and test out the “Crew skills system” I could die a happy man. 🙂
