Patch 5.3 interview with Ghostcrawler
Mystery of the Unborn Val'kyr
The latest patch 5.3 news
All of the latest Mists of Pandaria newsBlizzard is tracking 180,000 bugs in WoW
by Adam Holisky 
Sep 17th 2009 at 1:00PM
During the keynote today at the Austin Game Developers Conference, Executive Vice President Frank Pearce and Production Director J. Allen Brack spoke at length about the internal workings of the WoW team and how they get their jobs done.
One of the more stunning things to come out of the keynote, which we'll have fully written up for you later today, is the fact that there are just under 180k bugs Blizzard is tracking in WoW. That means their bug database has 180,000 entries which are in some stage of being fixed (have been fixed, have not been fixed, or being worked on).
To me this number seems very large for a video game. I can understand an operating system like Windows 7 having an unreasonably large number of bugs in it like this, but for a video game -- even one as complex as WoW -- that number is quite astounding.
It does raise the inevitable question: what is Blizzard doing to fix all these? And how does this relate to the extremely long wait times for GM contact in game? We also learned that Blizzard only employs 2500 worldwide in Customer Service. That includes things like phone bank operators, GMs, forum mods, etc...
One of the more stunning things to come out of the keynote, which we'll have fully written up for you later today, is the fact that there are just under 180k bugs Blizzard is tracking in WoW. That means their bug database has 180,000 entries which are in some stage of being fixed (have been fixed, have not been fixed, or being worked on).
To me this number seems very large for a video game. I can understand an operating system like Windows 7 having an unreasonably large number of bugs in it like this, but for a video game -- even one as complex as WoW -- that number is quite astounding.
It does raise the inevitable question: what is Blizzard doing to fix all these? And how does this relate to the extremely long wait times for GM contact in game? We also learned that Blizzard only employs 2500 worldwide in Customer Service. That includes things like phone bank operators, GMs, forum mods, etc...
While those 2500 people might seem like a lot, and in many ways it is, the recent customer service level of WoW shows that these resources might not be enough.
On that note as well, I'm quite surprised that Blizzard actually released these numbers. I loath what the forums will look like in a day when people start quipping to Nethaera or Ghostcrawler about the 180,000 bugs. But it was Blizzard's decision to put that number out there. Here's hoping it doesn't skyrocket to 200,000 when Ony starts to spawn more Whelps in patch 3.2.2.
Filed under: Analysis / Opinion, Bugs, Blizzard, News items
Reader Comments (Page 4 of 5)
maniraptor Sep 17th 2009 2:11PM
Other big numbers from the keynote this morning:
* 1.5 million art assets
* 27 total hours of audio
* over 37,000 NPCs in the game
* for each "patch" you download for which ever client you're running, 126 different patches have to be developed and tested (in part because they have 10 localizations to support, on both OSX and PC)
Murdock Sep 17th 2009 2:14PM
Need more Oracles, they eat bugs all the time! If you think about it, that # is about right. Think about all the little cockroaches, flys, etc you see when you are out running around. Then, when you start to think about AQ20 and AQ40, along with OK and AN this seems pretty reasonable.
I wonder how many humans they are tracking? Gnomes?
...oh, wait errors in code, NVM.
Aaron Sep 17th 2009 3:19PM
HAHAHA. Now THAT is funny! I needed a laugh. /thank
Hangk Sep 17th 2009 2:14PM
What everbody else said: for a large software project that has been around for as long as WoW has, this number is, if anything, remarkably small.
The vast majority of these 180,000 bugs fit into one or more of the following categories:
- Fixed bugs. Just because a bug gets fixed doesn't mean it gets deleted from the tracker.
- Bugs filed against internal versions of the game that were never released, even to PTR.
- Bugs filed against the WoW server software, or other internal components.
- Duplicate bugs, where a single problem has two (or more) symptoms and both get filed as bugs.
I have no idea how many open, extant, player-affecting bugs they are tracking, but I can assure you that it's a tiny fraction of the total number of bugs in their tracker.
Ed Sep 17th 2009 2:15PM
Fun fact: 179,642 of those bugs come from the quest "Deep Ocean, Vast Sea".
Esau Sep 17th 2009 2:29PM
"To me this number seems very large for a video game."
If this number in any way reflected poorly upon Blizzard in reality, do you think the slide you pictured would have made it into an industry presentation made by Blizzard?
You guys post a lot of stuff here, most of its fun but clearly you don’t step back to read some of it.
Kia Sep 17th 2009 2:30PM
This might seem large to the uniformed, but I can tell you this isn't out of the ordinary at all. As someone that's compiled and run their own Ragnarok Online server, even an MMO as "simple" as that can have bugs popping up in the tiniest places, even if they don't effect much. I can imagine WoW would have, well, obviously, thousands.
Beli Sep 17th 2009 2:31PM
I do software development in the medical device industry, and I have to say... that number doesn't really surprise me. We ran some static analysis tools and automatically generated defects against some of our code base... for a somewhat simple web interface, there were over 10k defects generated we had to look into. Of course, most of them ended up being items that weren't an issue, but it just goes to show how the defect count can't really be used as an indicator of the software's health.
Nathifa Sep 17th 2009 3:47PM
Probably been said already in the comments thread, but for a computer game this large that has been running for almost 5 years now, this doesn't seem like a lot. Comparing it to an OS like Windows 7 is stupid. W7 hasn't been around for 1/4th the time WoW has been and if its anywhere close to 180k that wouldn't be surprising.
Stuart Thompson Sep 17th 2009 2:39PM
I'm not sure I could comment on how many bugs I would expect from a video game of this size. In many ways it is partly an operating system as it hosts other programs in the form of lua scripts that are loaded, managed, pre-empted, evented to, and hosted much as Windows hosts executables.
I wouldn't be surprised to find that many of these bugs are within tools that most of us never see. The fact that "the 3D model viewer experiences an import error when merging meshes that have invalid terminators", for example, is probably an active bug that is being worked on by some developer but that in reality has no effect on the people playing the game.
I can't say if I think 180k is high or not. I have no control sample to measure against. I've worked on some pretty large codebases for Fortune 500 companies and can attest that bug rates in the thousands are not uncommon on non-operating-system projects, but WoW is such a new beast in terms of the software that it's hard to say.
vaarsej Sep 17th 2009 2:41PM
there's still that bug that makes a boar spawn on top of a cactus in the Orc/Troll starting zone.
Aaron Sep 17th 2009 2:40PM
Waiter, I found a bug in my soup. =/
Aaron Sep 17th 2009 2:41PM
Wanna bet the 25man raiding disconnect bug isn't even in there? :p
Seraphna Sep 17th 2009 4:40PM
Your connection dropping when you raid is your connection, not the game, nor a bug. I raid nearly every 25 man each week, I never have drop issues.
Bossy Sep 17th 2009 2:42PM
Look at it this way.
In Wow they track the bugs and kill them.
In most other MMORPG's I've played they don't even bother.
dj_stevie_c Sep 17th 2009 2:46PM
I work on a rather well known Football game (no, not for EA or Konami) and our bug database extends to more than 40,000 entries, however that being said only 3500 are "Active" at this point.
I think if I was doing what I do for blizzard my head would explode.
However. These bugs would probably break down to:
10-20 'crashes'
100 'fatal' errors (getting completely stuck, crashinge etc)
20,000 gameplay issues
And the rest minor/cosmetic things.
Still a VAST ammount of bugs to keep track of. I salute their QA Managers.
As for "How on earth does that many bugs get in a game"
It's like a domino effect, a bit of armor you put in could send a cascade of bugs that affects everything from that armor to bank slots to everything, the ammount of checking that goes into a game this size is mind boggling, that it appears as bug lite as it does is an incredible achievement.
You never know how testing is until you get involved with it but believe me, it's not "playing games for a living" :D
Blizzard are doing an amazing job.
dj_stevie_c Sep 17th 2009 2:49PM
Not to self, don't type while at work. 2nd "Crashinge" should've said "Lockups'"
biohazard Sep 17th 2009 3:20PM
180,000 bugs.....
....and they are all in Silithus. Where are the Roughnecks when you need em?
danawhitaker Sep 17th 2009 3:21PM
No no no. You're forgetting the bugs in HAN and HOK.
biohazard Sep 17th 2009 3:24PM
WTB [Epic Can of Raid] 1c....
"DO YOU WANNA LIVE FOREVER?!?!
(love that movie)