Officers' Quarters: Grading your raiders
by Scott Andrews 
Jul 16th 2007 at 11:00AM
Every Monday Scott Andrews contributes Officers' Quarters, a column about the ins and outs of guild leadership.
When your raids are going smoothly, there's almost a superstition among players not to look too deeply into exactly what everyone is doing. It just works: Players are clicking, the loot is flowing, confidence is high, and no one questions it. But when your raid hits a snag, you can either start blindly pointing fingers or figure out what you're lacking. Well-run guilds take the latter course. This week's reader wants to know exactly how to do that:
It would be great to see a post on your Officer's Quarters blog about how to measure/observe the raid's performance. For example, we are stalling on the Curator, and it would be great to hear some different techniques on how to measure who's getting all the heals, where is mana going, why exactly are people dying, etc., in how to assess performance and adjust.
Thanks! Great reading your stuff, keep it up.
-- Ciacco -- Malygos, 70 human rogue
Thanks for writing, Ciacco! I think there are more than a few guilds who stalled out on Curator for a week or two when they first started clearing the zone. Mine did. More than anything, the fight is a DPS check and it's crucial that your "pewpew'ers" bring the pain. It's easy to look at your members' gear using the Armory and see who's behind the curve, but there are also lots of intangible skill factors that gear won't necessarily tell you about.
Problems like breaking CC, pulling aggro, or going OOM are easy to diagnose. Everybody sees the priest with the empty mana bar or the mage that gets flattened by Gruul. But to figure out who isn't living up to their statistical potential is another matter. For that, you need the raw data.
For a long time, the popular choice for obtaining such data was the DamageMeters add-on. But more and more people are beginning to realize that this add-on can do more harm than good. After trying it out once, I've never run the add-on again and I hate it when people use it as a contest. In the old days, I can't tell you how many times people would pull aggro on a Molten Destroyer -- or (much worse) Onyxia -- because they wanted to be #1 on the meters. You have to measure success and failure by the raid's success or failure, not individual achievements. It doesn't matter how many touchdowns you score if the other team kills you and all your players. (That's probably mixing metaphors, but I'm sure you get the point.)
My guild has recently started using a Web site called WoW Web Stats. One of my officers discovered this awesome tool and now we are hooked on it. To continue the metaphor, it's like a fantasy-sports stat tracker for your raid. Basically it records and uploads your combat log and generates an incredible array of statistics. For instance, it will tell you exactly how much damage a warlock did to a specific mob with each spell, including what percent were crits and how much damage was mitigated. Or you can tell whose Steady Shot hits harder on average and by how much. It will tell you how many times someone died, who killed them, and when. And who healed whom and how effectively. You even get the same data for all the party's pets and all the mobs you fight. Like DamageMeters, it helps to have multiple people gathering the data so you can recognize inaccuracies.
As far as I know, nothing else can give you such a sweeping tabulation of everything that's going on during your raids. The question is, Once you have all this information, what do you do with it? Remember that the point isn't to talk smack about who gets the biggest Eviscerate crit -- it's to figure out who isn't pulling their weight so your guild can help them improve. Blame only gets you so far. At some point you have to buckle down and work hard to fix your raid's weaknesses.
/salute
Send Scott your guild-related questions, conundrums, ideas, and suggestions at scott.andrews@weblogsinc.com. You may find your question the subject of next week's Officers' Quarters!
Filed under: Officers' Quarters (Guild Leadership)
Reader Comments (Page 1 of 1)
DavidC Jul 26th 2007 4:23PM
Oh Please: "After trying it out once, I've never run the add-on again".
Ignorance is NOT how you run a guild. I suppose you don't use KLH or DeadlyBoss mods either?
Damage meters are a *great* way to get a very good idea of who is performing and who is UNDER performing. In our guild "healing" meters are always posted first, to recognize how important healing is to the guild.
Good meters, like SWSTATS have a lot of great features ... like being able to isolate one player and see exactly what they were doing. Great tool for class leaders to help junior members improve their game, or visa versa, for a junior member to see what the other person is doing that is so much more effective (damage or healing).
Advocating the non-use of damage meters in this column is quite frankly ridiculous. Obviously anybody can blog ... not everyone SHOULD blog.
Ana Jul 16th 2007 11:24AM
Just a quick additional note about WWS. We had an unusually tough time on Aran one evening, and when we looked over the WWS logs it appeared that our mages weren't counterspelling. However, that's because WWS doesn't know how to "read" the "Player Interrupts Mob's Spell" message and filters it out of the logs.
A very, very small thing, but one more grain of salt to take into account. I think the author is working on a fix.
Freehugz Jul 16th 2007 11:50AM
WWS isn't the greatest way to find your raid weaknesses.
If you want to find out what your raid problems are, talk to your more experienced players and have them keep on eye on others doing their same role.
A healer watching his other healers will be able to pinpoint problems with healing faster than a tank with pages of data. Once you've found the problem, talk to the class leader and have them sort out the problem.
A note to raid/guild leaders: Don't try to fix problems regarding classes you haven't raided as!
Jalann Jul 16th 2007 12:17PM
I use SWStats. The best thing to do is to use a combination of Meters and what #2 said. Also if you plan on being a "Raid Monitor", then pick a Hunter, Mage, or Warlock to play as. It is easier to see standing in the back and you should be able to contribute to the fight while watching the others.
Combatc87 Jul 16th 2007 12:43PM
SWStats or Recap both work well. SWStats has the potential to cause lag, but if others are running it, it will sync up and average the data to get the most accurate results.
I have personally spent more time with Recap, and it's a great tool. It's extremely customizable based on what stats you want to see. It also saves a page for the previous fight, which you can isolate and review apart from the total page. It can record everything that WWS can.
Siedre Jul 16th 2007 1:12PM
The two in-game statistical analysis tools are Recap and Recount. You can get both from the Ace site. Recap has been around for ages and Hawksy does a nice job keeping it stable. Recount is only in beta, so it has potential for inconsistencies, but it has a nice graphical interface that Recap doesn't have. These do much of the same analysis that the WWS website does, and they're viewable in game. I'd recommend checking them out to see how you like them compared to WWS.
Lisa Poisso Jul 16th 2007 1:23PM
I agree with Freehugz/#2. Having experienced members watch what others of their class/role are doing is hands down the best way to pinpoint problems. Mix it up! Shuffle some experienced "watchers" into your struggling raid for a week in backseat roles, so they can concentrate on watching for problems and supporting situations that go south. Not only will you gain quick insights into where your problems lie, but your less experienced players will get the benefit of seeing how the more experienced players pull things off when the going gets tough. An added benefit: both teams will come away with some fresh ideas on how to approach different encounters!
jess Jul 16th 2007 3:05PM
@5 if you read his entire post he says he doesn't use dmg meter but he does use wws stats. I would second the use of those becuase they break down just about eveyrthing a player can do when fighting and is a great tool for giving people feedback on what they're doing wrong or right.
I think that annother important aspect in making sure people are pulling their own wieght is also to see who is bringing thier own food/pots/etc to raids, who is showing up consistently, who can take directions and who does research about bosses before fights. Those are all things that are important too.
And specifically about the Curator fight, good luck and remember it's an endurance fight so mana regen gear is great to have and make sure the dpsers are focusing on the flares.
john Jul 16th 2007 3:12PM
what i thought you picked players you didn't like and booted them.
Tam Jul 16th 2007 3:19PM
"9. what i thought you picked players you didn't like and booted them."
You know... I hate seeing these responses. Its more like "You review the data and inform the players that aren't pulling their own weight. If they don't step up, then you boot them on progression attempts until they step it up on other farm attempts."
Its working together, not working against.
Scott Andrews Jul 16th 2007 4:33PM
Thanks for the insights, guys!
I definitely agree that class leaders are a huge asset in assessing raid members of each class. However, Ciacco requested how to find specific performance measurements, so I based my answer around that. Also, it can be tough -- even for a CL -- to assess someone who appears to be doing everything perfectly but is lacking in the raw numbers without using some kind of measurement tool.
Maybe I'll do a column about CL's soon. In the meantime, thanks for reading -- and keep the feedback coming!
Ciacco Jul 16th 2007 6:08PM
Hey, Ciacco here, thanks so much for responding. As you guessed, it was just a 1 week stall on Curator - guild dropped him on Friday with 'relative' ease once everyone focused and tore in, along with a tweak on positioning and healing assignments... (I wasn't in the raid, lol, but was proud of the guild)!
WWS looks great, I'll definitely check it out if everyone else agrees - thanks and keep the posts coming!
Kryptonls Jul 17th 2007 5:52AM
Curator stalled my casual guild for at least two, if not three consecutive nights. We got there in the end - it's just a matter of being patient and trying out new things, reshuffling the raid group and bringing in a limit on people still wearing greens.
Grodzan Jul 17th 2007 8:10AM
Personally, I believe people who wear greens are, in general, an unnecessary liability in a raid. There are plenty of quality blues and crafted epics available before you start raiding.
However, I would take a very good player with a few greens over an inattentive one that is decked out in epics any day. Play ability trumps equipment, but both is really necessary to make raiding/heroics easy.
I personally don't have anything against damage meters. If used correctly, they can help you spot weak links, figure out what they are doing wrong, and allow you to address it. Even as a competitive tool it is useful. People just have to realize that you get no credit for being #1 on the damage meters if you pull agro.
Take this all with a grain of salt, as I am not a hardcore raider. I think some of the best ways to help though is to encourage everyone to do their research of both the fights and their class. Spend time talking with your class leader about strategy/gear. Paying attention during the raid is useful too. If you see someone pew-pewing Curator while there are adds out, that is something that is easy to correct.
Ana Jul 17th 2007 8:43AM
On the CL/meters issue, ideally guilds would have both. As Scott said, there are things even an experienced CL can miss in the thick of battle. Additionally, having the numbers is a lot more persuasive than just saying, "You didn't look like you were decursing as often as Healer X."
One of the side benefits of WWS is that it's resulted in a lot *less* work for our leaders. Our members seem to have been hungry for this kind of exhaustive breakdown of how (I use the term quantitatively, not as "good or bad") they're playing. Not everyone has a computer that can run one of the in-game assessment tools - this way they don't have to install anything, and they can look at it at their leisure rather than in the middle of a trash pull. They seem to love it, and it's been generating a lot of great discussion on our forums.
That's just been our experience though. You have to fit the tools to the culture of the guild. It probably works so well for us because we're solidly in the working stiffs age bracket. I'd much rather read over stats on my lunch break than try to do so in the limited hours I have to play each evening.
Valyre Jul 17th 2007 9:08AM
What no one seems to have mentioned is how cumbersome and time-consuming WWS stats is unless you purchase their membership. I seem to sit in queue for at least a minute for every new analysis of the data. Multiply that by 7 healers for 5 bosses and you're looking at significant quality time with the loading screen.
SWStats is a very powerful tool when you need answers right away. It's very easy to compare the output of 2 or more members, including their ability selection, average output by ability, maximum output by ability, total output by ability, who they're healing, who's healing them, how often they are demagic/curse/diseaseing, how many times they have died, etc. The list goes on and on.
You can take that data and see exactly what discrepancies exist between a player that is doing very well and another of the same class that isn't. You don't even have to know anything about the class. For example, if you have two priests healing the MT and one keeps going OOM, you can quickly find the cause by comparing them. Just look for the differences between the two. SWStats can show if they're spamming inefficient spells, if they're healing the MT plus 10 people around them, if they're average heals are less and they have to heal more to keep up, etc.
The overall damage and healing meters are the most minor functions it can do, imo. If you're only using SWStats for that, you're missing out.
Kat Jul 18th 2007 1:38AM
Often, I think damage/healing meters do more harm than good, it often makes the person shoot for being #1 on that meter and not doing their job. When I raid I am going to do my job for that fight/raid and not stress where I stand on a meter. I've seen mages not be quick to resheep things or decurse, cause they are more worried about their dps output. I've seen healers rush to place hots and heals on all raid members after a wipe to up their ranking on the healing meter.
The 10 man kara really made it competitive for larger guilds, in that if you aren't perceived as doing well, like being on the top of a meter, they will just give your spot to someone else.
Phil Jul 18th 2007 5:41PM
WWS logs are as useful as your raider's understanding of statistics. Thus they can be incredibly useful, or total lies. There are mobs that throw everyone's numbers off (warriors will always look better than other melee dps in AoE pulls because of whirlwind, for example), so it is most useful as a way to track individual performance over time and note things like how many crushing blows your warriors were taking, or what happened right before Gruul ate the Hateful Strike tank alive. The trick I use is to come up with the question first (how much damage am I taking during Curator and how can I reduce that amount? Why does the tank keep dying?) and then go look for data to answer my questions.
Auzara Aug 6th 2007 9:24AM
Meters:
Cut and pasted from our Combat Log forums
"We post combat logs because we belong to the school of thought that the more information you have, the better of a player you can become. The sole purpose of posting these logs is so that you can spend time evaluating what you did in a raid and think about how you can improve. We don't care if you did 50% of the raid damage by yourself, there is always something you can do to improve your play."
Meter can be a powerful tool, but tools are meaningless in the hands of someone who doesn't know how to use them.
When you evaluate your players consider their role in a raid. Did they have a spell to interrupt, a cube to click, were they in a group stacked to provide synergy bonuses? Any of these things can affect a person's overall output in a raid.
Measuring a person's overall output isn't always valuable information. Checking the events that happen right before a tank dies on the WWS report can provide you with valuable insight about a healer's sense of timing and spell selection. Their position on the overall healing meter provides much less valuable information. I've known many healers that can spam their way to the top of a healing meter and yet remain completely unable to keep a tank alive when push comes to shove.
Class Leader Feedback:
I think the idea of class leaders has to be change to accommodate 10/25 man raid content. Now that raid sizes have become smaller and "off specs" have becomes so accepted that some classes have trouble defining a main spec, I think it's difficult to conceptualize your raiders by class. Our guild has identified raid roles: Tanks, Magic DPS, Physical DPS and Healers.