Interview: WowLens and a National Science Foundation grant help researchers mine player data

Researchers at the University of Minnesota, Carnegie Mellon University and the University of Pittsburgh, under a project by the National Science Foundation, are using an addon called WowLens to gather peer review data. The project aims to use the wealth of people, resources and data floating around Azeroth in compiling data for research projects. It personally reminds me of Folding@home, but with statistics instead of computer processor cycles for medical calculations.
I got to have some questions answered about the project by Professor John Riedl of the University of Minnesota Computer Science department and Daniel Moy, one of his colleagues and students. If you would like to get involved with the project, simply head over to the WowLens site and download the client. You run the program before opening World of Warcraft, answer a few survey questions, and at the completion of a group, you answer a set of questions pertaining to your experience with the players in that group. Let's let the researchers themselves tell you about their project.

Professor John Riedl: My group is GroupLens Research at the University of Minnesota. You can read our Wikipedia page here. We wrote much of the page, but it's the most complete telling of the story.
In the past seven or so years, we've been working with our colleagues from the social sciences at the University of Michigan, Carnegie Mellon University and the University of Pittsburgh, to apply insights from how groups of people work together to improve online collaborations. For the past three years, we've been using World of Warcraft as one of our study platforms. We're particularly interested in developing new intelligent user interface technologies that can make teams in WoW work together more effectively.
The current plug-in is one of those pieces of research. We're exploring how what is known about group formation from the social psychology literature can be applied to help make WoW groups happier and more effective at their goals.
Daniel Moy: The overarching research we're working on is a National Science Foundation grant between three universities (Carnegie Mellon University, University of Minnesota, University of Pittsburgh), looking at online groups. This covers groups in a variety of different settings, including both Wikipedia and World of Warcraft. Specifically for World of Warcraft, we're looking at figuring out how to understand and improve group satisfaction, cohesion and coordination. Finally, WowLens itself is looking at whether or not we can use an in-game rating system (of peer ratings, not arena ratings!) to help players form better groups and in general have more successful, happy game experiences.
AS: What interested you guys about Wow? Was it the size of the player base? Are members on the team players?
Moy: Yes, the large player base size is a definite attraction. If millions of people use a system, chances are there could be interesting research questions to answer. There's also quite a bit of existing academic research into various aspects of World of Warcraft. As for the second part of the question, most of the group has played WoW at one point in time, including one of the professors. Only two of our current group members are ACTIVE players, though a couple more keep up with the important news, trends, etc.

AS: What were the contributing factors to choosing WoW as the testing ground?
Professor Riedl: We're particularly intrigued by WoW because of the team aspects of the game. We like the way small "work groups" (parties) and larger "organizations" (guilds) interact over time. One of the challenges for WoW is how these small groups have to form over a short period of time, and then fade away, replaced by other small groups. These sorts of dynamic reformings are unusual in physical world groups, but seem much more common in online communities. What tools will best support this new community behavior? How can it be made more fun and effective?
Several of us are players. Daniel has a long history with the game. Tony (the lead grad student here at Minnesota), Bo (the lead grad student at Carnegie Mellon University) and I have also played, though not to the expert level. Kristina (the lead grad student from Pitt) and Vanessa (a M.S. student from Minnesota) have played to end game. On a personal note, my son Kevin has played to the expert level, and I've enjoyed talking about the game with him.
We are very impressed with the game as an enormous-scale online community. The level of interaction among players is fun and interesting, and the scope of the world is breath-taking.
Moy: MMOs are useful in their own right for a many reasons. Some of these include complex or obscure task solving, spontaneous group forming and the insane amount of time users tend to play. World of Warcraft specifically does have a alluring amount of research data, with orders of magnitude more players than other MMOs.
AS: Who developed the addon for you guys?
Moy: I did, in its entirety. I played as Pharaunmizz on Greymane and had a bit of an addon affection complex already going when we started. When we were looking for ways to get research done, we tossed around the idea of using an addon. I had to learn Lua and the WoW API to actually implement it all, which was a long and painful process. Big, big thanks to wowwiki, wowprogramming and #wowuidev.
Professor Riedl: We're impressed with the Lua environment for extending the WoW interface and would love to build further add-ons to make players' WoW interactions more positive. We'd love to hear any ideas you have for addons that guilds would appreciate!
AS: What can readers do to help contribute to the work you do?
Professor Riedl: The most important is to use the addon! We're very excited to get user feedback and learn how to make the addon more valuable.
We'd particularly love the opportunity to work closely with some of the large guilds to build tools with and for them that would help their interactions. Please encourage your readers to contact us at wowlens@cs.umn.edu. The great thing is that we have research support to build cool tools for WoW players. Let us know what the community would value, and we'll figure out if there are interesting research questions to explore!
Moy: The first is to use our addon, WowLens, and religiously submit data. We've built a clean little addon that runs in-game like any other addon, and a data uploader that will upload data to our servers, download an updated data set for you, sync everything up and automatically start World of Warcraft. This uploader is meant to be run instead of the usual WoW shortcut to start the game; it should handle starting the game itself cleanly for the user.
AS: What other uses do you think addons can haev in research besides data collection? Any other impacts on the research world?
Moy: A good question! Passive data collection, while useful, isn't as fun and can't answer a lot of the interesting research questions. The potential for using addons in active research experiments is very, very appealing. Also, simply having a user base with an addon linked to a research community, perhaps on an outside website, could be invaluable, especially if it gains any popularity with the player base. Again, if anyone has any thoughts about future directions, we're always willing to listen and explore new ideas.
Professor Riedl: Over the long-term we'd like to create tools that make peoples' social lives richer and more fulfilling. We are enthusiastic about WoW as an online community with a powerful social component, and would love to explore how research can make it even more satisfying.
Once again, thank you to Professor Riedl and Daniel Moy for taking the time to answer my questions. If you'd like to help out with the Wowlens project, check out their website and provide some feedback!
Filed under: Add-Ons, Interviews
Patch 5.2 interview with Dave Kosak
Inside an old alt's vault
The latest patch 5.2 news
All of the latest Mists of Pandaria news





Reader Comments (Page 2 of 2)
Naysayer101 May 11th 2010 2:13PM
Panopticon?
The fact that you know your being "watched" will influence behavior and give skewed results.
Thomas Baldwin May 11th 2010 12:09PM
I just want to say that I really REALLY hope they do come up with some kind of peer rating system for PUG's. Difficult to do it right, for sure, but would be SO beneficial if someone gets it right. Think of rating individuals (once per grouping with that character) and then being paired with people that are similar to you in rating. Might make the following interaction less common: random character in PUG ninjas loot. Party chat says "JHAHAHA N00beRz! I got my PHAT lewt! I'm outtie! PEACE!" And if not...well, at least I won't be grouped with them.
I'll be downloading this addon and rooting for their cause!
Icyslush May 11th 2010 12:04PM
No question, the installer is a bad idea. Anything that runs an exe to install, let alone requires you to use a different shortcut to start WoW is a bad idea and a bad precedent. I'm not saying there's anything harmful in this addon, but given the prevalence of trojans, keyloggers and other malware, NO addon should be configured to install this way. I won't be using it until they change this.
Portals May 11th 2010 12:13PM
Maybe they posted the article here to see what the group reaction to the project itself would be by reading the comments? ...sneaky scientists
Adam Holisky May 11th 2010 12:26PM
Yes, that's it exactly. Government money being used to make WoW a better game...
or they're using WoW as a means to study online groups, which you know, will be important when doing anything online with more than one person.
/headdesk.
Grovinofdarkhour May 11th 2010 12:47PM
Dey took our jerbs!
James May 11th 2010 1:12PM
Where the hell did all of these start coming from? I see them on about every article. Didnt start untill about recently too.
And dont get me started on the grammar... makes me think that the gold farming companies also invest in hot topic or something -_-
Cataca May 11th 2010 1:14PM
"government/tax payer monies is being used to figure out how to make wow a better game"
It's not being used to make wow a better game. It's being use to look at how social groups interact.
Learning to interact with a social group better impacts...pretty much everything and everyone.
Take my hospital for example (Also a government installation). Every employee has a computer that they use for almost every action. If a computer isn't involved then the action is at least documented in a computer.
A major problem is communication between different departments of the hospital. We could take data that is gathered from this addon and apply it to better structuring communication and interfaces within the department so that we may better interact with others.
You can apply this to just about everything that requires interaction between you and another person. Very valuable information in every venue!
nieboh May 11th 2010 1:39PM
I don't know what you mean by "recently". There were a whole slew of adds like these being spammed last year between Thanksgiving and Christmas. This one is particularly intriguing to me though. The English is so bad, I litterally have no idea what they're getting at.
"Do not look at the money." Seriously what???
"Just a few seconds you have time." LMAO what???
"Cool enough beautiful?" I think I'm going to try using this as a conversation starter/pick-up line.
nieboh May 11th 2010 1:44PM
OK, a little irony there in somplaining about their English and making what appears to be a spelling mistake...that was a typo. I have trouble with double key strokes sometimes while on this laptop. Literally. See? I do know how to spell it.
Still hoping for an edit feature some day.
nieboh May 11th 2010 1:51PM
"somplaining"? seriously now, wtf? I think this keyboard is doing it intentionally to mess with me. Clearly I was "complaining" and not "somplaining". I don't even know what somplaining is.
I seem to have been sidetracked.
Anyhow, about the article, it's a cool idea and sounds like an interesting research project/gateway for future research projects, but I'm in agreement with a lot of the others. The whole idea of installing a new game launcher just sounds bad to me and will prevent me from participating in their study. I'm sure there isn't anything malicious in their software, but it's a bad practice and not something that should be encouraged by this site.
Amaroese May 11th 2010 2:46PM
I didn't read the post but I noticed "haev" in bold text.
GG Spellcheck!
Daniel Moy May 11th 2010 4:39PM
Hey all, developer here,
First off, thanks for all the input. I'd like to address and clear up a couple of the points.
We indeed do have a stand-alone addon, and it does in fact do in-game data syncing. The executable uploader is not required to update, so feel free to just head over to Curse (http://wow.curse.com/downloads/wow-addons/details/wowlens.aspx) or WowInterface (http://www.wowinterface.com/downloads/info16252-WowLens-PlayerRating.html) and just use the addon.
Here's the caveat: in order for the research team to get useful data, you have to use the uploader, or sync in-game with someone and hope that down the chain someone is using the uploader. World of Warcraft unfortunately does not allow communication with the outside game.
If you're really paranoid about the binary, you can compile it yourself with the open source code available via http://wowlens.cs.umn.edu/download/extra.
I agree, the website needs some work :(
Cataca: That would be awesome to look into, but it's unfortunately a catch-22. The only way to get data out of the game is via an uploader (or intrusive survey process), which... yea. Given that you can get called for keylogging by posting a link to the official Wow forums in-game, I think tin-hat wearers are definitely there in large numbers (or are just disproportionately loud).
Thanks for all the input guys :)
Jabberwoky May 11th 2010 5:02PM
Hey Daniel. I would be very interested in using this addon but my network has a very strick policy against loading third party executables for gaming. Have you taken a look at how warcraftrealms.com uploads data? I use their addon almost daily without a problem.
Daniel Moy May 11th 2010 8:09PM
Hi Jabber,
That would definitely be an option. We would probably want to implement a quick way of getting downloads for the user as well (the end-user benefit from the uploader actually comes from the automatically downloaded dataset). Also, the average Wow user doesn't know where their saved variables are stored, but we could look into getting something like this up for people who want to just support the research :)
I'll pass on the word.
Jamie May 12th 2010 10:39AM
I was really excited, but after using the addon, I now believe it is unlikely to succeed. You have fill out a lengthy survey after every time you leave party. Its kind of nuisance. And for PUG instances, you probably won't even remember the name of every body who was in your group. How can you rate them if you don't remember their toon
Bloodvein May 16th 2010 3:02PM
My only concern with this is that it seems to be shaky on the TOS line. Me personally will not partake in this interesting study unless sanctioned by Blizzard on their website. Simply because there has been add-ons in the past that transmitted and received information such as this; which, quickly became a bannable offense. Blizzard has stated that they do not want any kind of add-on that transmits and receives information from outside sources via the internet and so on.
In short, my worries is not that you are trying to steal my keys but that Blizzard in their infinite wisdom may frown on this add-on. I do however wish you the best of luck in your endeavors.