From Zul'Gurub and Ahn'Qiraj onward, we have made extensive use of our Public Test Realms in order to help find bugs and improve the tuning of our raid content. We have an extensive team of Quality Assurance testers and an internal raid group, but it's impossible to replicate the sheer breadth of information we get from having hundreds of real raid groups attempting diverse strategies as they experience encounters for the first time. We learn a tremendous amount from that testing, both in the form of written feedback on these forums, which is carefully read by every encounter designer, and from spectating raids and observing how they fare.
Now, that's what we get out of the PTR. Speaking pragmatically, we understand that not everyone's motivation in logging on to the PTR is to give us feedback. They want to check out the new content firsthand, see what's coming to the live servers in a few weeks or months, and, in the case of raiding, get some competitive edge that will help them clear the content faster on the live servers when the patch is released. That's fine. We thoroughly appreciate everyone who takes the time to give measured feedback, or goes out of their way to methodically test new features and report bugs they find. But at the end of the day, we'd rather have a packed PTR with a small percentage giving direct feedback, than a nearly-empty PTR where everyone gives it. Bugs that require very specific conditions are much more likely to occur with thousands of testers than with hundreds, and it's clearly beneficial to everyone for us to find and fix those issues before they affect millions in a live environment.
When it comes to raid testing in particular, there's a symbiotic relationship: The bleeding-edge guild that intentionally wipes to a boss at 5% so they can set up their boss addons, try out unusual strategies, and practice execution is there for competitive reasons. But we gain information even just from silently observing that will help us make our encounters better, and fundamentally, that is the purpose of raid testing on PTR. Our raid testing times are scheduled to coincide with our core work hours, so that designers can observe as much of the testing as possible, with the exception of LFR and Flexible tests which require more open-ended scheduling given their nature and target audience (and even then, if you were doing a PTR Flexible raid this past weekend, odds are decent that I or another developer was spying on your group – creepy, I know). When it comes to Heroic testing, we try to keep the testing windows as brief as possible, in order to minimize the impact of PTR testing on the progression "race."
But that symbiotic relationship requires that we be around in some capacity to observe. Players who find a way to access bosses (especially Heroic bosses) that aren't currently being tested should let us know, so we can correct the problem. We discovered that over the weekend a progression raid group accessed the Siege of Orgrimmar raid and spent hours on a Heroic boss in the middle of the night, after copying fresh characters with generic names, presumably with the hope of going unnoticed. Those actions go against the intent of the PTR, the spirit of fair play, and were clearly intended to go against our wishes for specific boss testing periods. PTR testing is a privilege, and we reserve the right to revoke PTR access from those who have shown an intent to abuse it.
If you come across an oversight in our testing protocols, or a serious, abusable bug or exploit in the course of your testing of a boss encounter on PTR, and you want to let us know without alerting everyone, please note that we can read edited or deleted forum posts. (On the live servers, please use the in-game bug reporting interface.) Otherwise, carry on with the fantastic feedback that we've been receiving about all facets of 5.4 content thus far, and we look forward to seeing you on the Public Test Realm!