This was fascinating to listen to, so I thought I'd share the insights with WoW Insider readers. This is not verbatim quoting throughout, as I simply can't write that fast, but it covers the raid-making process as they described it.
How do you make a raid?
You start with the lore. Ion and John talked us through the process of making the Firelands raid, so they began from a set of conditions. As Ion put it, "We consider what the idea is. What is this raid going to be about?" So they sit down with the encounter design team as well as the game producers in a meeting room and talk it out.
Ion mentioned the team's major considerations in the creation of Firelands: the preceding story of Deathwing's return, as well as the corruption of the elemental aspects -- Al'Akir having been one of those in Throne of the Four Winds -- and the Hyjal quest line. The designers want the lore to make sense, and as far as possible to be interwoven into the raid. They mentioned the cutscene-heavy nature of Dragon Soul as one possible way to do that, adding that Dragon Soul was one raid which had to be very lore-heavy, a lot of loose ends had to be tied up. But they were trying to avoid that for Firelands.
So based on the lore, the decision was made to have the final boss of the Firelands be Ragnaros. At this point, all they knew for sure is that, so once the overall objective of having Ragnaros was set, they worked backwards from there.
How big should it be? At the early stages of creating Firelands, there had been a plan for a parallel raid in Vash'jir, but that had to be dropped due to resources, so more was put into making Firelands a bigger raid because they knew it'd be the only one that tier.
They also assessed what the overall creative direction should be. With Ragnaros being the end boss, a fiery feel certainly made sense, but how would that be best expressed? Would nods back toward Ragnaros' previous home be good? Or should they steer away from that? And what major characters from lore would it make sense to include, if any? Fandral Staghelm, with his corruption and madness and position as the first Druid of the Flame, made perfect sense, so the decision was made that he should be included.
"What do we think would be cool ideas?" This was the next question that was asked, in a second meeting. When Ion said this, I was hit by a wave of jealousy, considering how great it would be to work in creative design at Blizzard. Anyhow, my green-eyed monsters aside, Ion told us that someone at the meeting suggested a giant spider boss, which was generally agreed to be a cool idea, and another person had they thought that it would be amazing to have a volcano within the raid that a giant fire bird came out of.
And working back from Ragnaros, there was a plan to have him in a keep, given that he was the lord of the elemental plane of fire, so a bridge-keeper made sense. That became Baleroc. There was also the idea to incorporate a giant fire turtle boss, as Ion explained there had been one in Hyjal's quest line and the idea cropped up elsewhere in the lead-up quests. But the turtle boss was not considered good enough to make it in and was cut in favor of other ideas. He was commemorated, though, with the family of fire turtles you could seek out. The list of bosses was set. So what next?
Encounter mechanics came next. In the example of Beth'tilac, there was the idea of setting up a fight on two separate levels, forcing the raid to split into two groups, one up on the upper level and one lower down. Then they fleshed out this idea, so the detail that the top layer is where the boss fight takes place is added, as well as the look and feel of the web, how you get up and down, what the people downstairs do with the adds, and so on. At this point, no numbers were set -- that's not what the encounter design team does -- nor were the abilities set. This is very much a design overview.
Then the idea was sent over to gameplay programmers to assess the technological needs of the encounter designers' ideas. Did they have the technology to implement the ideas? Would they need to design new technology? Were there elements of the encounters that would cause too much lag, lower frame rates too much, or put too much of a strain on servers? If new technology would be required, did Blizzard have the resources and capability to design it? Was it even possible within the system?
John gave a couple of illustrative examples. The webs you had to climb to get up into Beth'tilac's web were actually vehicles; you were entering a vehicle to get up there, then automatically ejected in a preset way, but they wanted you to have access to your abilities as you went up into the web and not to feel like you were in a vehicle. All that was new technology. And during the Spine of Deathwing fight, Deathwing was actually a building, but a building with moving parts that moved in sync -- again, new technology.
Designing the zone came next. The design of a zone forces players to approach content in a certain way. Corridors and doors are one approach, gating content until certain bosses are defeated, while Firelands took a more free approach. But they didn't want players to use this free approach to skip huge chunks of the raid, so Shannox was brought in.
Shannox was a new challenge for Blizzard's team. Since he could be pulled anywhere in his patrol zone, his mechanics had to work anywhere. There were no room constraints such as those for Valiona and Theralion, where Valiona's Deep Breath was clearly tied to the room's design. So Shannox was given random directional mechanics such as traps and his spear mechanic, as well as his dogs.
The time taken to create the raid's art, abilities, numbers and complete mechanics, varied according to the complexity of encounters. Ion and John told us that Baleroc, a fairly simple fight to create with only a few abilities, took about a week to build. Ragnaros, on the other hand, where the heroic version of the fight was almost a completely new encounter from their perspective, took two full months from inception to completion. The entire game is built using a mixture of a proprietary tool called WoW-Edit, wherein numbers, data, role and performance of spells, and so forth are laid out and edited. Lua code is used as an underlying framework.
QA testing is the next step. Blizzard's internal QA team is first up, going into the encounters with invincible god mode and ensuring that basic mechanics are working as intended. Once that's established, the next group to gain access are the internal design testing team, mostly comprised of raiders. Due to numbers restriction, the Encounter Design tested the encounters as a 10-man raid, then drafted in members of other teams to test out the 25-man settings.
Next up, the raid is opened to PTR or beta, depending on whether it's an expansion or a patch. Here, the raids are opened for a few hours at a time, with encounter designers watching throughout, essentially flying overhead. They're looking for different, creative ways that players are dealing with mechanics and whether those methods should be allowed or prevented.
Beta raids difficulty is usually increased on live servers. The combination of factors like latency and the inability to really practice the content as much as usual, and the need to see how players deal with encounters mean that the beta difficulty needs to be less taxing.
Both Ion and John stressed how undesirable it was to make a raid harder. If you release a raid boss who is initially easy and some players beat him, and then you tune him up to make him harder to beat, the playerbase understandably reacts very negatively. If, however, you release really difficult bosses and then make them easier, they react more positively.
The raids are then monitored very closely indeed for the first couple of weeks for bugs. If you're in a world-first guild, you likely have an audience of GMs flying overhead, watching your every move. Previously, encounters have been hotfixed as raids progress, but this is always something that Blizzard tries to avoid, as its unpredictable nature is seen as undesirable.