It's funny how misconceptions get started.
Are you ready to spill some blood in the name of the Light? For Velen? For Argus?
-- Vindicator Boros, What Argus Means To Me
We know that the orcish Horde led by Blackhand, corrupted by the blood of Mannoroth and manipulated by Gul'dan for his master Kil'Jaeden was a force of unrelenting bloodlust and cruelty, and that they waged a genocidal war on their draenei neighbors that came close to wiping them out. Not a very numerous race in the first place, the draenei were ultimately overwhelmed by the orcs.
Many seem to combine this with the fact that Velen led the first draenei from their homeworld of Argus to prevent them from being corrupted by the offer Sargeras made to Archimonde, Kil'Jaeden and Velen to accept his gifts and become his servants to assume that the draenei are pacifists. That they're a culture that eschews war and flees from conflict. They note that the draenei have numerous times fled before the advance of the Burning Legion, moving from world to world before the dimension ship they were aboard crashed on what would become known as Draenor.
When combined with their mastery of and reverence for The Holy Light and their association with the naaru, who themselves are beings that exude said Holy Light, it's easy to see why people make this mistake. They look at Velen, who prophecizes that one day the mortal races will unite against the Legion, and see this as a desire for peace. But it isn't, exactly. Velen isn't preaching peace - he's arguing that we're fighting the wrong war.
It's easy for those from the perspective of Azeroth, a world that has twice fought off the Legion, to see flight from said Legion as either a form or cowardice or a refusal to engage in violence. But it was and is neither. The Legion has invaded countless worlds, and until it reached Azeroth it was always successful. The Legion does not merely consist of Eredar Warlocks - a host of demonic races (including the Nathrezim, the Annihilan pit lords, the Shivan and many others) flock to its banner in droves. The Legion can throw nearly infinite numbers at a world, limited only by the portals they use to invade in many cases. Any attempt to defeat the Legion has to account for this vast numerical superiority, and the draenei have often been placed in the situation of having the decide whether to flee and survive, or stay and fight and be overwhelmed by an enemy that can gleefully spend hundreds if not thousands of its own to kill one of yours and call it a bargain.
The draenei are many things. Two things they are not - they are not pacifists, and they are not stupid, either. Had they been able to flee Draenor when the orcs first became corrupted, they probably would have. But for the first time in twenty-five thousand years, Kil'Jaeden had outmaneuvered them. Think on that - for twenty five thousand years, Kil'Jaeden the deceiver matched wits with Velen, and lost. But that's the trouble with an enemy who seeks to destroy you, and has the resources you lack. You have to win every single time. He only has to win once. Given that there still are draenei alive right now, I suppose we can say he didn't quite - but if killing almost all of your worst enemies' people isn't victory, it's still too close to call for the draenei. Already hardened by millennia of unceasing pursuit, those that remain are a grim, determined and absolutely not pacifist group.
Seeing Romnar go down brought back memories to Maraad of battling the undead, and his crystalline hammer no longer merely parried but began to come down with crushing force on the invaders. Once he had cut loose the bonds of mercy, the rest of the draenei followed, and the beginnings of a slaughter were writ in the blood of the refugees.
-- Prophet's Lesson, Marc Hutcheson
The draenei do not love war, or see it as particularly noble or honorable, it's true. They are not enamored of bloodshed. They do not sing songs of valor on the battleground, or commit acts of genocide to prove their prowess. Dominating the weak and crushing their enemies are not concepts that have much appeal to them. They know mercy, and restraint - forever haunted by the actions of those that were once their own people, they are constantly aware (and afraid of) their own potential for darkness and evil action. This is why they are at time slow to respond to a threat, and often choose to retreat rather than come into conflict - not because they are incapable or unsuited to it, but rather because they are aware of just how suited to it they are. One needs only look at Archimonde, shattering Dalaran to dust with a few spoken words, to realize the potential for power and cruelty every draenei fears lurking within his or her own heart. It's tempting to look at the draenei response to the broken as a reflection of this fear - that without the presence of the Holy Light, they will inevitably fall to the madness of the Man'ari Eredar.
Draenei know mercy. They force themselves to employ it. But they are not perfect - not even the Prophet is perfect - and they have at times lost that mercy. The draenei did not go meekly to their demise when the orcs attacked - even while Velen tried to discover why the formerly cool but not-aggressive relationship with the orcs had become violent, and tried to explain to them the truth of Oshu'gun for both their people's benefit, the draenei defended themselves, to the orcs' sorrow. Many orcs died before the rise of the warlocks, and many more died after - every orc victory in the war came at a terrible cost. The orcs resorted to aging their own children into adulthood early in order to throw them into the conflict as cannon fodder, mass demon summonings, and finally the drinking of the Blood of Mannoroth because they had to. There was no other way for them to win, and Gul'dan knew his own neck was on the chopping block if he lost. The draenei did not sit back and pray when all of this was going on - they fought, and killed.
They were not pacifists then. They are not pacifists now. They do not all believe the same things - they all respect Velen (he's not just their leader, he's an actual Prophet who has led them for over 25,000 years) but none are drones who lockstep fall into line instantly with what he says or does - and often Velen doesn't say very much in the first place. As we see in Prophet's Lesson, Velen often becomes detached, searching his visions instead of focused on the now, leaving the day to day leadership of his people in the hands of others. Some of those others have different ideas of what the proper course of action is - some don't see the point of staying on Azeroth as part of the Alliance, for instance. Others would like revenge - a path Velen sees as futile at best, and a distraction from the real war he has seen in visions. But even when Velen appears in all his glory and declares his intent, there are those that do not simply bow to his wishes.
The easiest way to counter this impression of the draenei is to play one from 1 to 20, and experience the entirety of the draenei starting zone on Azuremyst and Bloodmyst Isle. You're essentially recruited into the Hand of Argus, a military organization equal parts spy agency and special forces, and in that capacity you're tasked with the assassination of an eredar and her blood elf minions, and upon the completion of this incredibly dangerous and violent series of missions, you are lauded not only as a hero of your people but the Prophet himself comes to congratulate you. The same Prophet Velen who ordered you to kill all those blood elves and that eredar, Sironas, in the first place.
Preferring peace to war isn't pacifism. It's simply common sense for a people who are outnumbered and hunted by a force willing to murder every world in existence. Their choice to limit the use of magic and settle in isolation wasn't out of a desire for ascetic retreat from conflict, it was a desire to continue existing and evade the Legion, especially with Oshu'gun destroyed so that retreat was impossible. The naaru that had piloted that vessel was no longer 'alive' (in a sense, naaru life cycles aren't the same as our own) and so, options were limited.
The draenei are not pacifists. There are most likely pacifists among them, as there are among almost all peoples, but it is most definitely not a universal trait.
While you don't need to have played the previous Warcraft games to enjoy World of Warcraft, a little history goes a long way toward making the game a lot more fun. Dig into even more of the lore and history behind the World of Warcraft in WoW Insider's Guide to Warcraft Lore.