Before you can do anything with pet battles, you have to be at least level 5 and have up to 10g, depending on whatever discount you may have.
Now travel to your capital city and chat with the Battle Pet Trainer there. Varzok is located near the flightmaster in Orgrimmar, and Audrey Burnhep hangs out near the Cataclysm portals in Stormwind City.
You only need to train on one of your characters, as all of the training, quests, and battle pets are account-wide. This means that if you train with your main character in Stormwind, your level 1 blood elf can start collecting pets for you in Silvermoon.
But wait! There's more!