Just in time for the new Caverns of Time preview from Blizzard, I recently came across a photoset of the Caverns as they exist now, courtesy of this German WoW blog.
I had to put lots and lots of text through the Babelfish, but from what I can tell, this guy reached the Caverns by climbing through the mountains (probably an exploit, try it at your own risk) in the pirate area on the east coast of Tanaris. Here's what he actually says, in as good a translation as I could get:
"To describe with difficulty. But I am at that time from the rear over the Piratenlager. Then the mountain highly, there in a few certain places down and is one hops sometime in it."
Seems like that's the only way in at this point, because there's four gigantic elite dragons currently guarding the entrance (also along the coast of Tanaris). I know this because I went to visit those dragons, and while I was able to sneak past three of them, the big one (Anachronos) finished me off with one hit. Now, you'd think we'd have to defeat those guys, but Nethaera hinted earlier today that maybe defeating them isn't what we're supposed to do...
At any rate, It seems like the preview and the information already in the game has done exactly what Blizzard planned: get everybody drooling over the prospect of going back throughout the history of Azeroth in a brand new instance. Burning Crusade is still on tap for a Q4 release-- can't wait!