Or the scale itself tapers off.
Through one expansion? Sure the legendary will be good for the first few raids, might even last up to the very last thrust. Next expansion? Eh... its starting to show its age, and next thing ya know, its replaced with a mid-level dungeon blue.
So lets take Sulfuron's hammer, it last to about Black Temple, and partway into sunwell. Now its not the best, but it makes the weapons there, on the lower end at least, side-grades and not real upgrades.
You hit Northrend though, and that axe from the anguish tournament in Zul'drak is a clear upgrade. And if you hit 80 still using it, well, its about a high-powered green to low heroic blue.

Or make it scale to a mid-level expansion weapon. Something that could be replaced in early raiding, but would be useful during heroic runs. That way you'd still get to see people carrying them around every so often.

Personally, I always thought that an item capable of "upgrading" the legendaries should drop from the last boss of the expansion, but only to an epic level... For example, a sword that was outstanding on its own could have dropped from K'J in the sunwell, with a "use" effect that you could place on thunderfury, upgrading it to an equivilant epic. (Perhaps with the flavor text of "Energized by the Sunwell itself") That way it wouldn't matter if someone was farming thunderfury, it would still require them to have a level and skill appropriate drop to begin with to get the old legendaries up to snuff.

TL;DR version:
Perhaps a weapon from the "ultimate boss" of an X-pac be "fused" to an old legendary to make them useful again, if not OP. I think that would be a reasonably elegant way to make it work. *shrug*

Actually I have no idea why Blizz links item quality only to ilvl. I think it should be linked to ilvl and level. This way Thunderfury may be legendary for a level 60 character, but only epic for a 63, and then maybe only a lowly common weapon for a level 67. Blizz definitely links ilvl to player level, it's how they populate the quest rewards and why they have level restrictions, so I don't see why it should be impossible for gear quality tooltips to slide along with your level. Then we wouldn't be talking about something that is always Legendary, but obtainable as Legendary and useful up until some max level at which other drops and quest rewards are better... which seems to be Blizz's intent.