A very similar situation recently happened to me. During 4.2 I quit the raiding guild I was because I believe, "If you think you can do a better job than someone else, do it". So I started my own raiding guild.

5 months later, I had NO RAIDERS. In fact, I only had a few people in my guild.

I joined a pug one night and had a great time, and I remembered that I LOVE to raid, so I quickly spoke with them about joining their guild. Not only me, but I secured spots for some of my guildies as well.

After convincing (and it took some convincing) my guildies to disband the guild that we'd spent 5 months getting to level 6 (we're slow, leave us alone), I had an epiphany: I didn't want my guildies to leave the guild so that we could play together. I wanted them to leave so that I wouldn't feel guilty about "leaving them". In other words: I was being selfish (in case you're wondering how this case is similar).

I logged in, spoke with a guildie that I trusted the most, told him I would give him leadership of the guild to do with as he pleased.

They're still going. They've hit level 7 now (they have 3 active members), but they're happy. I have my 7 toons in the raiding guild, so I'm happy. Just goes to show you, sometimes, you just have to play the game you want to play and not worry about others around you