The obvious solution to this would be to have the lockouts work like talent trees in a sense, you can kill bosses within a section in any order (optional bosses or simply those that have no specific order in which to be killed) but there is some condition which has to be filled for the next 'tier' of bosses to be available (Either a specific boss dead, or all bosses in a section).

If you're saved on an optional boss and the condition is still not fulfilled you can only access bosses you have not killed personally, however you can also join any raid which has already killed the bosses which you have already completed.

In order to prevent people from killing bosses multiple times then it would be necessary to lock a player out of all bosses in the previous section if they kill a boss and become saved to the following 'tier' of bosses.

This system is suggested by the fact that it was trialled on the bosses of Ulduar and Icecrown citadel, both raids which have clear and obvious sections and divisions with well defined progression levels or tiers.