There is an add-on that is considered "out-of-date" but it still works very nicely which I have enabled if I want to run undisturbed:

"LeaveMeAlone"

(this link is the direct download link from the Curse development site)

http://static.curseforge.net/content/files/302/259/LeaveMeAlone-1.0.zip

Here's their write-up on what it is capable of doing:

* Option to block incoming Duels.
o Togglable option to block incoming Duels.
o Togglable option to show message about incoming Duels.
o Customizable message to be sent "/w" to the offending player.
o Specify groups that are allowed to ask for Duels: party, guild, friends.
* Option to block incoming Party Invites.
o Togglable option to block incoming Party Invites.
o Togglable option to show message about incoming Party Invites.
o Customizable message to be sent "/w" to the offending player.
o Specify groups that are allowed to ask for Invites: guild, friends.
* Option to block incoming Guild Invites.
o Togglable option to block incoming Guild Invites.
o Togglable option to show message about incoming Guild Invites.
o Customizable message to be sent "/w" to the offending player.
o Specify groups that are allowed to ask for Guild Invites: party, friends.
o Block Guild Charter Signature Requests simultaneously with Guild Invites
* Log that shows who was blocked/allowed, clears on logout.
* Track offending players and auto-ignore after user-specified number of offenses.
* Integration with FuBar - show list of Players blocked this session.
* Auto-Accept Party/Duel Invites from Friends/Guildmates
* Option to block all Trades