I've never heard of this happening - I don't have any problems with losing the UI when attacked but it sounds like a vehicle-type issue. Do you have anything set up that auto-dismounts you from a vehicle under certain circumstances?

I have checked and double-checked all the settings... I am baffled. I guess I will just have to disable all addons whenever that one comes up. Just annoying...

Sounds it most likely is Bartender4. Try disabling it and see if it stops. If so, maybe you have your Bartender4 vehicle bar (or is it the petbar?) set up to disappear in combat? That's the only thing I can think of.

You should get Bartender Dual Spec to go along with Bartender. I'm pretty sure that will keep your problem from happening, as it swaps the main bar to the vehicle bar.

Also in Bartender, make sure that you have "Vehicle Bar" enabled and thrown into a corner somewhere. That way, you'll always have the exit button when it's available.

Last time I picked up a turtle, I didn't have an action bar, but pressing 1 still punted the turtle, either at the water or at a mob that was attacking me (1-shots the mob, btw).