Tranphobia
The fear or hate of transvestites, or the fear of having a sexual relation with someone and discovering them to be a transvestite.
A grown man who wont have sex with a woman but is not attracted to men, either suffers from the mental instability of disliking sex, or suffers from Tranphobia