Some genetic males undergo voluntary castration, with and without hormonal supplementation. Here, we investigate the sexual function of 163 such individuals (average age = 52 ± 16-years-old). Specifically, we explored how hormonal, social, and psychological factors play a role in their sexual function. In this study, 47% identified their gender as “eunuch”, and 36% identified their gender as “man.” Furthermore, 64% were in a relationship (80% with a woman), and 28% had a strong attraction for both sexes (Kinsey 2–4). We found that castrated individuals with androgen supplementation have higher sexual function than those without any hormone supplementation. Individuals with anxiety symptoms reported better sexual parameters (better orgasm satisfaction, easier to achieve an erection and an orgasm) than those without anxiety symptoms. Among those low in anxiety, individuals without hormone supplementation had a weaker sex drive and more difficulty in getting sexually aroused than those on supplemental estrogen or androgen. Hierarchical multiple regressions indicated that control variables (i.e., age, depressive symptoms, time since castration, type of hormone supplementation) accounted for 13–30% of the variances in sexual parameters. Including sexual guilt in the models predicted 4–8% additional variances in all sexual parameters except for ease in getting and maintaining an erection. Adding in participants’ sexual trauma scores accounted for an additional 3.7% in the strength of their reported sex drive. In conclusion, various factors—hormone therapy, anxiety, sexual guilt, and childhood trauma—may influence the sexual outcomes for castrated individuals.