Women with serious mental illness (SMI) occupy a social position in which their experiences are simultaneously influenced by stigmatisation, institutionalisation and gendered dimensions of trauma and power. Women with SMI are stigmatised in society, pushed to the margins and left vulnerable to victimisation and rejection, particularly if hospitalised in psychiatric institutions. Using modified labelling theory, I investigate how women hospitalised with SMI, especially those who have histories of sexual abuse and trauma, experience sexuality and perceive men and masculinity. I extend modified labelling theory by empirically analysing the role of social status and power in the labelling process. Specifically, I focus on gender, and analyse interviews with 55 women in US psychiatric hospitals, focusing on gendered experiences of trauma, stigma and attitudes about sexuality. I show how trauma increases the salience of stigma and potential for retraumatisation, both of which are amplified by the institutional setting. I find meaningful differences in the narratives of women who have experienced trauma and those who have not. Modified labelling theory helps explain how labelling can perpetuate self‐stigma, which threatens women’s self‐esteem, safety and trust in others. The intersection of these experiences extends modified labelling theory and may have profound implications for recovery.