Sexual quality of life (SQoL) challenges are prevalent and persistent for people with a cancer diagnosis. Using a qualitative design, this study investigated the impact of a cancer diagnosis on SQoL in participants (N = 18) with a current or past cancer diagnosis of various sites and severity. Thematic analysis identified three themes: gender attachment, vulnerability, and growth vs. fixed mindset. The theme of gender attachment related to the impact of understanding SQoL based on heteronormative definitions of masculinity and femininity. The second theme of vulnerability suggested that pre-existing relational or personal challenges can amplify the impact of a cancer diagnosis on SQoL. The third theme of growth vs. fixed mindset explored how cognitive flexibility and optimism can support SQoL, while a rigid and pessimistic mindset can exacerbate SQoL challenges. The findings of this study build on the traditional medical understanding of SQoL and can inform SQoL support in cancer care settings.