While most studies on sexuality in later life report that sexual desire declines with age, little is known about the exact nature of age effects on sexual desire. Using self-reported dyadic sexual desire relating to a partner, dyadic sexual desire relating to an attractive person, and solitary sexual desire from a large (N > 8000) and age diverse (14.6–80.2 years) online sample, the current study had three goals: First, we investigated relationships between men and women’s sexual desire and age. Second, we examined whether individual differences such as gender/sex, sexual orientation, self-rated masculinity, relationship status, self-rated attractiveness, and self-rated health predict sexual desire. Third, we examined how these associations differed across sexual desire facets. On average, the associations between age and both men and women’s sexual desire followed nonlinear trends and differed between genders/sexes and types of sexual desire. Average levels of all types of sexual desire were generally higher in men. Dyadic sexual desire related positively to self-rated masculinity and having a romantic partner and solitary desire was higher in people with same-sex attraction. We discuss the results in the context of the evolutionary hypothesis that predict an increase of sexual desire and female reproductive effort prior to declining fertility. Our findings both support and challenge beliefs about gender/sex specificity of age effects on sexual desire and highlight the importance of differentiating between desire types.