In a sample of urban youth (N = 1,070), we examined the links between primary caregiver affect (i.e., warmth and hostility) and two measures of sexual behavior in adolescence—early sexual initiation and sex with multiple partners. We also examined the extent to which neighborhood disadvantage moderated associations between caregiver affect and adolescent sexual behavior. We found that caregiver hostility was positively associated with early sex and sex with multiple partners in neighborhoods characterized by high levels of disadvantage, but inversely associated with both sex outcomes in neighborhoods characterized by low levels of disadvantage. Caregiver warmth, on the other hand, was inversely associated with early sexual initiation and sex with multiple partners in all neighborhoods, regardless of neighborhood disadvantage.