Social withdrawal during adolescence and early adulthood is particularly problematic due to the increasing importance of social interactions during these ages. Yet little is known about the changes, trajectories, or correlates of being withdrawn during this transition to adulthood. The purpose of this study was to examine the normative change and distinct trajectories of withdrawal in order to identify adolescents and early adults at greatest risk for maladjustment. Participants were from a Dutch population-based cohort study (Tracking Adolescents’ Individual Lives Survey), including 1917 adolescents who were assessed at four waves from the age of 16 to 25 years. Five items from the Youth Self Report and Adult Self Report were found to be measurement invariant and used to assess longitudinal changes in social withdrawal. Overall, participants followed a U-shaped trajectory of social withdrawal, where withdrawal decreased from ages 16 to 19 years, remained stable from 19 to 22 years, and increased from 22 to 25 years. Furthermore, three distinct trajectory classes of withdrawal emerged: a low-stable group (71.8%), a high-decreasing group (12.0%), and a low-curvilinear group (16.2%). The three classes differed on: shyness, social affiliation, reduced social contact, anxiety, and antisocial behaviors. The high-decreasing group endorsed the highest social maladjustment, followed by the low-curvilinear group, and the low-stable group was highly adjusted. We discuss the potential contribution of the changing social network in influencing withdrawal levels, the distinct characteristics of each trajectory group, and future directions in the study of social withdrawal in adolescence and early adulthood.