Previous research was not able to identify an effect of the COVID-19 pandemic on family care-givers’ wellbeing due to cross-sectional data structures, care-giver-specific samples or non-probability sampling designs. In this article, by drawing on longitudinal data from a random and representative sample of the German population aged 40 and older, we overcome methodological limitations of previous research. We examine the impact of the COVID-19 pandemic on the wellbeing (life satisfaction, depressive symptoms, loneliness) of different groups of family care-givers (new care-givers, continuous care-givers, past care-givers) and non-care-givers in Germany. We derived a balanced sample of n = 3,875 individuals aged 43–97 (mean = 66.5 years, standard deviation = 10.2 years) from the German Ageing Survey (DEAS) to explore changes in wellbeing between 2017 (pre-pandemic) and June/July 2020 (pandemic). Using a first-difference approach, we found that the pandemic has a negative impact on both non-care-givers’ and family care-givers’ wellbeing, with increased levels of depressive symptoms and loneliness compared to pre-pandemic times. Changes in depressive symptoms of family care-givers did not significantly deviate from non-care-givers. However, continuous care-givers showed more pronounced increases in loneliness compared to non-care-givers. Our results suggest that family care-givers’ wellbeing merits particular attention when considering the long-term consequences of the COVID-19 pandemic on individuals’ wellbeing.