We aim to study Adverse childhood experiences’ (ACEs’) association with cognitive decline during adulthood. We reviewed data on subjective cognitive decline (SCD) for those ≥45 years of age. Logistic regression analysis, odds ratio, and 95% confidence interval (CI) assessed the association between ACEs and SCD. We analyzed an adult survey (N = 3900, mean age: 60.9 years, male: 46%). The most common ACE reported was household substance abuse (22.4%), followed by parental divorce/separation (20.9%). The prevalence of SCD in those having at least one ACE was higher than those without any (16.5% vs. 8.1%, p