Cognitive impairment is experienced by many individuals with major depressive disorder (MDD) and is significantly related to sustained disability. Recent work has begun to explore the relationship between childhood adversity (CA) and cognitive impairment in MDD, but this work is limited by unreliable measures of CA. Furthermore, no previous research has examined whether CA relates to cognitive remediation response. The current study clarifies how CA and clinical characteristics of illness explain cognitive variance. In addition, we investigate whether CA is associated with response to cognitive remediation. Thirty-nine individuals who completed cognitive remediation were rerecruited to complete a retrospective interview on CA. Results showed that CA, repeated depressive episodes, and earlier age at diagnosis were associated with poorer cognition. We did not observe a difference in treatment response based on CA. Findings suggest that CA is an important variable to consider when examining the expression of depressive illness and areas for intervention.