Although the benefits of physical activity (PA) on mental well-being are well established, the vulnerability of women during pregnancy and the postpartum period poses challenges in studying the effects of PA on postpartum depression (PPD). This study investigated the association between total and domain-specific PA levels during pregnancy and postpartum depressive symptoms. The study included 105 healthy pregnant women whose PA status was evaluated by the Serbian version of the Physical Activity Questionnaire during Pregnancy (PPAQ-SRB) and their postpartum mental health by the Edinburgh Postnatal Depression Scale. Multivariable logistic regression was used to explore the independent relationship between PPAQ-SRB scores and the prediction of the PPD symptom occurrence. Our analysis revealed that the development of PPD symptomatology was associated with a lower “Total PPAQ-SRB score” (odds ratio (OR) 0.81; 95% confidence interval (CI) [0.70–0.93]; p = 0.03) and “Total Activity score” (OR 0.82; 95% CI [0.71–0.93]; p = 0.03) as well as lower scores of light-intensity PA (OR 0.81; 95% CI [0.69–0.96]; p = 0.013), moderate-intensity PA (OR 0.82; 95% CI [0.71–0.94]; p = 0.005), “Household/care” (OR 0.85; 95% CI [0.73–0.98]; p = 0.028), and “Occupational” activities (OR 0.80; 95% CI [0.78–0.95]; p = 0.012). Lower levels of light-to-moderate-intensity household and occupational activities appeared to increase the risk of PPD, indicating the importance of circumstances under which PA is performed. Hence, our findings indicate that PA during pregnancy can mitigate mood disorders in postpartum mothers. Moreover, reduced participation in light-to-moderate-intensity household and occupational activities seemed to increase the risk of PPD.