Parental stress has been identified as a major affecter of caregivers’ psychological well-being and a risk increaser for unwillingly placing children with disabilities in the care of others. Recognition of effective means to ease caregiving burdens must guide policymaking and will help to provide better and tailored support and intervention for the children. This study aims to characterize caregivers of children with intellectual disabilities who are about to seek outside support and care and to identify the conditions under which psychological well-being and parental stress are affected using parental stress as a mediating factor. We found that parental stress and the child’s depressive feelings strongly affected psychological well-being. We also found an indirect relation of restricted caregiver social activities through parental stress. To protect psychological well-being of caregivers, support services should address depressive feelings among children with intellectual disabilities, facilitate caregivers’ social activity, and reduce stress.