Design and Implementation of an Integrated, Continuous Evaluation, and Quality Improvement System for a State-Based Home-Visiting Program

To describe the design and implementation of an evaluation system to facilitate continuous quality improvement (CQI) and scientific
evaluation in a statewide home visiting program, and to provide a summary of the system’s progress in meeting intended outputs
and short-term outcomes. Help Us Grow Successfully (HUGS) is a statewide home visiting program that provides services to at-risk
pregnant/post-partum women, children (0–5 years), and their families. The program goals are to improve parenting skills and
connect families to needed services and thus improve the health of the service population. The evaluation system is designed
to: (1) integrate evaluation into daily workflow; (2) utilize standardized screening and evaluation tools; (3) facilitate
a culture of CQI in program management; and, (4) facilitate scientifically rigorous evaluations. The review of the system’s
design and implementation occurred through a formative evaluation process (reach, dose, and fidelity). Data was collected
through electronic and paper surveys, administrative data, and notes from management meetings, and medical chart review. In
the design phase, four process and forty outcome measures were selected and are tracked using standardized screening and monitoring
tools. During implementation, the reach and dose of training were adequate to successfully launch the evaluation/CQI system.
All staff (n = 165) use the system for management of families; the supervisors (n = 18) use the system to track routine program activities. Data quality and availability is sufficient to support periodic
program reviews at the region and state level. In the first 7 months, the HUGS evaluation system tracked 3,794 families (7,937
individuals). System use and acceptance is high. A successful implementation of a structured evaluation system with a strong
CQI component is feasible in an existing, large statewide program. The evaluation/CQI system is an effective mechanism to
drive modest change in management of the program.