Abstract
To describe the design and implementation of an evaluation system to facilitate continuous quality improvement (CQI) and scientific
evaluation in a statewide home visiting program, and to provide a summary of the system’s progress in meeting intended outputs
and short-term outcomes. Help Us Grow Successfully (HUGS) is a statewide home visiting program that provides services to at-risk
pregnant/post-partum women, children (0–5 years), and their families. The program goals are to improve parenting skills and
connect families to needed services and thus improve the health of the service population. The evaluation system is designed
to: (1) integrate evaluation into daily workflow; (2) utilize standardized screening and evaluation tools; (3) facilitate
a culture of CQI in program management; and, (4) facilitate scientifically rigorous evaluations. The review of the system’s
design and implementation occurred through a formative evaluation process (reach, dose, and fidelity). Data was collected
through electronic and paper surveys, administrative data, and notes from management meetings, and medical chart review. In
the design phase, four process and forty outcome measures were selected and are tracked using standardized screening and monitoring
tools. During implementation, the reach and dose of training were adequate to successfully launch the evaluation/CQI system.
All staff (n = 165) use the system for management of families; the supervisors (n = 18) use the system to track routine program activities. Data quality and availability is sufficient to support periodic
program reviews at the region and state level. In the first 7 months, the HUGS evaluation system tracked 3,794 families (7,937
individuals). System use and acceptance is high. A successful implementation of a structured evaluation system with a strong
CQI component is feasible in an existing, large statewide program. The evaluation/CQI system is an effective mechanism to
drive modest change in management of the program.
evaluation in a statewide home visiting program, and to provide a summary of the system’s progress in meeting intended outputs
and short-term outcomes. Help Us Grow Successfully (HUGS) is a statewide home visiting program that provides services to at-risk
pregnant/post-partum women, children (0–5 years), and their families. The program goals are to improve parenting skills and
connect families to needed services and thus improve the health of the service population. The evaluation system is designed
to: (1) integrate evaluation into daily workflow; (2) utilize standardized screening and evaluation tools; (3) facilitate
a culture of CQI in program management; and, (4) facilitate scientifically rigorous evaluations. The review of the system’s
design and implementation occurred through a formative evaluation process (reach, dose, and fidelity). Data was collected
through electronic and paper surveys, administrative data, and notes from management meetings, and medical chart review. In
the design phase, four process and forty outcome measures were selected and are tracked using standardized screening and monitoring
tools. During implementation, the reach and dose of training were adequate to successfully launch the evaluation/CQI system.
All staff (n = 165) use the system for management of families; the supervisors (n = 18) use the system to track routine program activities. Data quality and availability is sufficient to support periodic
program reviews at the region and state level. In the first 7 months, the HUGS evaluation system tracked 3,794 families (7,937
individuals). System use and acceptance is high. A successful implementation of a structured evaluation system with a strong
CQI component is feasible in an existing, large statewide program. The evaluation/CQI system is an effective mechanism to
drive modest change in management of the program.
- Content Type Journal Article
- Pages 1-16
- DOI 10.1007/s10995-011-0906-6
- Authors
- Bridget K. McCabe, Bureau of Health Services, Tennessee Department of Health, Cordell Hull Building, 4th Floor, 425 5th Avenue North, Nashville, TN 37243, USA
- Dru Potash, Bureau of Health Services, Tennessee Department of Health, Cordell Hull Building, 4th Floor, 425 5th Avenue North, Nashville, TN 37243, USA
- Ellen Omohundro, Bureau of Health Services, Tennessee Department of Health, Cordell Hull Building, 4th Floor, 425 5th Avenue North, Nashville, TN 37243, USA
- Cathy R. Taylor, Bureau of Health Services, Tennessee Department of Health, Cordell Hull Building, 4th Floor, 425 5th Avenue North, Nashville, TN 37243, USA
- Journal Maternal and Child Health Journal
- Online ISSN 1573-6628
- Print ISSN 1092-7875