A Review of Evaluation Research
by Alan J. Hawkins
The Bottom-line First. From the beginning of the federal Healthy Marriage and Relationships Education (HMRE) policy initiative, there has been an impressive body of serious evaluation work on the effectiveness of relationship education programs designed to help lower income couples form and sustain healthy relationships and marriages. Large-scale, rigorous, randomized controlled trial evaluation studies reveal promising successes, disappointing failures, and nuanced findings in-between. Critics’ claims that the HMRE initiative has been a failure are challenged by a thorough investigation of the research in this area. Still, federal policy needs to support more innovative approaches and strategies to increase the reach of relationship education services and improve their effectiveness. The initiative needs to move beyond a focus on program success to population impact.
Note: This blog is an executive summary of a lengthier blog written for the American Enterprise Institute and published on their website. To read the full report, click here.
We are 15 years into a federal policy initiative to help disadvantaged individuals and couples form and sustain healthy relationships and stable marriages. Family instability contributes to a host of poorer outcomes for children (and adults), and the Administration for Children and Families (ACF) budget picks up a sizable portion of the price tag to try to ameliorate associated problems. To address this problem, ACF has funded hundreds of community-based organizations to provide relationship education services to youth, young adults, cohabiting parents, and married couples to help them gain the knowledge and skills that strengthen romantic relationships. Collectively, these programs are known as the Healthy Relationship and Marriage Education (HMRE) initiative.
From the start, ACF launched a rigorous evaluation agenda. And over the past decade, more than 50 evaluation studies have examined these programs’ effectiveness, including three ACF-funded, large-scale, multisite, random-assignment evaluations.
What have we learned? If you have been listening to policy pundits and scholarly observers, you would be convinced that this policy initiative was a resounding failure. Many pundits have panned the HMRE initiative, and most scholarly observers have been critical. Serious scientific organizations also have been pessimistic. For instance, a recent report from the prestigious National Academy of Sciences concluded that “an ambitious attempt [by the federal government] to develop programs that would improve couple relationship skills, promote marriage, and improve child well-being failed to achieve its goals.”
I disagree with the critics’ conclusions, which, in my view, are based on an early and limited range of evaluation work. I have been closely observing this policy initiative from the beginning, nearly 20 years now. A careful examination of the ongoing, developing work on ACF’s HMRE policy initiative contradicts the death sentence many have prematurely pronounced. Instead, it reveals large, serious, and rigorous evaluation work that shows promising successes, disappointing failures, and nuanced findings. Certainly, in comparison to other social policy initiatives with greater public funding, much less early evaluation work, and even less evidence of success, ACF’s HMRE policy initiative is promising and merits continued policy development and empirical research.
Overall, evaluation research has shown that low-income individuals and couples are interested in these programs. More than two million individuals have completed the programs. Despite startup challenges that affect every new federal social policy initiative, participants report enjoying the programs and say they help.
But is this positive reaction from participants confirmed by rigorous impact evaluation studies? So far, evidence is mixed on whether these programs enhance relationship stability. Some studies show they have a small effect on helping distressed, low-income married couples increase their commitment and remain married. There is no evidence yet that these programs increase the chances that unmarried couples will marry (but may help some stay together longer). Growing evidence shows that couples can learn to reduce destructive conflict and experience less physical and emotional abuse. In addition, growing evidence demonstrates that these programs can improve couples’ positive communication skills, understanding, warmth, support, and co-parenting.
Some studies show positive benefits on individual mental health. Also encouraging is evidence from many studies that the most disadvantaged and distressed couples that come to these programs are the ones that benefit the most. Importantly, emerging evidence shows that children of parents who participate in these programs exhibit fewer behavioral problems, likely a benefit of reduced parental stress.
Holes in the evaluation research remain, especially the longer-term effects of relationship literacy education programs for youth and young adults. And there is plenty of room for programs to increase the magnitude of their effects. Going forward, ACF needs to support innovative approaches and strategies to increase the reach of relationship education services and improve their effectiveness. The initiative needs to move beyond a focus on program success to population impact. This may mean adopting a public health mindset as much as a helping professional or human service approach. Moving the needle on relationship quality and family stability will be the ultimate measure of success for ACF’s HMRE policy initiative.
To read the full report, click here.