Different Ways to View Program Effectiveness

A Person-Oriented Approach to Evaluating Couple Relationship Education

by Jennifer Griffith

The Bottom-Line (at the Top): There are different ways to gauge whether RE programs are working. The most typical way is to measure the average change of participants. But another valuable way is to focus on variation in how participants change or stay the same, identifying different patterns of change of participants and those who benefit more or less. A recent study with a large, mostly Hispanic sample of couples looked at change with both approaches. It was couples who started the program in the mid-range of communication skills and relationship satisfaction who benefitted the most from the RE program. 

For a long time the primary question posed to relationship educators has been, “Is couple relationship education (CRE) effective?” So, we measure the average level of change of participants in our classes from before to after (and sometimes comparing their change to average change in a control group of non-participants) to determine if RE works. If the average level improves, then we say the program works. This is one approach (researchers call it a variable-oriented approach), but it’s not the only thing that’s valuable to know. Logically, we know that some RE participants will benefit more than others. It’s also important to understand not just average change but variation in how RE affects participants, that is, who changes and who doesn’t, who changes the most and the least. Some evaluation studies have tried to get at this question too. Recently, a team of researchers from Texas Tech University and the University of Minnesota employed a new statistical approach (person-oriented approach) to get at variation in participant change. In the study entitled, “A Person-Oriented Analysis of Couple and Relationship Education,”[1] these researchers tried to identify different groups of CRE participants by how they changed going through a program.

This study followed 1,086 mostly lower income and Hispanic couples[2] who attended a 20-hour PREP-based Within Our Reach (WOR) program focused on healthy communication, conflict resolution, and other activities intended to improve relationship skills. Participants completed a survey before their first session and a post-program survey following their last session. Survey questions focused on perceived communication, satisfaction, commitment, confidence, and conflict resolution.

There were a lot of interesting findings in this study, but let’s focus on the different patterns of change (and compare those to the more typical way of looking at program effectiveness, that is, looking at average change). Here is a summary of those findings:

  • Results with (Typical) Variable-Oriented Analyses: On AVERAGE, both women and men reported improved couple communication at the end of the WOR That’s good news, but let’s look at it from another analytic perspective . . .
  • Results from Person-Oriented Analyses: The researchers found three distinct groups that had different patterns of change as a result of the program:
    • Group 1 – “Tune-Up” Couples: A large percentage of participants (44%) were in this group. They reported pretty high levels of communication skills and relationship satisfaction before beginning WOR, with fairly small improvement at post-test (although still statistically significant). This makes sense because they didn’t have a lot of room for improvement. (Researchers call this a “ceiling effect.”) These couples seemed to be interested in WOR more as a tune-up of their relationship than a major repair.
    • Group 2 – “Major Repair” Couples: This group included a smaller percentage of participants (16%). Pre-program measures showed their communication skills and relationship satisfaction levels were pretty low, with only small improvements at post-test (although, again, statistically significant). Perhaps these couples came to WOR wanting to do some major repair work on their relationships. They had a lot of room for improvement, but they only improved a little. A little improvement is better than none, but it would be nice to see more.
    • Group 3: “Moderate Repair” Couples: The rest of the participants (40%) fell into this third group. Pre-program measures showed this group as mid-range – not too high, not too low – for both communication skills and relationship satisfaction, but they experienced sizable improvements at post-tests, much larger than the other groups. Perhaps these couples came to WOR wanting to work on some specific issues, wanting more than a “tune up,” like Group 1, but not needing a major overhaul, like Group 2. Whatever their situation, they seemed to get the most benefit from the program.

I think there are some interesting questions and implications for relationship educators (and evaluators) based on these findings:

  • Are mid-range couples the sweet spot for RE? This study found that couples whose relationship skills and satisfaction fell in the midrange got the most out of the program. Now, we need to be cautious here because a good number of studies have found (using different statistical techniques) that the most distressed couples are the ones who get the most benefit from RE.[3] But maybe for Hispanic participants, who made up the bulk of this study’s sample, mid-range couples may benefit most. So . . .
  • Should we slacken our efforts to include well-functioning couples from participating in our RE programs? After all, this study showed that these couples – nearly half of the sample – only saw small improvements and they were already doing well. Should we save this space so that we can serve more couples who are struggling more in their relationships? Especially when resources are scarce, this could be a needed choice. But I want to be cautious here too. First, as a field, we want to send the message that RE is for everyone, even when things are going well. We preach the value of prevention, so it would send an odd message if we screened folks out of our RE pews because they don’t really need it. Moreover, I think it’s a real possibility that including couples who are doing well in our classes creates a good dynamic for the whole class that supports learning. If everyone in the class is struggling, it might send the unintentional message that no one is really able to build strong, happy relationships. I’m not sure about this idea, but I’m cautious about making RE feel more like it’s really just for those with problems. I worry that this would create more stigma for our classes.
  • But should we screen out couples who are experiencing major problems? This study showed that couples who begin the program at fairly low levels of relationship quality only improved a little. Maybe other more intensive services are needed for them. But again, I want to be cautious. First, these couples did benefit a little bit. That small improvement may give them a greater sense of hope that will keep them working on the relationship over time. That’s nothing to sneeze at. Second, as mentioned above, other studies suggest that the most distressed participants gain the most. Each study needs to be interpreted in the context of the broader set of studies. Third, these participants likely are more disadvantaged in many ways, and as a field, we don’t want to send a message that our services are not for them. And fourth, an important element of our RE services could be helping these disadvantaged couples who participate in our programs to gain access to other valuable services that can help address other needs (e.g., housing, employment, nutrition, literacy, addiction) and reduce stresses in their lives, which in turn could improve their relationships.
  • Should we change how we conduct evaluations of our programs? It’s good to know how our programs, on average, affect our participants. But just as helpful is identifying those groups that are benefiting more or less from our interventions. We need to be comfortable with the reality that some benefit more and some benefit less and even that some may not benefit at all. People are different and life is complicated. What we learn then should spark greater curiosity about why certain groups get more out of RE than others. And from this curiosity could come learning that helps us improve our services. Many evaluators now-a-days have the technical skills to analyze our programs with this more person-oriented approach.

Understanding better different patterns of change and who benefits more or less from RE programs doesn’t mean that we then just target those potential participants who may benefit the most. But knowing this should help us improve our services overall.



[1] DuPree, D. G., Whiting, J. B., & Harris, S. M. (2016). A Person-oriented analysis of couple and relationship education. Family Relations, 65(5), 635-646. doi:10.1111/fare.12222

[2] Participants were receiving some kind of government public assistance benefits (e.g., TANF, WIC, SNAP) and had a child eligible for the Head Start program. Also, there were 675 mostly female participants either not in a relationship or attending as a single, but they are not the focus of this blog.

[3] For a summary of this body of research see: Hawkins, A. J., Erickson Allen, S. E., & Yang, C. (2017). How does couple and relationship education affect relationship hope? An intervention-process study with lower income couples. Family Relations. doi: 10.1111/fare.12268