“Survey Says”… The Effectiveness of Program Satisfaction and Learning Questionnaires at Predicting Later Marital Quality

by Jennifer Griffith

The Bottom-line First: If program administrators don’t have the resources to do a rigorous, long-term evaluation of the impact of their relationship education programs, what other options do they have to document program effectiveness? As it turns out, recent research suggests that simple post-program surveys of participants’ reports of learning and intent to change are pretty good predictors of relationship outcomes down the road.

It can be difficult to know whether relationship education curriculum makes a difference in the long-run. What would be the purpose of creating programs with a hope for increased communication skills, marital quality, happiness, and a plethora of other outcomes if those interventions did not work longer than the duration of a program? But program administrators often don’t have the resources to follow-up with program participants a few years down the road to see how those participants are doing. Much more common is to have participants complete a brief survey at the beginning and end of the program to assess short-term change during the program and hope that it sticks. Also, these surveys often ask about overall learning and satisfaction with the program. But are these seemingly simplistic questions effective at predicting marital quality months and even years after the RE program concludes? In their study, Associations Between Participant Ratings of PREP for Strong Bonds and Marital Outcomes 1-Year Postintervention,[1] a group of researchers set out to answer this very question.

The study included 191 couples with at least one spouse actively serving in the Army.[2] The couples had each taken part in the military PREP[3]  program called Strong Bonds, a curriculum designed specifically for the Army and led by Army chaplains. Previous research has shown that this program strengthens couple relationships and significantly reduces the chances of divorce two years after the program.[4] The program consisted of a 1-day training and a weekend hotel retreat for participants – a total of 15 hours of content. Participants each filled out a baseline questionnaire before the program, and had follow-up surveys sent to them at 6-month intervals over the course of 4 years. The current study focuses on baseline, post, and one-year follow-up results. The researchers asked questions about the program overall, facilitator effectiveness, couple relationship confidence and intention to change, helpfulness of specific program elements, and impact on Army life. Were reports of program satisfaction and learning at the end of the program associated with relationship outcomes a year later? Their results are fascinating:

  • Communication Skills. Higher communication skills one year after the PREP for Strong Bonds program were associated with immediate post-program reports of confidence in communication skills, desire to work on their relationship, and overall program learning. Importantly, the couples who rated program effectiveness higher in these aspects showed greater sustaining of their communication skills after one year.
  • Marital Satisfaction. Participants who scored higher on the “ability to discuss issues constructively” and “intention to invest more in their relationship” items immediately after completing the program had higher marital satisfaction at their 1-year follow-up assessment.
  • Some Questions Were Not Moderators. Not all items on post-intervention surveys were successful at predicting marital quality 1 year after the program, such as general questions about the overall satisfaction with the program (not a lot of variation there) and ratings of the program facilitators’ effectiveness.

This study is good news to relationship educators on the effectiveness of simple participant ratings, and carries some valuable implications:

  • Have Confidence in Simple Surveys. Program leaders can give simple post-program surveys to participants with basic questions about their learning and have confidence that the positive results immediately after the program modestly predict important aspects of a couple’s relationship down the road. Questions about intent to invest more in the relationship may be especially valuable. A question about general satisfaction may not be all that revealing (although other research suggests that even such general questions may predict later outcomes, especially for more disadvantaged and distressed participants[5]).
  • Document Success Without Extensive Resources. Program leaders with scarce resources can use these simple, short-term evaluation surveys as proxies for expensive, long-term evaluation studies to document the effectiveness of their services because positive, short-term results are modestly correlated with longer-term couple outcome.
  • Use Ratings for Effective Follow-Up and Program Improvement. Once participant ratings are received, relationship educators and program leaders can use lower scores on items as a means of follow-up with a specific couples’ progress, perhaps using relationship check-ups (see our post Can Relationship Checkups Work?) or other means. In addition, these ratings can continue to serve as a way to refine curricula and increase program effectiveness for future participants.

 

Footnotes:

[1] Allen, E. S., Post, K. M., Markman, H. J., Rhoades, G. K., & Stanley, S. M. (2017). Associations between participant ratings of PREP for strong bonds and marital outcomes 1 year postintervention. Military Psychology29(4), 283-293. doi:10.1037/mil0000155

[2] Eligibility to participate in the PREP for Strong Bonds initiative was determined by whether the person was married, over 18, fluent in English, and if at least one spouse was on active duty in the Army.

[3] Prevention and Relationship Education Program.

[4] Stanley, S. M., Rhoades, G. K., Loew, B. A., Allen, E. S., Carter, S., Osborne, L. J., . . . Markman, H. J. (2014). A randomized controlled trial of relationship education in the U.S. Army: 2-year outcomes. Family Relations, 63, 482–495.

[5] Owen, J., Antle, B., & Barbee, A. (2014). Does adherence to relationship education curriculum relate to participants’ outcomes? Couple and Family Psychology: Research and Practice, 3, 99–109.