Tuesday, August 12, 2008

Knowing the research

Welcome back faithful readers (all 131 of you according to google analytics)! Since the first research project I conducted at Northwestern University, a meta-analysis entitled: "Training Spatial Skills: What Works, for Whom and for How Long?" was just recently submitted to Psychological Bulletin for publication (woo hoo!), I thought this would be a good time to share findings from some recent tutoring/mentoring research and discuss how they can help us in our quest to promote effective tutoring/mentoring here at Cabrini Connections and elsewhere.

As tutoring/mentoring programs such as ours and Big Brothers/Big Sisters become more and more widespread, there has been an increasing amount of research both evaluating individual programs, as well as synthesizing previous work, such as by using the same meta-analytic methods we employed in the aforementioned paper.

By the way, for the uninitiated, a meta-analysis essentially involves collecting data from a number of previous studies that all were investigating a related hypothesis and then analyzing that data to draw further inferences. For example, in our study, we wanted to know what types of trainings could improve children and adults' spatial abilities and for how long. So we collected hundreds of studies that looked at the effectiveness of particular types of training and then analyzed them all together in order to determine if they indeed can be improved with training and, if so, what the best ways to improve them are.

One of the most significant studies pertaining to tutoring/mentoring was published in 2000 and looked at just under 1000 youth who participated in Big Brothers/Big Sisters (BB/BS) programs around the country (1). Like any good experiment, it had a control group, roughly 500 kids who applied for a mentor at BB/BS but were not placed with one, that was compared with an experimental group, the other 500 kids who WERE placed with a mentor. This study found that kids who were paired up with a mentor at BB/BS:

-- were 46% less likely to initiate drug use

-- were 26% less likely to initiate alcohol use (that number reaches 50 percent for the girls in the programs)

-- were 33% less likely to hit someone

--Skipped half as many days of school

--Reported improved parent and peer relationships (this was especially true among boys)

Above all else, these results demonstrated conclusively that MENTORING WORKS and are cited by organizations such as ours to argue for the effectiveness of our programming and why it is necessary for the kids we serve. By showing the effectiveness of mentoring in a well-thought out, controlled and published experiment, this study laid the groundwork for future investigations into the effectiveness of particular types of tutoring/mentoring and specific programmatic content.

One of these such studies that analyzed different mentoring methods with the intent to determine the most effective practices, was a meta-analysis published in 2002. This study looked at 55 different studies with 575 effect sizes (quantifiable changes in the youth served). The general finding was that, when taken as a whole, mentoring programs do provide a positive impact on youth, but not as large as might have been expected. As might be expected, this news was a bit unsettling for many people in the mentoring community, since this wasn't an analysis of one particular mentoring program, but rather 55 programs, utilizing many different techniques, and it didn't report the huge positive impacts found in the aforementioned study and others.

However, the real benefit of this study is in the so-called "moderators of impact". These are the personal traits of the mentors and mentees, structures of the particular programs and the characteristics of the mentor-mentee relationships. When these "moderators" are examined, a much sunnier picture of youth mentoring is revealed. For example:

--The programs in the study that provided ongoing training for mentors, offered matches structured activities, set firm requirements around frequency of mentor-mentee contact,
offered mentor support services, or found ways to increase parent involvement showed
a greater impact. All these factors were strong predictors of higher outcomes for youth.

--The programs where youth felt most positive about their relationships also had the best outcomes.

--The impact of mentoring seemed to be greatest for youth who were most at-risk. Here is evidence that mentoring helps those who need it most. (i.e. youth living in extreme poverty such as that found in Cabrini Green and other housing projects all around Chicago).

Thus, this study really is a strong argument for the value of program quality. Simply signing a child up for a tutor/mentor program and sticking them in a room with an adult without careful consideration of the program structure is not going to lead to the most ideal outcomes for the youth (or the volunteer). Exhibit A: Ricky Hendon's $20,000 tutor grant scandal that broke last month. http://www.chicagotribune.com/news/nationworld/chi-afterschooljul20,0,6218298.story and http://nicolecabrini.blogspot.com/2008/07/answering-20000-question.html

Therefore, to ensure the best outcomes, programs such as ours need to take advantage of this research and offer useful training for our volunteers, create an environment and program structure that fosters the formation of strong mentor-mentee relationships and increase parent involvement, all things we are working hard to address here at Cabrini Connections. For example, I just finished creating the student and volunteer orientation packets that were carefully designed to help volunteers not only maximize their impact at the beginning of the school year, as they meet with their mentee, but also to help give them ideas and tools to strengthen the mentor-mentee bond. Additionally, this year we are organizing our first ever Welcome Back Brunch, which will give mentors an additional opportunity to meet their youth mentee's parents before the school-year starts and engage with them in a friendly and comfortable atmosphere so they can begin to work together to maximize their child's potential!

As you can see just from this brief post about a pair of mentoring studies, being familiar with the relevant research can greatly assist an organization such as ours to offer the best possible mentoring programming for our youth. Therefore I think it is especially important as someone who has gained a familiarity with the way this work is done and reported, to summarize and share this knowledge with people and organizations who can benefit.

Anyway, with this in mind, I'm thinking about doing a regular feature in my blog about the importance of "Knowing the Research". Any thoughts? I'd love some feedback on whether people would find this sort of thing interesting. Also, I can try to make the writing even less technical if I'm losing people with jargon. I've never done this before, so any constructive criticism would be much appreciated!


(1)Tierney, J.P., Grossman, J.B., and Resch, N.L. (2000). Making a Difference: An Impact Study of Big Brothers/Big Sisters. Public/Private Ventures

2)DuBois, D.L., Holloway, B.E., Valentine, J.C., and Cooper, H. (2002). Effectiveness of Mentoring Programs for Youth: A Meta-Analytic Review. American Journal of Community Psychology, Vol. 30, No. 2.

No comments: