Knowing the Research: Challenges in Educational Research

Building off of my last post, which discussed some possible ideas for new tutoring/mentoring research, I just wanted to discuss one important aspect of this mentoring research that I have yet to address. It is a methodological issue common to all disciplines utilizing experimental studies, but it is particularly important in education research. It is the issue of random assignment.

For those who aren't so scientifically inclined, one basic tenet of an experimental study is the inclusion of a control group, or a group of people who don't receive whatever the independent variable being tested is (in this case, enrollment in a tutor/mentor program), but are measured nonetheless to ensure that whatever gains observed in the other group (the experimental group) are the result of the particular intervention (or independent variable) and not some other factor. For example, if a group of students' math abilities are measured immediately before enrolling here at Cabrini Connections and then again after spending one year with the program, it is likely that they will show improvement. However, that doesn't necessarily mean that they improved BECAUSE they enrolled in our program, just that they improved WHILE they were enrolled in our program. The improvement might well have been caused by the students math curriculum in the schools, playing a computer game or even playing dice. We simply don't know if we only measure one group. However, in a controlled experiment we would have 2 groups of kids, one group that enrolls in Cabrini Connections for a year and another who remains uninvolved in any tutor/mentor program. Then, both groups are tested before and after spending a year in the program. Now, we can reliably compare the 2 groups, provided that they come from the same population of kids growing up in Cabrini Green and are more or less matched on variables like socioeconomic status, age, sex...etc, and see if the kids enrolled in the tutor/mentor program showed SIGNIFICANTLY MORE improvement in their math scores than others BECAUSE they were enrolled in Cabrini Connections.

This issue is a constant problem in educational research and studies like the aforementioned ones becuase of the difficulty in what is known as random assignment of participants. This is to say that, because most of the time the researchers running the studies cannot get a group of participants from one population and randomly select half of them to enroll in a given tutor/mentor program and the other half to continue living their lives exactly the same way as before. When researchers can actually do this, their experiments are called "Randomized Field Trials" or RFTs. Use of RFTs is increasing in educational research and is a hotly debated topic, as they often require more time and money to carry out. However, as I mentioned earlier, they do allow the researchers to draw conclusions about the effectiveness of a particular intervention in terms of causal relationships (i.e. enrolling at cabrini connections caused my math scores to improve). This is very important because it is not of much value to tutor/mentor program adminstrators, educators or concerned parents to draw the conclusion that children who participated in one particular type of program improved in certain areas, but that improvement could very well be due to other factors having nothing to do with their involvement with a tutor/mentor program. Using RFTs allow us to make concrete and useful conclusions, backed by scientific evidence.

Unfortunately, much of the research addressing the effectiveness of tutor/mentor programs does not utilize Randomized Field Trials, and thus fails to really tell us much of anything that can be of significant benefit in determining the best practices that should be used for our program. Therefore, I am of the opinion that to maximize the applicability of future research, RFTs should be used wherever feasible. Educators? What do you think? I'd be interested to hear some perspectives from veteran teachers that are actually in the schools where the majority of these RFTs are taking place and not just sitting in a desk all day at Tutor/Mentor Connection like myself.

hasta la proxima
Chris