An interesting Twitter thread by Julian Wolfson on the implications of removing the GRE from the grad school admissions process:

Grad school application season is just around the corner. So, it’s time for a thread on #grexit. Spoiler: I’m on the fence. Here’s why. 1/

First, let’s limit the conversation: I want to talk about #grexit for PhD admissions in (bio)statistics. Why just PhD? Admit rates are (much) lower, and the “financial barrier” argument for #grexit is more relevant for a fully-funded program. 2/

Next, my experience: I am DGS and sit on the admissions committee at @umnbiostat. We receive ~180 PhD applicants each year, and make ~25 first-round offers. All members of the adcom score every PhD applicant; there are no “automatic rejects” based on grades or test scores. 3/

The GRE is one of several components of the application we look at. I tend to put more weight on grades, letters, and GRE scores, and less on research/personal statements. Each committee member has her/his own internal rubric. 4/

I’ve heard several arguments in favor of #grexit, some of which are compelling and others which I don’t find persuasive. 5/

Before we go through them, hopefully we can agree that we all want to answer the following question: What effect would #grexit have on graduate admissions? This is a causal question, and so observational data are going to have to be interpreted cautiously in this context. 6/

Argument 1: The GRE is expensive, and discourages less well-off students from applying. I view this as the most compelling argument for #grexit. It is a barrier for some, but I don’t think we know how big an effect this will have on application numbers and pool diversity. 7/

Argument 2: The GRE is biased against groups which are traditionally underrepresented in grad school. I’m not completely convinced by the available evidence, but in any case these groups face many barriers getting to grad school, most of which are not addressed by #grexit. 8/

In fact, I worry that #grexit will have unintended negative consequences. For some students, a good GRE score is a key factor supporting their admissions case. Absent the GRE, adcoms may rely more on grades (school “reputation”) and letters which have their own biases. 9/

Argument 3: The GRE doesn’t predict success in grad school. Yes, I’ve seen the studies. No, I’m not convinced. All these studies have shown a lack of correlation conditional on being admitted to grad school. 10/

In causal/DAG language, being admitted to grad school is a collider, and conditioning on a collider leads to bias in estimation of (causal) effects. This is a well-known phenomenon called Berkson’s paradox:… 11/

NOTE: This doesn’t imply that the GRE is useful, only that the observed lack of correlation between GRE and “success” among admitted students may mislead us into concluding that the GRE is a useless data point for determining who is more likely to be successful if admitted. 12/

NOTE 2: There is a whole other discussion to be had about how we define “success” in grad school and whether that needs to change. But this thread is already really long. 13/

So, where does this leave us? I’d like to see more experimental evidence supporting #grexit. 14/

One thing I’d like to see us try at @umnbiostat is to hide applicant GRE scores from some reviewers and make it available to others, and see how much scores differ. This would provide a more reliable answer to at least one aspect of the causal question in 6. 15/

Even then, we need to be cautious, because the GRE may have a small impact on reviewer scores overall but be highly influential for “borderline” applicants who require the most attention in admissions. 16/

A plea: If your program/school is going #grexit, take some time to develop a data collection plan to assess how #grexit affects your admissions process. If you are considering #grexit, think about how you might do a pilot study to test the effects of #grexit on a small scale. 17/

Overall, I think it’s great that the academic community is having the #grexit discussion, and we need to respect both those who support #grexit and those who remain skeptical. end/

Max Masnick, PhD @max

© Max Masnick. Views expressed here are mine alone.