Let’s suppose you and 7 friends were sitting around a conference table in front of a box with roughly 150 applications in it. Each application is over 20 pages long including letters of recommendation, transcripts, personal statements, and at least 5 pages of useless information required by the graduate school. You have only 24 hours in a day and only 30 days to make your decisions. Everything else in your life continues exactly as is has previously. How would you go about deciding which 30 were your favorite?
That’s essentially how our admissions committee started things this year. The four graduate students on the committee decided to each review at least half of the applications, so every application got looked at a minimum of twice. The faculty generally started at the top of the physics GRE scores and worked their way down the list. After meticulously studying every aspect of about two dozen applications, it became very clear to me that there were some very good indicators of just how much time I really needed to spend on any given application. Here are a few ways to get to the top of the pile:
- Physics GRE percentile over 75
- Math and physics GPA of over 3.9
- Significant research experience with a great letter from your research adviser
- Physics GRE percentile under 15
- Multiple general GRE percentiles under 50
- GPA hovering right around 3.0
After it was all said and done, we had read all the applications, crunched the numbers, pondered the wording on letters of recommendation, and made the decisions on who got it and who didn’t, we went back and crunched the actual numbers to see if we could find some statistic that could differentiate between those admitted and those not. Ideally if we plotted this magic statistic on the x axis and the probability of admission on the y axis we would get a step function - everyone below some cut-off would be out and everyone above the cut-off would be in.
It actually turned out this time that the closest thing to a magic statistic was:
If you came out above some cut-off value, you had a roughly 70% chance of being admitted. If you came out below the cut-off you had something like a 4% chance of admission. It’s not a perfect measure, but it’s pretty darn good. More technically, it is necessary but not sufficient to have these quantitative measures on your side. The numbers really do matter. GPA and GRE scores really do measure in some way the overall quality of an application.
1. I think the thing I found most interesting is the weight you are putting on the quantitative part of the general/non-physics GRE. I've always been under the impression that the general GRE is largely ignored.
ReplyDelete2. "GRE scores really do measure in some way the overall quality of an application"
I will agree with quality of the application but I continue to see the correlation between how well you do on standardized tests and how well you do research is not that great.
I understand it is hard to replace standardized tests with anything objective but I want to see the best researchers admitted, not the best test takers.
3. I think when the only objective numbers in front of you are mainly standardized test scores people forget how poor they predict ones ability to do research. People become consumed with the only hard data they have at their disposal.
By the way Nick, I know it isn't you, this is just how graduate admissions committees operate.
ReplyDeleteBut as I've said before: I find it so ironic that scientists pride themselves in being data driven and yet their top criteria used for selecting the next generation of scientists have no data supporting it!
Example, this is what a study with a press release by Cornell concludes: Study of Graduate Record Exam [IE: GRE] shows it does little to predict graduate school success.
graduate school success
Joe,
ReplyDelete"People become consumed with the only hard data they have at their disposal."
I would agree that the fundamental challenge for any admissions committee is how to admit students who will become successful PhD scientists. I don't know how to do that but what I am trying to say here is that if you take a more complete review of the application including personal statements, letters of recommendation, bachelors-institution, tai-chi, etc., you can predict those results to some degree of accuracy by looking simply at numbers.
The hard data may not predict success in grad school, but it does predict admission just as well as, if not better than, all the subject stuff in the application process.
Nick,
ReplyDeleteWhat, tai-chi isn't important! :) No wonder I didn't get into CalTech.
By the way, despite my comments I'm sure you guys are doing a great job. (I don't want to send the wrong message.)
Let me also say that predicting future success in any field is extremely difficult. Large businesses can't do it. The National Football League can't do it. On some level this is the holy grail of human resources - how do you create a tool that predicts future performance? If you have any suggestions I am all ears.
ReplyDeleteI don't know Nick, I think the NFL did a pretty good job with Ryan Leaf. :)
ReplyDeleteAnd plus, if you use the helpful tools provided by E-Trade it is impossible to not get good predictions about the market so maybe admissions committees should partner up. :)
Nick, I was wondering what the threshold value was for your magic statistic M?
ReplyDeleteI recall someone telling me about a certain university (I want to say Santa Barbra but don't quote me on that) that used a simple rule to sort out whether or not to even look at an application. Their magic statistic was:
M = GPA - 3.0 + Physics GRE%
If M > 1.2 then they would look at your application and actually read your letters of recommendation, personal statement etc. If M < 1.2 then your application would be set aside and would not be considered.
I don't know specifically what criteria UNC uses, but I do know that it does NOT involve the general GRE in any way. When I applied here, got accepted, and came here, in September after classes started I got a call from the graduate office (not the department offices) saying that they had never received my general GRE scores (they had my physics scores). I checked with the department and they had never received my general scores either. So I had to quickly request that my scores be sent to the graduate office, just so they could put an "X" in their excel spreadsheet, or they were threatening to throw me out. I asked the department if they needed a copy as well and the secretary just said, "Meh." (literally)
Would they ever let a looser with a 3.something but a good sense of humor in? You know... for comic relief maybe? Don't dash my hopes to pieces! The NHL has finally given up on me, don't you send me packing too! I mean really, all those great discoveries by great scientists in the past were all pretty much by accident right? According to Gary Larsen, it was Einstein's secretary that came up with E=MC**2. So all this GRE, research experience etc... doesn't really mean that much. You need a good idea man. Someone who can say "Hey, lets build a time machine." or "Hey, cold fusion isn't really dead." So when do I start? =:)
ReplyDeleteQuantum,
ReplyDeleteI want to re-emphasize that our magic "M" was computed after the fact - we did not actually use it in evaluating applications. It just happened that M was the best predictor of admission of the various statistics we computed after the fact.
For us the magic value of M was 0.7, which is close to the 0.6 value when you consider that most (but not all) potential physics grad students score in the 80-90% range on the quantitative section of the general GRE.
Also, we definitely used the general GRE scored in evaluating applications. We've found through anecdotal evidence in our department and a slightly more systematic study Ohio State's physics department that scored below the 50th percentile spell trouble. That should go without saying for the math section, but it's also generally true for the verbal and writing sections. You need more than math to get a PhD.
ReplyDeleteNote the irony in the fact that my last comment had at least 3 typos while discussing the importance of verbal abilities.
ReplyDelete"I will agree with quality of the application but I continue to see the correlation between how well you do on standardized tests and how well you do research is not that great."
ReplyDeleteThis is my conjecture as well. I think I'm probably biased because, while I got above the 75th percentile on my GRE, I did not get an 800. Generally, like Joseph, I think Standardized tests are not a good measure of how successful one will be.
Personally, I think a lot of information about an individual can be gleaned from their writing. And a good chunk of being a good research scientist/engineer is being a good writer. I would likely place less emphasis on numbers and more on personal writing. As a result, if I prepared an application I would have applicants write several essays to demonstrate their abilities.
jmb275,
ReplyDelete"...if I prepared an application I would have applicants write several essays to demonstrate their abilities."
I will post on this topic in the near future, but for now let me say that the least helpful part of an admissions application is the "personal statement". Additionally, the personal statement is the component most open for outside influence - it's tough for a parent or adviser to impact an applicant's GRE score, but they can all but write the personal statement. In all but a few cases I found it to be almost completely useless.