The USNWR rankings are especially insidious because they're also not objective.
I wish I could find the citation for this, but I recall reading about how when it first started the person who created the ranking formula showed USNWR the list of top 10 schools and the Ivy League schools didn't do as well as expected. If HYP weren't at the top there was a flaw with the methodology, so they had to go back and get a new methodology that would conform with conventional wisdom.
Also back in '99 Caltech took the top spot. Again this was viewed with some disdain from the East Coast elite. They changed their methodology and added/emphasized a metric they call "value add", which is the difference between actual and expected graduation rates. Because CalTech is difficult to graduate from, they were seriously penalized.
IOW, rankings are often simply a way to perpetuate the status quo, but make it look objective.
Some mathematicians did a reanalysis a few years ago where they investigated how sensitive the rankings were to the particular choice of weightings. They found that the answer is "very sensitive", to the extent that you can produce a wide range of desired outcomes by carefully choosing the weights to assign to each criterion: http://www.sciencenews.org/view/generic/id/37207/title/Math_...
I think your broad point about "elite" schools being a matter of reputation rather than objective measurement is basically right. But some of your analysis seems weird.
On the graduation rate: why shouldn't a school be penalized for a low graduation rate? The point of those rankings (beyond selling magazines, of course) is to give prospective students an idea of how each school will help them in life. If I have a choice of going to school A from which I'm 90% likely to graduate and an otherwise equally ranked school B which is 50% likely to kick me out without a diploma, which am I going to choose?
And the bit about CalTech seems a little like sour grapes. I was at one of those "East Coast elite" schools in the 90's and don't remember anyone thinking anything but good things about CalTech (Pasadena, on the other hand...). Basically within its fairly narrow realm of focus CT is absolutely one of the "elite" schools everyone looks to.
"If I have a choice of going to school A from which I'm 90% likely to graduate and an otherwise equally ranked school B which is 50% likely to kick me out without a diploma, which am I going to choose?"
The "otherwise equally ranked school" part shows where a single-dimension ranking system falls on its face. Does it mean they accept the same quality of applicant? If so, graduating from the school with the lower graduation rate might be a much more impressive accomplishment. Or does it mean that the quality of education graduates receive is considered to be basically the same? Does that then mean the second one is just easier to get into?
How would all colleges having high graduation rates avoid pigeon-holing people even further into the box that their high-school performance marked them for? If everybody can graduate, the potential signaling is reduced from "got into and graduated from school x" to just "got in to school x."
> On the graduation rate: why shouldn't a school be penalized for a low graduation rate? The point of those rankings (beyond selling magazines, of course) is to give prospective students an idea of how each school will help them in life.
Its easy to look at this from the other side though.
I think a lot of people look at these rankings as the value of a degree earned from that institution. That is, they are looking at it from the side of the alums as opposed to the prospects. Suppose two schools have the same admissions criteria and everything else, but the graduation rate of one is much higher. I would think the school with the lower graduation rate would be viewed as more rigorous and therefore "better".
Even in high school, I never thought differently. >10 years later this is the first time I ever even thought that someone would consider the probability of their graduating (totally within their control). It was always 100% which would be the best school to be from...or which one would be the most fun. Graduation rate has nothing to do with either.
IMHO what weights are the right weights is very subjective - reducing all these colleges to a single comparison is stupid.
For example, if there is 100% graduation, I consider that very suspect and probably a bad sign. It's natural that some people don't finish college. WRT Caltech, they are desirable BECAUSE they offer such a challenging environment.
These rankings tend to not be objective, I agree. The sad part is they get away with it by simply defending their methodology of choice (emphasising certain areas).
"If HYP weren't at the top there was a flaw with the methodology, so they had to go back and get a new methodology that would conform with conventional wisdom."
Most importantly, I think it's important to understand that if HYP (and others in the Ivy League) weren't in the ranking nobody would take the ranking seriously.
So you have to have a smattering at least of what people are expecting to see or you don't have a list worth considering and reading.
If you are going to have a list of anything, in order to elevate lesser known members of the list, you have to have credible proof of known entities that everyone can relate to.
For example take the Rolling Stone list of the 100 greatest acts of all time. Particularly the top 10:
I wish I could find the citation for this, but I recall reading about how when it first started the person who created the ranking formula showed USNWR the list of top 10 schools and the Ivy League schools didn't do as well as expected. If HYP weren't at the top there was a flaw with the methodology, so they had to go back and get a new methodology that would conform with conventional wisdom.
Also back in '99 Caltech took the top spot. Again this was viewed with some disdain from the East Coast elite. They changed their methodology and added/emphasized a metric they call "value add", which is the difference between actual and expected graduation rates. Because CalTech is difficult to graduate from, they were seriously penalized.
IOW, rankings are often simply a way to perpetuate the status quo, but make it look objective.