All the Standards Fit to Test

How do we justly divvy up the resources we have in the world?

If you’re a natural law theorist or neo-reactionary you’ll say by birthright. If you’re an Austrian economist you’ll say its based on hard work. Perhaps not so incidentally, there is a way to satisfy both camps. If good genes give one a good intelligence, then good work follows from those who are born “gifted”. In that case, meritocracy and nepotism end up saying the same thing.

How would one get evidence for such a claim?

A numerical assessment would be preferable. And it turns out the works of Spearman show that regardless of the subject matter, measurements tend to correlate the same way. A correlation with many different factors should be connected to a common factor relevant to all those different areas. Thus It was proposed that there was a general ability to evaluate evidence and come to valid conclusions across a broad range of fields. If true, this means test results could be standardized across different environments. And if that is true, then we would want to know about such an ability.

But what is that common factor? How to determine it?

One could try to build up to complex behavior from simpler ones, but this may imply emergent properties, which can be problematic (see section H here). On the other hand, instead of looking for complex measures for a complex of traits, one could try a direct measure for a single complex.

Enter IQ tests. The results of which highly correlate with other factors such as careers:


Robust correlations like that of IQ and career point towards a real common factor being measured. And since 1994 there has been a scientific wing arguing that “IQ is strongly related, probably more so than any other single measurable human trait, to many important educational, occupational, economic, and social outcomes … [therefore] whatever IQ tests measure, it is of great practical and social importance”.

This being so, a true instrumentalist doesn’t care whether what’s being measured is a hive mind ability to play well with others or any other form of general acumen. The true instrumentalist says “epicycles be damned”, and focusing on empirical data proudly declares “no one cares how the testing correlates to a general intelligence, so long as it does.”


But the nature of correlations is that many factors will relate to one another. This raises another question: are the tests reducing intelligence down to the correct factor?

Proponents of IQ say: of course, it is a general factor, it should reduce down to all areas.

On the other hand, I (maybe all of us) can think of at least one person who commands a high intellectual ability, ie. “is book smart”, but has a deficit in conducting more general mundane abilities, ie. “has no street smarts”. A good theory of intelligence should allow for both polymaths and idiot savants. How else do you explain them?

It is at least plausible that there are multiple forms of intelligence. As far as I’ve seen, no one person has a monopoly on mastering all things. Polymath is just a sesquipedalian synonym for jack of all trades which is identical to master of none. As per Jerry Fodor:

“If you think there’s such a thing as General Intelligence, which is what IQ tests are supposed to measure, then you should also think that designing bridges and designing foreign policies manifest much the same kind of cleverness, albeit applied to different tasks….Whereas, if you’re on the [other] side, you won’t be surprised to find every sort of intellectual sophistication cohabitating with every sort of naivety, and will be disinclined to trust the obiter dicta of experts.”

Now, if we admit to needing multiple skill sets, then we can no longer continue by measuring one thing, however encompassing. This is an old criticism:

“Most psychometricians agreed with Thurstone that Spearman had been on the wrong track when he postulated a single common factor of intelligence.” – Peter Schönemann

It is also borne out with multiple theories that need to include multiple intellectual abilities to explain data. Marvin Minsky thought that our intelligence manifested thus:

“We get resourcefulness from having many resources, not from having one very smart one.”

If it is true that we rely on several different abilities, then it would seem counter-intuitive to reduce intelligence down to one thing. This was Stephen Jay Gould’s point when he pointed out that reification of abstract concepts into concrete entities is fallacious. Or put in simpler terms, as seen here in Interstellar:

Cooper: You’re ruling my son out for college now? The kid’s 15.

Principal: Tom’s score simply isn’t high enough.

Cooper: What’s your waistline? 32? With, what, a 33 inseam?

Principal: I’m not sure I see what you’re getting at.

Cooper: You’re telling me it takes two numbers to measure your own ass but only one to measure my son’s future?

So a new problem arises: if we rely on not one general intelligence but on several, which one should we lean most heavily on when assessing? Which intelligence do we pick?

If you’re an acolyte for standardized tests, then you’ll put a lot of trust into the manipulation of words and numbers, assuming they work as a proxy for surviving and thriving.

But then what about neurodiverse persons? Asperger thought autistic persons would be superior code breakers. Temple Grandin has done well and, arguably, so has Bill Gates. So it would appear that being knowledgeable in the conventional testable sense is not a good predictor for success or innovation.

Even in the abstract such tests seem insufficient. What about the boy who said the emperor has no clothes? That child was demonstrating an ability for change and insight that created new mutual knowledge. That’s not the type of person who would have scored high on IQ tests, either directly by circling boxes on a scantron, or indirectly via something like the Marshmallow Test.

There is also the corollary question of why “being smart” is restricted to impersonal knowledge as opposed to interpersonal knowledge. What about abilities for friendship or parenting or managing emotions? After all, those are the things we diagnose as pathological when missing or distorted. What about life lessons learned over time? Doesn’t wisdom come with age and experience?

Instead, we currently fetishisize the isolation of language and math as the desirable skills. There is a lack of any interest whatsoever in extralinguistic areas concurrent with fideism in language. This logocentrism is why IQ proponents and PoMo subscribers that allow for multiple scales of intelligence (as seen here and here and here) do not get along. In short, put eloquently by Virginia Woolf:

“[Moralists] are as much out in their scorn as we should be if we asked of the lily that it should be cast in bronze, or of the daisy that it should have petals of imperishable enamel.”

Or Einstein in a less baroque statement:

“I want to oppose the idea that the school has to teach directly that special knowledge and those accomplishments which one has to use later directly in life. The demands of life are much too manifold to let such a specialized training in school appear possible.”


The whole discussion so far rides, btw, on the assumption that the findings measured are valid. So we need to ask about their validity.

One problem to be accounted for is the changing heritability as one ages, ie. the Wilson effect. Intuitively, this should run counter to nature in the nature/nurture debate.

However, one could also argue that there is an amplification effect of genes as one ages. So if you’re Judith Harris, genetics trump parenting. Society matters lil’ or nil.

Such proclamations have not gone uttered without response. But for the sake of argument, let’s assume Harris’s side is correct. Even if genetics have a dominant role through amplification, those same genetics can be modified as well (lest we forget about epigenetics). As expressed here:

“Genetics is not fixed. Having a gene may be fixed, but whether you express this gene or not is most certainly under outside control. Consider gender. Absolutely genetic, correct? Not much one can do about it? But lizards can alter the sex of the progeny by changing the incubation temperature of the egg. Think about this. Now, is it not probable that the expression of the genes for [mental disorders] have a lot to do with how you are raised? We already know that environment affects gene expression, so I’m not speculating here.”

This admits to a modifiable intelligence, even if we grant genetic and IQ axioms. After all, our nature had to be modified at some point to make a phylogenetic tree, right?:

“Before such things can come/ you idiotic child

You must alter human nature!”/ and they all sat back and smiled.

Thought they, “an answer to that last/ it will be hard to find!”

It was a clinching argument/ to the Neolithic mind!”

– Charlotte Perkins Gilman

The validity of having a modifiable effect on intelligence over time has been shown by James Flynn. In the Flynn Effect, the average IQ measured between 1916 and 2016 went from 70 to 136. As he puts it:

“This deals a stunning blow to our confidence in the ability of IQ tests to compare groups for intelligence, at least when those groups are separated by cultural distance. Can anyone take seriously the notion that the generation born in 1937 was that much more intelligent than the generation born in 1907, to say nothing of the generation born in 1877? – James Flynn

Such a disparity means that either IQ is a bad measure (it labeled an entire generation as intellectual deficient) or IQ is a mutable measure (and makes measuring it a moot point).

When a factor becomes as pliable as this, to claim any sort of genetic essentialism is ridiculous. Logical parsimony dictates that a more likely explanation for higher general intelligence is as a product of higher social factors such as socioeconomic status or quality of life. As such:

“Every new study about social mobility further corrodes the story Americans tell themselves about meritocracy; each one provides more evidence that comfortable lives are reserved for the winners of what sociologists call the birth lottery. What appears to matter, a lot, is environment, and that’s something that can be controlled.”


Given the above, why then does this reasoning persist? Why generate such stupid questions and their ilk in pursuit of a will-o-the-wisp?

If you look at the history of this pursuit it will become apparent. The compiling of the statistics and the hereditary transmission of traits of intelligence was done by eugenicists. Its application was then used to assess military applicants for fitness during World War I. Those same principles also guided the crafting of the SAT assessment in 1926, with the same author stating: “American intelligence is declining, and will proceed with an accelerating rate as the racial admixture becomes more and more extensive.”

Needless to say, there’s no good justification behind this. There is much more variance among individuals in any group than there is between groups:


And the recommendations in the field most assuredly have a fair share of detractors.

I also hope it’s clear that thinking this way is sipping the status quo kool-aid. This is anything but a form of progress. This has been the point since ancient Greece:

“From the point of view of totalitarian ethics, from the point of view of collective utility, Plato’s theory of justice is perfectly correct. To keep one’s place is a virtue.” – Karl Popper

The motivation running contrary to this ancient morality, viz. liberalism, can be put thus:

“[Liberalism is] not a philosophical dogma but the plausible empirical claim that there will always be a gap between people with hope and people without it, people who know they have a chance in life and people who know they do not.” – Richard Rorty

For now, standardized testing of ability is a social idea that implies someone’s value. This use of testing means that:

“[People] are just numbers and the numerical becomes their horizon” – Søren Kierkegaard

Once given a label, a person’s possibilities for the future are reduced. Within the status quo, it provides post-hoc justification for things like hiring decisions. Then a vicious feedback occurs: the more IQ-like exams are used as screening devices for occupational access, the stronger the relationship between IQ and income. The relationship tightens and justifies itself. IQ measures test taking ability more than skill, and becomes a way for status groups to protect their domains. Plus ça change, plus c’est même chose.


This problem was foreseen by some prescient minds:

“Test results and numerical tables are further accumulated; consequent action affecting the welfare of persons are proposed, and even taken, on the grounds of – nobody knows what!” – Charles Spearman

And we’re left with a process that justifies one’s own biases. This is how the serpent devours its own tail.

We could reverse this. We could pay less attention to the ease of scoring scantrons brought on by standardization, and more to the fact that for long range criteria (ie. graduation) the utility of standardized tests such as the GRE are virtually zero. But I doubt such a change will happen. On our current trajectory, the following is more likely:

“Democracy will be replaced by something quite different. This will be neither military dictatorship nor Orwellian totalitarianism, but rather a relatively benevolent despotism imposed by what would gradually become a hereditary nomenklatura… Careers will be less open to talents and more open to connections with powerful persons… In short a return to something like the ancien regime.” – Richard Rorty

You still may not agree. You may think this is ranting and raving, and just being obstinate and dogmatic in the face of plausible scientific findings. That may be. But guess what? This is what the guy who is not hiring you is also doing. They use the scores as a means to justify their hires, unless it doesn’t suit their purposes and then suddenly what matters is being “well rounded” and “someone who thinks outside the box.”

Lest you think I’m prophesizing the end is nigh on a streetcorner in a paranoid state (even though I’ve had to say “told you so” before), let me put it this way. Ask yourself how frighteningly realistic does the following scenario seem:

“[Employers] look for reliable proxies of learning like race, gender, and physical appearance. Oh. Did you assume employers would be more influenced by grades than their own personal prejudices? “Wait a second, I graduated 4.0 from State, and the guy you hired had a 3.2 from State– the only reason you didn’t hire me is because I’m a woman!” Ok, this is going to sound really, really weird:  yeah.”

Rather than rant against those ranting, why not ask yourself: who is going to test the testers?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s