The problem of irreproducibility and the falling mask of (some) pseudo science

This week the “scientific world” (I’ll explain the quotes later on) was all abuzz with the publication, in Science, of the results of a group led by Brian Nosek that attempted to replicate 100 of the more salient experiments in the field of psychology (most of them in social psychology, a subfield that has been itself quite glamorous and attention grabbing in the last decade), and failed to do so in more than 60% of the cases. You can find the NYT take on here (Whoops, most psychology is make believe! ), the WaPo here (Jeepers, as replicable as the Horoscope!), the Economist (Hey! at least they are honest enough to recognize it publicly ) and Vox’s here (Add a sophisticated sounding word to your vocabulary: irreproducibility ). So it’s been kind of the big thing of the week, now that the Great China Slowdown seems to be receding from the public consciousness. Putting it charitably, if I can say I’m surprised at all is because they still found so many they could replicate (now when you go to the nitty gritty of what the studies purported to show you realize that they are pretty inconsequential, although in some cases the most fantastic extrapolations on how the mind works have been made using them as a fig leaf).

This resonates so much with how I think that it turns me a bit solipsistic: when the Universe corresponds so minutely with your preconceived ideas about it, it is difficult to avoid the suspicion that may be the only possible explanation is that it is just a product of your imagination. Just kidding, of course.

It is difficult to convey how much this assessment of the current state of the “scientific papers publishing industry” (that’s what most humanistic disciplines that aspire at a “scientific” status have effectively become) understates the magnitude of the problem, as it only looked at experiments and results published in the three most prestigious journals (prestige supposedly comes with being rigorous and exigent in the peer reviewing) of the field, and it had previously selected those that had both reached notoriety and… well, seemed comparatively easy to replicate. Had a random sample of articles in those same reviews been attempted, the “failure rate” would have been undoubtedly higher. Had the sample been widened to include more journals (necessarily with less cachet), the rate would have been (I’m guessing here, but “educatedly” so) astronomically higher. So I feel vindicated in my original belief that 90% of what passes for “psychological experimental science” today is a sad joke.

Not that I needed this confirmation, just reading the books written by their practitioners, and then checking the details of the original experiments they base them on should be more than enough to give any critical thinker pause. The most idiotic experiments are performed (make subjects inadvertently smell freshly baked bread and see if they report being more well inclined towards their fellow beings… crash rudely with them and see how that affects how friendly they seem to be towards an old lady having troubles with a copying machine… make them read a bloody paragraph about free will and measure if they are more prone to cheating in a subsequent exam! Really, you couldn’t make up all this idiocy), with tiny samples (between 20 and 30 subjects is considered normal) of amazingly homogeneous populations (it is not like the “researchers” work very hard to ensure the representativeness of their samples… 90% of experiments are done with students in the same universities where the experimenters work), which happen to be extremely poor representations of the whole of humanity (what has been called the WEIRD problem, as the subjects of western psychology experiments almost unfailingly come from a Western, Educated, Industrialized, Rich and Democratic background… which puts them in a tiny sliver of a minority of how most of the world lives today, or has lived for most of its history).

I traced the origin of that problem to none other than the notorious F, that from observations (and pretty edited ones at that) of a tiny bunch of Viennese Jewish wives of well-to-do liberal professionals towards the end of the XIX century purported to have discovered universal truths about how the mind of every single human being from the beginning of the species worked. He had a crush on his mom, held all his life a pissing contest with his dad (long after poor Jakob died and could piss no more young Sigmund was still hard at it), after telling his friends they were not totally taken aback by such revelations and, after being pressed enough, admitted to have maybe harbored similar feelings themselves, so, Presto! We could legitimately infer that  Old Gronk the Neanderthal had exactly the same feelings, and indeed they explain enormously why he painted all those bulls in Lascaux! And so did only slightly less old Schplink which most likely killed the old man and invented religion in the process!  And so did the Hebrew twelve tribes during the exodus (which repeated the killing taking not one Moses’ life, but two!) And, closer to our days, so did Leonardo da Vinci (who dreamed of vultures putting their feathers in his mouth, such transparent metaphor, the old fag! Only he never talked of vultures, but of small falcons, Ziggy just got his translation all jumbled, but that didn’t impede Pfister from “seeing” the shape of a vulture in the garments of the virgin of the rocks… never mind!), and so did that poor bastard Dostoievski, damned be his Slavic soul (and don’t give me that the guy was epileptic! It is all mental, psychological, he really was a hysteric!) Seen with some perspective, it is pretty silly to think that by projecting his prejudices (quite bigoted for our modern sensibilities) formed by observing that minimal bunch he could arrive at any truly universal knowledge… but that is exactly what he did, what he spent the rest of his life furiously defending. Oh, the stupendous amounts of vitriol in the attacks to whoever doubted the soundness of his methods! They were all small-minded, philistines, hypocrites and not clear-eyed enough; they clung to old fantasies like religion, nationalism, militarism, or any discredited philosophy; they were too simpleton, too unsubtle to appreciate the greatness of the scientific project that culminated in his theories.

And it is sad to report that for decades the educated world swallowed that drivel hook, line and sinker (well, Stalinism was the de facto only political truth for almost all of Continental Europe’s intelligentsia, so psychoanalysis was not the only kool aid the educated classes were busily consuming then). So it can’t come as a surprise that, even long after the abandonment of psychoanalysis the poor souls educated in psychology departments in Western universities cling to the idea that observing any casual behavior of a bunch of guys suspiciously similar to yourself is a perfectly valid application of the scientific method to confirm any ludicrous hypothesis you fancy to hold, as long as such hypothesis conforms to what the dominant reason tells you are the basic tenets of the only acceptable worldview:

·         The only substance that is really out there is matter

·         The only valid objects of knowledge are the (hopefully simple) laws that determine how matter evolves

·         There is no such thing as free will

·         There is no such thing as value, only pleasurable or displeasing states of mind

You would think that given those constraints the best thing to do for a honest, coherent student of psychology (or sociology, or politics, or economics for that matter) would be to claim for the closure of the corresponding department, to save unnecessary expenses to the public and spare other unfortunate souls like him (or her) the sheer waste of time. Not that this is an entirely novel idea, I humbly recognize that (half-jokingly, as they are wont to do) the guys at The Onion beat me to this by almost a year: Psych recognizes it is bogus and calls it quits

Of course, nothing of the sort will happen. Hapless boys and girls will continue attending to lectures where endless platitudes of dubious epistemic status will be hurled at them as if they were well established scientific facts. And in turn they will first participate as subjects (for some “beer money”) and afterwards design and carry out silly “experiments” themselves that will only be able to validate whatever theoretical construction they are supposed to buttress, without realizing that rather than expanding the landscape of solidly founded human knowledge all they are doing is adding more and more tautological constructions to an edifice (the dominant reason of the age) that admits of no such thing. By definition the dominant reason is impervious to empirical (or scientific) validation. Societies accept it because it happens to have some influence in how their members behave that gives them some advantage in an environment of inter-societal competition. Our current (desiderative) reason is not dominant because it somehow reflects better how things “really” are, but because it helped our society win two wars, first against fascism and afterwards against communism. And it did it by being better than the alternative models they used (one of which –communism– was an embodiment of the previously dominant reason in our society, the bureaucratic reason, and the other –fascism– of the one previous to that, the sentimental or romantic reason) at making its people (us) work longer hours and devote more of our energies to the production of material goods, material goods which in turn allowed us to have bigger, deadlier armies, better equipped, superior enough that in the end we didn’t even have to deploy them in the battlefield.

Now my most astute readers may have noticed that we do not inhabit a scenario of inter-societal competition any more, as there is a single, hegemonic, homogeneous world-system that is accepted as valid everywhere, from Beijing to New York, from Mogadishu to Canberra. There are some fringe elements within that single system that purportedly oppose it (Islamic extremist would be the most belligerent example), and some currents equally within the system that may show some minor discrepancies with the established theoretical discourse (both nominally communist China and crony Russia may defend their peculiarities as alternatives to the predominantly Western model), but I would argue that none of them is either extended enough or different enough as to imply a real threat to our (and theirs) current rationality. So, if that were the case, the whole edifice may come crumbling down, as its final justification is that it enables the societies espousing it to more effectively compete against other societies of similar might (as the production of more material goods showed itself to be a successful strategy against fascism and communism, but doesn’t seem to work all that well against either jihadism or softer versions of totalitarianism), so if there are none of the latter the former losses all meaning. Indeed. That’s why I’ve been announcing that the days of this flavor of dominant reason (which I call desiderative reason) are already numbered, and why I have started thinking in how it may evolve (that is, what the features of the successor rationality may be). What I can share is that there will most likely not be any place in the new rationality for the most cherished tenets of what today constitutes psychology. It will be abandoned, and future generations will look with a mixture of amusement and disdain towards what today passes as a “science”, much as we look to Aristotelian physics and smile thinking how could our forebears be so naïve, and believe that such amount of mumbo jumbo and gibberish was fit to keep adults engaged.