Hello intrepid….readers. When last we spoke, I was teaching you all how to approach things that look like science but are obviously marketed for a massive audience. Can a Facebook quiz really tell you which side of your brain is more dominant? Hint: no. In short, I talked at you about approaching pseudoscience with skepticism that would make Dana Scully proud. But now, we’re going to dig deeper into why that quiz is an exemplary pile of fluff. In this segment, I’m going to use it to soapbox about the murky waters of psychological testing. In the next {and final, promise!} part, we’ll actually discuss left/right hemispheres and what those really mean and do.
Any sort of test claiming to have a degree of significance in its results has to be tested, retested, validated, and generally stand up to damn rigorous scrutiny. Down to how the questions are phrased, even. The amount of work that goes into developing psychological assessments is astounding and a ton of background research and testing needs to occur before you can use it to find any sort of conclusive result. And even then, the results are very nuanced and are understood to be taken as general indicators rather than end-all, be-alls on the matter.
For example, IQ testing doesn’t actually tell you how smart you are or how successful you’ll be. It tells you how you compare to others fitting roughly the same demographic as you in specific areas like working memory or verbal comprehension, which are generally lumped together to form a vague definition of “intelligence” or, more accurately, specific areas of cognitive ability. IQ tests are also subject to a lot of cultural bias and are so far from perfect it’s not even funny. This isn’t the place for me to rant but sometimes Psychology Today has some good places to start {with more sources, so I don’t have to} and I generally find this article to be a decent jumping off point to explore the matter further.
Perhaps at a later time I’ll dive into “intelligence” and rant about how messed up our perception is of it and how much false value is placed on doing well on standardized tests. Long story short, no human’s abilities can be boiled down into numbers and compared to a norm and give a complete picture. Having a high IQ means you’re more or less really good at the type of tasks in the IQ test but say nothing about the rest of you. In fact, multiple studies have found that IQ is a worse predictor of things like academic achievement as compared to, say, self-discipline. TL;DR on that link: in this fairly decent study, IQ appears to be a worse predictor of academic achievement than self-discipline as demonstrated by a few different analyses and measures of academic achievement and self-discipline. Neurotopical: reading, so you don’t have to {unless you really want to}.
Another issue in psychometric assessments is the fact that there are SO MANY sources of unreliability. New tests have to be researched and tested against similar batteries, validated in multiple studies, scrutinized, torn apart, critiqued, updated, and basically put through hell before they can be used by researchers with the caveat of “this test is probably indicative of this quality.” Interestingly enough, some researchers from Ohio State actually developed a pretty damn accurate measurement of narcissism…which is only one question. It’s called SINS, or the Single Item Narcissism Scale. Love it. As with all tests, factors like inaccuracy in self-reporting, differences in administration and scoring between researchers, cultural bias, aaand honestly this sample of a book chapter breaks it down better and in more depth if you actually care…all add up and are very unavoidable.
Luckily, statistical analysis takes some of this into account but we cannot have a completely accurate, quantifiable measurement of any one quality so always take such things with a grain of salt. MBTI, I’m looking at you {and specifically not going to address that here…if you’re curious as to what my take is on the MBTI, scroll up, read the first few paragraphs again, then imagine me raising my eyebrows and giving you a skeptical look. If you have no idea what I actually look like, imagine someone giving you a skeptical look. Guys, I’m an INTP/ Taurus/ Gemini cusp, I can’t help that I’m like this…whatever “this” is. All I know is none of those categorizations say anything meaningful about me on a scientific level, just sayin’. You’re welcome to have your beliefs but please don’t confuse them with my science.}
In short, hopefully I’m making you a paranoid freak who is going to question every number and measurement they see from now on, especially as it applies to psychological testing. But also, recognize that it’s often the best we have and overall, really well-developed psychological assessments do tend to at least tell us something about the person being evaluated and point us in a general direction. Assuming the participant is not lying. In the words of Dr. House, everybody lies. But some tests actually have questions designed to catch lies or exaggerations but I’m actively not going to tell you too much about those so that our clever ways of catching dishonesty is not revealed to the world.
So back to the original point….if our gold standard psychological tests don’t really tell us anything conclusive about what we’d expect said tests to reveal, what snowball’s chance in hell would a Facebook fluff quiz website have in producing any sort of meaningful result about your brain {or any brain, really}? Unsubstantiated metrics will often be wildly inaccurate and inconclusive, end of story.
Do you want me to stop beating this dead horse of a Facebook quiz over the head or nah….? No? Cool, then stay tuned for the final installment of vaguely related tangents where we finally get to the good stuff, ie. discussing what the whole left brain/ right brain deal is. Thanks for reading!