How Do You Think?

Out to dinner recently, a friend and I were discussing an organization whose name implies one thing, when in actuality, what they promote is entirely the opposite.  We both racked our brains to come up with the name of that organization with no success.   Days later, without any recent thought of the elusive name – the words Discovery Institute sprung forward in my mind.  It was a spontaneous and surprising recall that brought me relief and pleasure.  “Ah Ha!  That’s what we were trying to remember the other night.  Yes!” I said to myself.   These types of memories are called Mind Pops.

They are also referred to as involuntary semantic memories.  As was the case in my example, they are completely involuntary in that this type of recall occurs without any current conscious, active thought.  In the more scholarly term (involuntary semantic memories), the word semantic suggests that the relevant recall springs forth from one’s semantic knowledge – for example, most commonly the item recalled is a word, phrase, image, melody, or a proper name that one has learned or has previously been exposed to.  These recall events pop into conscious thought  (i.e.,  your “mind“), without current conscious active pursuit – thus the origin of the more compelling descriptor Mind Pops.

These memory events are a relatively new topic of research revealing, as was the case in my example, that such events are not always truly random.  Although the memory may be irrelevant at the exact moment that it pops into awareness, they usually are linked to one’s past experiences.  Sometimes they occur with no conscious awareness of the the trigger itself.   In my example, there was an event that consciously set the stage for my Mind Pop (i.e., striving to recall the Discovery Institute), but some Mind Pops are more mysterious.

Kvavilashvili and her colleague George Mandler, propose that “the completely out of the blue” Mind Pops are often explained by “long-term priming.” Priming itself is an interesting topic, but essentially it is a phenomena whereby your behavior can be altered by exposure to stimuli that enters your unconscious (implicit) memory.  Research has demonstrated that people can be primed to be more polite and patient if unwittingly exposed to words in an unrelated task that lists concepts associated with being polite and patient.   People will walk more slowly if they are implicitly primed with words associated with the elderly.  Furthermore, recall of trivia is better if people are asked to think about the role of being a college professor before being asked the trivia questions relative to folks asked to first think about being a soccer hooligan (with other variables held constant).

This unconscious priming sets the stage for these mysterious out of the blue Mind Pops.  Subconscious exposure to an image, a word, a song, or a scene serves as the trigger for later Popping.  As the word subconscious implies, the exposure occurs completely outside of conscious awareness.  When Kvavilashvili and Mandler asked subjects to journal their Mind Pops, there were numerous examples where the Pops had no clear, or very subtle, triggers.   “Most of the information we encounter on a daily basis activates certain representations in the mind,” Kvavilashvili explains. “If you go past a fish and chips shop, not only the concept of fish may get activated but lots of things related to fish, and they may stay activated for a certain amount of time—for hours or even days. Later on, other things in the environment may trigger these already active concepts, which have the feeling of coming out of nowhere.”  Kvavilashvili noted that “I got curious about [Mind Pops] because they seemed so random and out of the blue, but these mind pops are genuine fragments of knowledge about the world. What it shows us is that our subconscious often knows the meaning of an experience, even if consciously we don’t.”

Researchers like Dr. Lia Kvavilashvili are finding that Mind Pops are quite common.  I’m sure that  you have likely experienced such events yourself.  Kvavilashvili suggests that they are most often words or phrases rather than images or sounds and that they usually occur in the midst of some routine activity such as engaging self care.  In other words, they are most likely to occur when your mind is not focused on the task at hand and is free to wander.  A variant of this phenomena is the Tip of the Tongue (TOT) experience – where you may be struggling to remember a name or a word and it feels as though it is right on the tip of your tongue; yet, you just can’t spit it out.  Then later, when you have stopped actively pursuing it, the word surfaces.  That letting go of pursuit allows your implicit (unconscious) memory do its work.

Although almost everyone experiences Mind Pops, there seems to be an increased frequency of Mind Popping in individuals with mental health issues.  Researchers Keith Laws, Lia Kvavilashvili, and Ia Elua, conducted some preliminary research whereby they compared the frequency of Mind Pops in 37 individuals with schizophrenia, 31 people with depression, and 26 individuals with no mental health issues.  On average, individuals with Schizophrenia reported 3-4 Mind Pops a weeks, while individuals with depression reported 1-2 a month, and healthy individuals reported 1-2 every six months.  Invasive thoughts that bleed through consciousness are indeed some of the prominent features of schizophrenia and depression, so these categorical differences do make sense.

In my personal correspondence with Dr. Kvavilashvili, she differentiated Mind Pops from the Involuntary Autobiographical Memories I described in a previous post titled The Guilt-Empathy Connection.  In that post I discussed a similar phenomena whereby “serenity seems to occasionally pave the way for a sequence of thoughts triggered by a song or a smell, or anything really, that ushers in a blast from the past.  A cavalcade of memories then flow forth both effortlessly and seamlessly.  And all of this occurs outside of conscious control.  For me, it often begins with a pleasant memory, but it can take a circuitous route, bringing me to memories that I would prefer remain inaccessible.  The ending point is usually a moment in time where I come face to face with a mistake I made – usually a long forgotten unintentional misstep that reveled a less sensitive or perceptive side of my persona.”  Dr. Kvavilashvili noted that there seem to be “personality and individual difference variables at play” in my type of guilt based Involuntary Autobiographical Memories.

In a cursory review of the literature, I did come across a study by Dr. Dorthe Berntsen and she wrote that “The involuntary [autobiographical] memories more frequently referred to specific episodes, came with more physical reaction, had more impact on mood, and dealt with more unusual and less positive events.”  This coincides with my anecdotal experiences (for whatever that is worth).  For me, these events were indeed outliers, they were negative and viscerally so, and they did significantly affect my mood.  Mind Pops are quite different from such Involuntary Autobiographical Memories in that the Pops are more semantic in nature (rather than biographical or experiential), and the Pops tend to be more positively experienced.

Although Mind Pops and Involuntary Autobiographical Memories are commonplace, they certainly constitute manifestations of our amazing and incredibly complex brain.  Please share your interesting Mind Pops or Involuntary Autobiographical Memories in the Comments section below so that you can showcase the amazing capabilities of your brain.  And when you have one of those “out of the blue” Mind Pops look deep to find the source of the subconscious trigger – you might be amazed by your inattentional blindness or the vastness of what your mind’s eye takes in beyond what you see.

References:

Berntsen, D., and Hall, N. M., (2004).  The episodic nature of involuntary autobiographical memories. Memory & Cognition. Jul; 32(5): 789-803.

Cowen, Mark, (2012).  ‘Mind-pop’ frequency increased in schizophrenia patients.  MedWire News.com

Guild, G. (2010).  Are You a Robot? can I Program Your Responses?  How Do You Think? http://geraldguild.com

Guild, G. (2012).  The Guilt – Empathy Connection.  How Do You Think? http://geraldguild.com

Elua, I., Laws, K., and Kvavilashvili, L.. (2012). From mind-pops to hallucinations? A study of involuntary semantic memories in schizophrenia.  Psychiatry Research. V. 196 (2), Pgs. 165-170.

Jbar, Ferris, (2012). Mind-Pops: Psychologists Begin to Study an Unusual form of Proustian Memory.  Scientific American.com

Kvavilashvilia, L., and Mandler, G. (2003). Out of one’s mind: A study of involuntary semantic memories.  Paper shared by author in personal correspondence.

Science Daily (2012). Mind-Pops More Likely With Schizophrenia.  ScienceDaily.com

The year 2011 proved to be a challenging year.  A number of serious health issues in close family members took center stage.  The frequency of my posts declined in part due to these important distractions but other factors also played a major role.  Although I published fewer articles, the number of visits to my blog increased substantially.

Over the course of the year, I had 18,305 hits at my website by 15,167 unique visitors, accounting for over 25,000 page views.  I had visitors from every state in the Union and visits from people from 140 nations around the world.  Visitors from the United States accounted for the vast majority of those hits, but the UK, Canada, and Australia also brought in a large contingent of visitors.

One article in particular far outpaced all other posts.  My post on Brain Waves and Other Brain Measures accounted for as many visits as the next three most popular posts combined.  Of my posts published in 2011, only four made it to this year’s top ten list.  The other six were published in 2010.  Of those six from 2010, four were also on the top ten list last year.

Great interest persisted in my post entitled Nonmoral Nature: It is what it is.  This review of Stephen Jay Gould’s most famous article sustained a number two ranking for a second straight year.  I had also reviewed in 2010 a very popular New York Time’s article by Steven Pinker entitled The Moral Instinct.  This article moved up a notch this year, ultimately ranking number three.  My critical article on the Implicit Associations Test ranked number four this year, versus a number six ranking last year.  And my Hedgehog versus the Fox mindset piece ranked number ten this year, compared to a number seven ranking last year.

So here is the Top Ten list for 2011.

1. Brainwaves and Other Brain Measures (2011)
2. Non Moral Nature: It is what it is (2010)
3. Moral Instinct  (2010)
4. IAT: Questions of Reliability and Validity  (2010)
5. Where Does Prejudice Come From?  (2011)
6. Cognitive Conservatism, Moral Relativism, Bias, and Human Flourishing  (2011)
7. What Plato, Descartes, and Kant Got Wrong: Reason Does Not Rule.  (2010)
8. Intuitive Thought  (2010)
9. Effects of Low SES on Brain Development  (2011)
10. Are you a Hedgehog or a Fox?  (2010)

It’s interesting to me that this list includes the very foundational issues that have driven me in my quest.  And each was posted with great personal satisfaction.   This encompassing cross section of my work is, in fact, a good starting point for those who are new to my blog.  There are several popular 2011 posts that ranked outside the top ten but ranked highly relative to other posts published in 2011.  These other posts include:

• Tribal Moral Community
• Schadenfreude
• Narrative Fallacy
• What not to say to someone with cancer: And what helps
• Parenting Style May Matter for Infants & the Economically Deprived
• Narrative Fallacy: A “Sign” of Cognitive Maturation

One article I published late in 2011 has attracted significant attention.   I believe that it is perhaps one of the most important posts I’ve written.  As I was writing this retrospective, Conspicuous Consumption and the Peacock’s Tail was far outpacing all other posts.

The most emotional and personally relevant articles pertained to significant problems in healthcare in the United States and my wife’s battle with breast cancer.  These articles include: (a) What not to say to someone with cancer: And what helps; (b) Up and Ever Onward: My Wife’s Battle With Cancer; (c) Cancer, Aging, & Healthcare: America, We Have a Problem; (d) We’re Number 37! USA USA USA!; and (e) Tears of Strength in Cancer’s Wake.  The latter pertains to perhaps the proudest parental moment of my life.

Another very important issue that I wrote a fair amount about includes the pernicious affect of poverty on child development.  Clicking here takes you to a page that lists all of the articles on this topic.  Knowing the information in this series should motivate us, as a society, to truly evaluate our current political and economic policies.

One of my favorite articles tackled my long standing curiosity about the geology of the place I live.  The article itself did not get a lot of attention, but I sure loved writing it.

This two-year journey, thus far has resulted in perhaps unparalleled personal and intellectual growth.  It has changed the way I look at life, the world around me, and my fellow human beings.   It is my sincerest hope that those who have seen fit to read some of my material have experienced shifts of perception or at least a modicum of enlightenment.

The bottom line:

The human brain, no matter how remarkable, is flawed in two fundamental ways.  First, the proclivities toward patternicity (pareidolia), hyperactive agency detection, and superstition, although once adaptive mechanisms, now lead to many errors of thought.  Since the age of enlightenment, when human kind developed the scientific method, we have exponentially expanded our knowledge base regarding the workings of the world and the universe.  These leaps of knowledge have rendered those error prone proclivities unessential for survival.  Regardless, they have remained a dominant cognitive force.  Although our intuition and rapid cognitions have sustained us, and in some ways still do, the subsequent everyday illusions impede us in important ways.

Secondly, we are prone to a multitude of cognitive biases that diminish and narrow our capacity to truly understand the world. Time after time I have written of the dangers of ideology with regard to its capacity to blindfold its disciples.  Often those blindfolds are absolutely essential to sustain the ideology.  And this is dangerous when truths and facts are denied or innocents are subjugated or brutalized.  As I discussed in Spinoza’s Conjecture:

“We all look at the world through our personal lenses of experience.  Our experiences shape our understanding of the world, and ultimately our understanding of [it], then filters what we take in.  The end result is that we may reject or ignore new and important information simply because it does not conform to our previously held beliefs.

Because of these innate tendencies, we must make additional effort in order to discover the truth.

We humans are very good at dividing ourselves up into groups.  We accomplish this in a multitude of ways.  Even within homogeneous groupings we tend to find subtle ways to carve people out.  It is far easier however, when people vary by gender, ethnicity, race, class, neighborhood, region, nationality, religion, and/or sexual orientation.  For some reason we are drawn to and comforted by others that share physical resemblance, culture, attitude, values, history, important symbols, and affiliations.  Conversely, we are threatened by those in the outgroup.  Why is this?  What drives us to carve out, cast away and divide our fellow human beings into camps of “us” and “them?” Is it a byproduct of socialization or perhaps a part of our nature?

I saw this very clearly growing up in a small rural town in Western New York.  Even though we were all white middle class Christian kids for the most part, we effectively divided ourselves into camps – some actively participating in the parceling and others passively falling victim to it.  There were the popular kids, the tough kids, the village kids, and the farm kids.  And as we became more “sophisticated,” the parcels emerged with more universal group titles such as the heads, the jocks, the brains, the nerds, etc.  Some kids traversed multiple groups quite effectively while others fit into no group at all.

It wasn’t until I went to college that I was immersed with young adults who parceled out their peers in even more “enlightened” ways.  I went to SUNY Geneseo where the student body was very similar to that of my home town, again, largely a white middle class subset of New York State – but a bit more diverse geographically and religiously.  The most striking division was imposed by students from Westchester County, Long Island, and New York City who looked at their fellow New Yorkers emanating from any location west of the Hudson River as being inferior.  This “geographism” was shocking to me.  I was clearly in the inferior outgroup.

On top of that, there were sorority and fraternity groupings, valuations made by respect for one’s major, and more subtly by the size of the town one came from.  All this being said, I enjoyed college, learned a lot, and have great respect for the institution today.  I am not singling out any one town or university – I suspect that my experience was no different than that most kids encountered growing up.  The point is this – we are seemingly driven to parcel ourselves.  Even during my doctoral training in Cincinnati there was “geographism” whereby people from Kentucky (just across the Ohio River) were cast in a relative negative light by Ohioans much as New Yorkers downcast people from Pennsylvania or New Jersey.  On another level, think about the antipathy between cat lovers and dog lovers.  Then there are Yankee fans and Red Sox fans (insert any sports team where fans divide themselves with similar acrimony).  It is every where!

I was very fortunate to have a mother who encouraged me to respect diversity and not to judge others by group affiliation.  She spoke out against or talked with me privately so that I would not emulate other role models who were not so open minded.  I have always been thankful for her influence.  And because of her I have in maturity always tried to emulate her.  It’s not always easy – but I do try.  Something tells me that one’s level of prejudice is not simply a function of having a great role model or a bad one.  This tendency is so universal and plays out in very subtle ways that are not always evidenced as explicit overt racism or sexism.

Evidence, as it turns out, is increasingly supporting my hunch.  Group prejudices are evident even in pre-vocal babies (Mahajan, 2011). This growing body of research has been supplemented by an ingenious set of studies of prejudice in nonhuman primates published recently in the Journal of Personality and Social Psychology.  The primary author, Neha Mahajan, from Yale University, was kind enough to share with me her paper entitled The Evolution of Intergroup Bias: Perceptions and Attitudes in Rhesus Macaques.

The researchers conducted seven different in-vivo experiments to explore whether old world monkeys, with whom we shared a common ancestor more than 30 million years ago (Hedges & Blair, Ed., 2009), evidence human-like intergroup bias.  This preliminary work establishes that we do share this trait, suggesting that prejudice may in fact be a part of our very nature.  It appears that prejudicial thinking has been adaptive from an evolutionary perspective or at least has been a vestigial stow away linked with some other trait that has been naturally selected.

There is some danger in this notion.  If we accept prejudice as a part of our nature, we may be more inclined to devote less effort to address it from a social perspective.  The authors are careful to point out, however, that previous research has established that prejudices can be re-mediated  through exposure and teaching or conversely entrenched through poor modeling.  These results do not diminish the influence of nurture, instead the authors highlight the importance of understanding that our brains are pre-wired for prejudice.  I have discussed human prejudice before within the context of the Implicit Associations Test (IAT) that suggests that our biases are implicit (unconscious).  Although implicit attributes are difficult to measure, there is good reason to believe that we do universally, inherently, and unknowingly harbor biases.  We must accept this and build programs upon this understanding with targeted evidenced based strategies to combat such erroneous thinking.  It is part of who we are – and once again, evidence of how flawed the human brain is.  Hate, bullying, homophobia, and racism – they all are part of our “monkey-brain.”  Here’s hoping we can rise above it.

References:

Grewal, D. (2011).  The Evolution of Prejudice: Scientists See the Beginnings of Racism in Monkeys. Scientific American: MIND. April 5.

Hedges, S. B., & Blair, S. (Eds.).  (2009).  The Timetree of Life. New York, NY: Oxford University Press.

Mahajan, N., Martinez, M. A., Gutiezzez, N. L., Diesendruck, G., Banaji, M., & Santos, L. R.  (2011).  The Evolution of Intergroup Bias: Perceptions and Attitudes in Rhesus Macaques. Journal of Personality and Social Psychology. Vol. 100, No. 3. 387-405.

The Implicit Associations Test (IAT) is a very popular method for measuring implicit (implied though not plainly expressed) biases. Greenwald, one of the primary test developers, suggests that “It has been self-administered online by millions, many of whom have been surprised—sometimes unpleasantly—by evidence of their own unconscious attitudes and stereotypes regarding race, age, gender, ethnicity, religion, or sexual orientation.” (2010). It purports to tap into our unconscious or intuitive attitudes at a deeper level than those that we are able to rationally express. The best way to get an idea of just what the IAT is, is to take it. If you haven’t done so already, go to the Implicit Associations Test website and participate in a demonstration of the Race Test. It takes about ten minutes.

I tend to have a skeptical inclination. This in part stems from the training that I benefited from in acquisition of my PhD in psychology. But it is also just part of who I am. Psychology is, in itself, a rather soft science – full of constructs – and variables that are inherently difficult to measure with any degree of certainty. I learned early in my training that there are dangers associated with inference and measuring intangibles. In fact, my training in personality and projective measures essentially focused on why not to use them – especially when tasked with helping to make important life decisions. Why is this? All psychological measures contain small and predictable amounts of unavoidable error – but those based on constructs and inference are particularly untenable.

This is relevant because as we look at thinking processes, we are dealing with intangibles. This is especially true when we are talking about implicit measures. Any discussion of implicit thought necessitates indirect or inferential measures and application of theoretical constructs. So, with regard to the Implicit Associations Test (IAT), one needs to be careful.

Currently, increasing evidence suggests that our intuition has a powerful influence over our behavior and moment to moment decision making. Books like Blink by Malcolm Gladwell and How We Decide by Jonah Lehrer point out the power of intuition and emotion in this regard. Chabris and Simons in their book, The Invisible Gorilla, make a strong argument that intuition itself sets us up for errors. Gladwell perhaps glorifies intuition – but the reality is, it (intuition) is a powerful force. Gladwell uses the story of the IAT as evidence of such power. Essentially, if the IAT is a valid and reliable measure, it provides strong evidence of the problems of intuition.

I am motivated to shed some light on the IAT – not because of my personal IAT results, which were disappointing, but because the IAT has the risk of gaining widespread application without sufficient technical adequacy. Just think of the ubiquitous Meyers-Briggs Personality Inventory and the breadth and depth of popular use and appeal that it has garnered (without a shred of legitimate science to back it up). Real decisions are made based on the results of this instrument and frankly it is dangerous. The Meyers-Briggs is based on unsubstantiated and long out-of-date Jungian constructs and was built by individuals with little to no training in psychology or psychometrics. This is not the case for the IAT for sure, but the risks of broad and perhaps erroneous application are similar.

The authors of the IAT have worked diligently over the years to publish studies and facilitate others’ research in order to establish the technical adequacy of the measure. This is a tough task because the IAT is not one test, but rather, it is a method of measurement that can be applied to measure a number of implicit attitudes. At the very foundation of this approach there is a construct, or belief, that necessitates a leap of faith.

So what is the IAT? Gladwell (2005) summarizes it in the following way:

The Implicit Association Test (IAT)…. measures a person’s attitude on an unconscious level, or the immediate and automatic associations that occur even before a person has time to think. According to the test results, unconscious attitudes may be totally different or incompatible with conscious values. This means that attitudes towards things like race or gender operate on two levels:
1. Conscious level- attitudes which are our stated values and which are used to direct behavior deliberately.
2. Unconscious level- the immediate, automatic associations that tumble out before you have time to think.
Clearly, this shows that aside from being a measurement of attitudes, the IAT can be a powerful predictor of how one [may] act in certain kinds of spontaneous situations.

So here is one of the difficulties I have with the measure. Take this statement: “The IAT measures a person’s attitude on an unconscious level, or the immediate and automatic associations that occur even before a person has time to think.” Tell me how one would directly and reliably measure “unconscious attitude” without using inference or indirect measures that are completely dependent on constructs? I am not alone in this concern. In fact, Texas A&M University psychologist Hart Blanton, PhD, worries that the IAT has been used prematurely in research without sufficient technical adequacy. Blanton has in fact published several articles (Blanton, et al., 2007; Blanton, et al., 2009) detailing the IAT’s multiple psychometric failings. He suggests that perhaps the greatest problem with this measure concerns the way that the test is scored.

First you have to understand how it all works. The IAT purports to measure the fluency of people’s associations between concepts. On the Race IAT, a comparison is made between how fluent the respondent pairs pictures of European-Americans with words carrying a connotation of “good” and pictures of African-Americans with words connoting “bad.” The task measures the latency between such pairings and draws a comparison to the fluency of responding when the associations are reversed (e.g., how quickly does the respondent pair European-Americans with words carrying a “bad” connotation and African-Americans with words connoting “good.”). If one is quicker at pairing European-Americans with “good” and African Americans with “bad” then it is inferred that the respondent has a European-American preference. The degree of preference is determined by the measure of fluency and dysfluency in making those pairings. Bigger differences in pairing times result in stronger ratings of one’s bias. Blanton questions the arbitrary nature of where the cutoffs for mild, moderate, and strong preferences are set when there is no research showing where the cutoffs should be. Bottom line, Blanton argues, is that the cutoffs are arbitrary. This is a common problem in social psychology.

Another issue of concern is the stability of the construct being measured. One has to question whether one’s bias, or racial preferences, are a trait (a stable attribute over time) or a state (a temporary attitude based on acute environmental influences). The test-retest reliability of the IAT is relatively unstable itself. Regardless, according to Greenwald: “The IAT has also shown reasonably good reliability over multiple assessments of the task. …. in 20 studies that have included more than one administration of the IAT, test–retest reliability ranged from .25 to .69, with mean and median test–retest reliability of .50.” Satisfactory test-retest reliability values are in the .70 to.80 range. To me, there is a fair amount of variance unaccounted for and a wide range of values (suggesting weak consistency). My IATs have bounced all over the map. And boy did I feel bad when my score suggested a level of preference that diverges significantly from my deeply held values. Thank goodness I have some level of understanding of the limitations of such metrics. Not everyone has such luxury.

As I noted previously, the IAT authors have worked diligently to establish the technical adequacy of this measure and they report statistics attesting to the internal-consistency, test-retest reliability, predictive validity, convergent validity, and discriminant validity, almost always suggesting that results are robust (Greenwald, 2010; Greenwald, 2010; Greenwald, et al, 2009; Lane, et al, 2007) . There are other studies including those carried out by Blanton and colleagues, that suggest otherwise. To me, these analyses are important and worthwhile – however, at the foundation, there is the inescapable problem of measuring unconscious thought.

Another core problem is that the validity analyses employ other equally problematic measures of intangibles in order to establish credibility. I can’t be explicit enough – when one enters the realm of the implicit – one enters a realm of intangibles: and like it or not, until minds can be read explicitly, the implicit is essentially immeasurable with any degree of certainty. The IAT may indeed measure what it purports to measure, but the data on this is unconvincing. Substantial questions of reliability and validity persist. I would suggest that you do not take your IAT scores to heart.

References

Azar, B. (2008). IAT: Fad or fabulous? Monitor on Psychology. July. Vol 39, No. 7,  page 44.

Blanton, H., Jaccard, J., Christie, C., and Gonzales, P. M. (2007). Plausible assumptions, questionable assumptions and post hoc rationalizations: Will the real IAT, please stand up? Journal of Experimental Social Psychology. Volume 43, Issue 3, Pages 399-409.

Blanton, H., Klick, J., Mitchell, G., Jaccard, J.,Mellers, B., Tetlock, P. E. (2009). Strong Claims and Weak Evidence: Reassessing the Predictive Validity of the IAT. Journal of Applied Psychology. Vol. 94, No. 3, 567–582

Chabris, C. F., & Simons, D. J., 2010. The Invisible Gorilla. Random House: New York.

Gladwell, M. 2005. Blink: The Power of Thinking Without Thinking. Little, Brown and Company: New York.

Greenwald, A. G. (2010).  I Love Him, I Love Him Not: Researchers adapt a test for unconscious bias to tap secrets of the heart. Scientific American.com: Mind Matters.   http://www.scientificamerican.com/article.cfm?id=i-love-him-i-love-him-not

Greenwald, A. G. (2009). Implicit Association Test: Validity Debates. http://faculty.washington.edu/agg/iat_validity.htm

Greenwald, A. G., Poehlman, T. A., Uhlmann, E., & Banaji, M. R. (2009). Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology. 97, 17–41.

Lane, K. A., Banaji, M. R., Nosek, B. A., & Greenwald, A. G. (2007). Understanding and using the Implicit Association Test: IV. What we know (so far) (Pp. 59–102).  In B. Wittenbrink & N. S. Schwarz (Eds.). Implicit measures of attitudes: Procedures and controversies. New York: Guilford Press.

Lehrer, J. 2009. How We Decide. Houghton Mifflin Harcourt: New York.