See all Lex Fridman transcripts on Youtube

Speaker 1

00:00

Let me ask you to this question, whether it's bell curve or any research on race differences, can that be used to increase the amount of racism in the world, can that be used to increase the amount of hate in the world?

Speaker 2

00:16

My sense is there is such enormous reservoirs of hate and racism that have nothing to do with scientific knowledge of the data that speak against that. That no, I don't want to give racist groups a veto power over what scientists study.

Speaker 1

00:44

The following is a conversation with Richard Heyer on the science of human intelligence. This is a highly controversial topic, but a critically important 1 for understanding the human mind. I hope you will join me in not shying away from difficult topics like this, and instead, let us try to navigate it with empathy, rigor, and grace.

Speaker 1

01:06

If you're watching this on video now, I should mention that I'm recording this introduction in an undisclosed location somewhere in the world. I'm safe and happy, and life is beautiful. This is Alex Friedman Podcast. To support it, please check out our sponsors in the description.

Speaker 1

01:23

And now, dear friends, here's Richard Heyer. What are the measures of human intelligence and How do we measure it?

Speaker 2

01:31

Everybody has an idea of what they mean by intelligence. In the vernacular, what I mean by intelligence is just being smart, how well you reason, how well you figure things out, What you do when you don't know what to do. Those are just kind of everyday common sense definitions of how people use the word intelligence.

Speaker 2

01:56

If you want to do research on intelligence, measuring something that you can study scientifically is a little trickier. And what almost all researchers who study intelligence use is the concept called the G factor, general intelligence. And that is what is common, that is a mental ability that is common to virtually all tests of mental abilities.

Speaker 1

02:26

What's the origin of the term G-factor, by the way? Such a funny word for such a fundamental human thing.

Speaker 2

02:32

The general factor, I really started with Charles Spearman. And he noticed, this is like, boy, more than 100 years ago. He noticed that when you tested people with different tests, all the tests were correlated positively.

Speaker 2

02:53

And so he was looking at student exams and things. And he invented the correlation coefficient, essentially. And he, when he used it to look at student performance on various topics, he found they all the scores were correlated with each other and they were all positive correlations. So he inferred from this that there must be some common factor that was irrespective of the content of the test.

Speaker 1

03:23

And positive correlation means if you do well on the first test, you're likely to do well on the second test, and presumably that holds for tests across even disciplines, so not within subject but across subjects, so that's where the general comes in. Something about general intelligence, So when you were talking about measuring intelligence and trying to figure out something difficult about this world and how to solve the puzzles of this world, that means generally speaking, not some specific test, but across all tests. Absolutely right, and people get hung up on this because they said, well, what about the ability to do X?

Speaker 1

04:06

Isn't that independent?

Speaker 2

04:08

And they said, I know somebody who's very good at this, but not so good at this, this other thing. And so there are a lot of examples like that, but it's a general tendency. So exceptions really don't disprove, you know, your everyday experience is not the same as what the data actually show.

Speaker 2

04:30

And your everyday experience, when you say, oh, I know someone who's good at X, but not so good at Y, that doesn't contradict the statement of about, he's not so good, but he's not the opposite. It's not a negative correlation.

Speaker 1

04:46

Okay, so we're not, our anecdotal data, I know a guy who's really good at solving some kind of visual thing. That's not sufficient for us to understand actually the depths of that person's intelligence. So how this idea of g-factor, how much evidence is there, how strong, you know, given across the decades that this idea has been around, how much has it been held up that there's a universal sort of horsepower of intelligence that's underneath all of it, all the different tests we do to try to get to this thing in the depths of the human mind that's a universal stable measure of a person's intelligence.

Speaker 2

05:34

You used a couple of words in there, stable and.

Speaker 1

05:38

Are we gonna have to be precise with words? I was hoping we can get away with being poetic.

Speaker 2

05:42

We can, there's a lot about research in general, not just intelligence research, that is poetic. Science has a phonetic aspect to it, and good scientists are very intuitive. They're not just, hey, hey, these are the numbers.

Speaker 2

05:59

You have to kind of step back and see the big picture. When it comes to intelligence research, you asked how well has this general concept held up? And I think I can say without fear of being empirically contradicted, that it is the most replicated finding in all of psychology. Now some cynics may say, well big deal, psychology, we all know there's a replication crisis in psychology and a lot of this stuff doesn't replicate.

Speaker 2

06:28

That's all true. There is no replication crisis when it comes to studying the existence of this general factor. Let me tell you some things about it. It looks like it's universal, that you find it in all cultures.

Speaker 2

06:44

The way you find it, step back 1 step, the way you find it is to give a battery of mental tests. What battery? You choose. Take a battery of any mental test you want, give it to a large number of diverse people, and you will be able to extract statistically the commonality among all those tests.

Speaker 2

07:08

It's done by a technique called factor analysis. People think that this may be a statistical artifact of some kind, it is not a statistical artifact. What is factor analysis? Factor analysis is a way of looking at a big set of data and look at the correlation among the different test scores and then find empirically the clusters of scores that go together.

Speaker 2

07:35

And there are different factors. So if you have a bunch of mental tests, there may be a verbal factor. There may be a numerical factor. There may be a visual spatial factor.

Speaker 2

07:45

But those factors have variants in common with each other. And that is the common, that's what's common among all the tests, and that's what gets labeled the G factor. So if you give a diverse battery of mental tests and you extract a g-factor from it, that factor usually accounts for around half of the variance. It's the single biggest factor, but it's not the only factor.

Speaker 2

08:12

But it is the most reliable, It is the most stable, and it seems to be very much influenced by genetics. It's very hard to change the G factor with training or drugs or anything else. We don't know how to increase the G factor.

Speaker 1

08:34

Okay, you said a lot of really interesting things there. So first, just to get people used to it in case they're not familiar with this idea, G factor is what we mean. So often there's this term used, IQ, which is the way IQ is used, they really mean G factor in regular conversation.

Speaker 1

08:58

Because what we mean by IQ, we mean intelligence, and what we mean by intelligence, we mean general intelligence, and general intelligence in the human mind from a psychology, from a serious rigorous scientific perspective actually means G factor. So G factor equals intelligence, just in this conversation to define terms. Okay, so there's this stable thing called g-factor.

Speaker 2

09:23

You said,

Speaker 1

09:23

now factor, you said factor many times means a measure that potentially could be reduced to a single number across the different factors you mentioned. And what you said, it accounts for half, half-ish. Accounts for half-ish of what?

Speaker 1

09:48

Of variance across the different set of tests. So if you do for some reason well on some set of tests, What does that mean? So that means there's some unique capabilities outside of the G factor that might account for that. And what are those?

Speaker 1

10:07

What else is there besides the raw horsepower, the engine inside your mind that generates intelligence?

Speaker 2

10:13

There are test-taking skills. There are specific abilities. Someone might be particularly good at mathematical things, mathematical concepts, even simple arithmetic people, some people are much better than others.

Speaker 2

10:34

You might know people who can memorize, and short-term memory is another component of this. Short-term memory is 1 of the cognitive processes that's most highly correlated with the G factor.

Speaker 1

10:53

So. So all those things like memory, test-taking skills account for variability across the test performances. But you, so you can run, but you can't hide from the thing that God gave you, the genetics. So that G factor, science says that G factor's there.

Speaker 1

11:15

Each 1 of us have.

Speaker 2

11:16

Each 1 of us has a g-factor.

Speaker 1

11:19

Oh boy.

Speaker 2

11:19

Some have more than others.

Speaker 1

11:21

I'm getting uncomfortable already.

Speaker 2

11:22

Well IQ is a score and IQ score is a very good estimate of the g-factor. You can't measure g directly, there's no direct measure. You estimate it from these statistical techniques.

Speaker 2

11:39

But an IQ score is a good estimate. Why? Because a standard IQ test is a battery of different mental abilities. You combine it into 1 score, and that score is highly correlated with the G factor, even if you get better scores on some subtests than others.

Speaker 2

12:00

Because again, it's what's common to all these mental abilities.

Speaker 1

12:04

So a good IQ test, and I'll ask you about that, but a good IQ test tries to compress down that battery of tests, like tries to get a nice battery, the nice selection of variable tests into 1 test. And so in that way, it sneaks up to this G factor. And that's another interesting thing about G factor.

Speaker 1

12:28

Now you give, first of all, You have a great book on the neuroscience of intelligence. You have a great course, which is when I first learned, you're a great teacher, let me just say. Your course at the teaching company, I hope I'm saying that correctly.

Speaker 2

12:45

The intelligent brain.

Speaker 1

12:47

The intelligent brain. Is when I first heard about this G factor, this mysterious thing that lurks in the darkness that we cannot quite shine a light on, we're trying to sneak up on. So the fact that there's this measure, stable measure of intelligence, we can't measure directly.

Speaker 1

13:05

But we can come up with a battery test or 1 test that includes a battery of variable type of questions that can reliably or attempt to estimate in a stable way that G factor. That's a fascinating idea. So for me as an AI person, it's fascinating. It's fascinating there's something stable like that about the human mind, especially if it's grounded in genetics.

Speaker 1

13:31

It's both fascinating that as a researcher of the human mind and all the human psychological, sociological, ethical questions that start arising, it makes me uncomfortable. But truth can be uncomfortable.

Speaker 2

13:51

I get that a lot about being uncomfortable talking about this. Let me go back and just say 1 more empirical thing. It doesn't matter which battery of tests you use.

Speaker 2

14:08

So there are countless tests. You can take any 12 of them at random, extract a G factor and another 12 at random and extract a G factor. And those G factors will be highly correlated, like over 0.9 with each other. So it is a ubiquitous.

Speaker 2

14:26

It doesn't depend on the content of the test, is what I'm trying to say. It is general among all those tests of mental ability and tests of mental, you know, mental abilities include things like, geez, playing poker. Your skill at poker is not unrelated to G. Your skill at anything that requires reasoning and thinking, anything, spelling, arithmetic, more complex things, this concept is ubiquitous.

Speaker 2

15:00

And when you do batteries of tests in different cultures, you get the same thing.

Speaker 1

15:05

So this says something interesting about the human mind that is a computer is designed to be general. So that means you can, So it's not easily made specialized, meaning if you're going to be good at 1 thing, Miyamoto Musashi has this quote, he's an ancient warrior famous for the Book of 5 Rings in the martial arts world. And the quote goes, "'If you know the way broadly, "'you will see it in everything.'" Meaning if you do 1 thing, it's going to generalize to everything.

Speaker 1

15:46

And that's an interesting thing about the human mind. So that's what the G factor reveals. Okay, so what's the difference, if you can elaborate a little bit further, between IQ and G factor, just because it's a source of confusion for people.

Speaker 2

16:03

And IQ is a score. People use the word IQ to mean intelligence, but IQ has a more technical meaning for people who work in the field. And it's an IQ score, a score on a test that estimates the G factor.

Speaker 2

16:20

And the G factor is what's common among all these tests of mental ability. So if you think about, it's not a Venn diagram, but I guess you could make a Venn diagram out of it, but the G factor would be really at the core, what's common to everything. And what IQ scores do is they allow a rank order of people on the score. And this is what makes people uncomfortable.

Speaker 2

16:46

This is where there's a lot of controversy about whether IQ tests are biased toward any 1 group or another. And a lot of the answers to these questions are very clear, but they also have a technical aspect of it that's not so easy to explain.

Speaker 1

17:04

Well, we'll talk about the fascinating and the difficult things about all of this, but... So by the way, when you say rank order, that means you get a number and that means 1 person, you can now compare. Like you could say that this other person is more intelligent than me.

Speaker 1

17:22

Which is a- Well, what

Speaker 2

17:23

you can say is IQ scores are interpreted really as percentiles. So that if you have an IQ of 140 and somebody else has 70, the metric is such that you cannot say the person with an IQ of 140 is twice as smart as a person with an IQ of 70. That would require a ratio scale with an absolute 0.

Speaker 2

17:49

Now you may think you know people with 0 intelligence, but in fact, there is no absolute 0 on an IQ scale. It's relative to other people. So relative to other people, somebody with an IQ score of 140 is in the upper less than

Speaker 1

18:07

1%,

Speaker 2

18:09

whereas somebody with an IQ of 70 is 2 standard deviations below the mean. That's a different percentile.

Speaker 1

18:18

So it's similar to like in chess, you have an Elo rating that's designed to rank order people. So you can't say it's twice 1 person. If your ELO rating's twice another person, I don't think you're twice as good at chess.

Speaker 1

18:36

It's not stable in that way, but because it's very difficult to do these kinds of comparisons. But, so what can we say about the number itself? Is that stable across tests and so on or no? There are

Speaker 2

18:50

a number of statistical properties of any test, they're called psychometric properties. You have validity, you have reliability, reliability, there are many different kinds of reliability. They all essentially measure stability.

Speaker 2

19:06

And IQ tests are stable within an individual. There are some longitudinal studies where children were measured at age 11. And again, when they were 70 years old and the 2 IQ scores are highly correlated with each other. This comes from a fascinating study from Scotland.

Speaker 2

19:26

In the 1930s, some researchers decided to get an IQ test on every single child age 11 in the whole country. And they did. And those records were discovered in an old storeroom at the University of Edinburgh by a friend of mine, Ian Deary, who found the records, digitized them, and has done a lot of research on the people who are still alive today from that original study, including brain imaging research, by the way. Really it's a fascinating group of people who are studied.

Speaker 2

20:07

Not to get ahead of the story, but 1 of the most interesting things they found is a very strong relationship between IQ measured at age 11 and mortality so that, you know, in the 70 years later, they looked at the survival rates and they could get death records from everybody and Scotland has universal health care for everybody. And it turned out if you divide people by their age 11 IQ score into quartiles and then look at how many people are alive 70 years later. I know this is in the book, I have the graph in the book, but there are essentially twice as many people alive in the highest IQ quartile than in the lowest IQ quartile. It's true in men and women.

Speaker 1

21:05

Interesting.

Speaker 2

21:06

So it makes a big difference. Now why this is the case is not so clear since everyone had access to healthcare.

Speaker 1

21:15

Well there's a lot, and we'll talk about it, you know, just the sentences you used now could be explained by nature or nurture. We don't know. Now, there's a lot of science that starts to then dig in and investigate that question.

Speaker 1

21:31

But let me linger on the IQ test. How are the test design, IQ test design, how do they work? Maybe some examples for people who are not aware. What makes a good IQ test question that sneaks up on this G factor measure?

Speaker 2

21:48

Well, your question is interesting because you want me to give examples of items that make good items and what makes a good item is not so much its content but its empirical relationship to the total score that turns out to be valid by other means. So for example, let me give you an odd example from personality testing. Nice.

Speaker 2

22:15

So there's a personality test called the Minnesota Multiphasic Personality Inventory, MMPI. Been around for decades.

Speaker 1

22:24

I've heard about this test recently because of the Johnny Depp and Amber Heard trial. I don't know if you've been paying attention to that. But they had psychologists.

Speaker 2

22:31

I have not been paying attention

Speaker 1

22:33

to it. They had psychologists on the stand and they were talking, apparently those psychologists did, again, I'm learning so much from this trial. They did different, a battery of tests to diagnose personality disorders.

Speaker 1

22:50

Apparently there's that systematic way of doing so and the Minnesota 1 is 1 of the ones that there's the most science on. There's a lot of great papers which were all continuously cited on the stand, which is fascinating to watch. Sorry, a little bit of attention.

Speaker 2

23:06

It's okay, I mean, this is interesting because you're right, it's been around for decades. There's a lot of scientific research on the psychometric properties of the test, including what it predicts with respect to different categories of personality disorder But what I want to mention is the content of the items on that test. All of the items are Essentially true false items true or false.

Speaker 2

23:33

I prefer a shower to a bath. True or false, I think Lincoln was a better president than Washington. What have all these, what does that have to do, And the point is the content in these items, nobody knows why these items in aggregate predict anything, but empirically they do. It's a technique of choosing items for a test that is called dust bowl empiricism.

Speaker 2

24:05

That the content doesn't matter, but for some reason when you get a criterion group of people with this disorder, and you compare them to people without that disorder, these are the items that distinguish. Irrespective of content, it's a hard concept to grasp.

Speaker 1

24:22

Well, first of all, it's fascinating. But from, because I consider myself part psychologist because I love human-robot interaction, and that's a problem, half of that problem is a psychology problem because there's a human. So designing these tests to get at the questions is the fascinating part.

Speaker 1

24:46

Like how do you get to, like what does Dust Bowl empiricism refer to? Does it refer to the final result? Yeah, so it's the test is Dust Bowl empiricism. But how do you arrive at the battery of questions?

Speaker 1

25:04

I presume 1 of the things, now again, I'm going to the excellent testimony in that trial, the explanation, because they also, they explain the tests, that a bunch of the questions are kind of make you forget that you're taking a test. Like it makes it very difficult for you to somehow figure out what you're supposed to answer.

Speaker 2

25:31

Yes, it's called social desirability. But we're getting a little far afield because I only wanted to give that example of Dust Bowl empiricism. When we talk about the items on an IQ test, many of those items in the Dust Bowl in Pearson method have no face validity.

Speaker 2

25:52

In other words, they don't look like they measure anything. Yes. Whereas most intelligence tests, the items actually look like they're measuring some mental ability. So here's 1 of the-

Speaker 1

26:05

So you were bringing that up as an example as what it is not.

Speaker 2

26:08

Yes.

Speaker 1

26:08

Got it.

Speaker 2

26:09

Okay, so I don't wanna go too far afield on it.

Speaker 1

26:12

Too far afield is actually 1 of the names of this podcast, so I should mention that.

Speaker 2

26:19

Far afield.

Speaker 1

26:19

Far afield. Yeah, so anyway, sorry. So they feel the questions look like they passed the face validity test.

Speaker 1

26:28

And some more than others. So For example,

Speaker 2

26:30

let me give you a couple of things here. If I, 1 of the subtests on a standard IQ test is general information. Let me just think a little bit, because I don't want to give you the actual item.

Speaker 2

26:44

But if I said, how far is it between Washington DC and Miami, Florida, within

Speaker 1

26:52

500

Speaker 2

26:53

miles plus or minus? Well, you know, it's not a fact most people memorize, but you, you know something about geography. You say, well, I flew there once.

Speaker 2

27:04

I know planes fly 500 miles. You can kind of make an estimate. But it's also seems like it would be very cultural. So there's that kind of general information.

Speaker 2

27:20

Then there's vocabulary test. What does regatta mean? And I choose that word because that word was removed from the IQ test because people complained that disadvantaged people would not know that word just from their everyday life. Okay, here's another example from a different kind of sub-test.

Speaker 2

27:47

What's regatta, by the way? Regatta is a... I think I'm disadvantaged. A sailing competition, a competition with boats.

Speaker 2

27:55

Not necessarily sailing, but a competition with boats.

Speaker 1

27:58

Yep, yep. I'm probably disadvantaged in that way. Okay, excellent, so that was removed.

Speaker 1

28:03

Anyway, what you were saying.

Speaker 2

28:05

Okay, so here's another subtest. I'm gonna repeat a string of numbers, and when I'm done, I want you to repeat them back to me.

Speaker 1

28:12

Ready? Okay, 742816. That's way too many. 742816.

Speaker 2

28:25

Okay, you get the idea. Now the actual test starts with a smaller number, you know, like 2 numbers, and then as people get it right, you keep going, Adding to the string of numbers until they can't do it anymore Okay, but now try this. I'm gonna reap.

Speaker 2

28:41

I'm gonna say some numbers and when I'm done I want you to repeat them to me backwards.

Speaker 1

28:46

I quit

Speaker 2

28:47

Okay Now so I gave you some examples of the kind of items on an IQ test. Yes. General information, I can't even remember all, general information, vocabulary, digit span forward and digit span backward.

Speaker 1

29:06

Well, you said I can't even remember them. That's a good question for me. What does memory have to do

Speaker 2

29:13

with geometry? Let's hold on. Okay, all right.

Speaker 2

29:16

Let's just talk about these examples. Now some of those items seem very cultural and others seem less cultural. Which ones do you think, scores on which subtests are most highly correlated with the g factor?

Speaker 1

29:39

Well, the 2 advances less cultural.

Speaker 2

29:43

Well, it turns out vocabulary is highly correlated, and it turns out that digit span backwards is highly correlated.

Speaker 1

29:56

Now that you've... How do you figure?

Speaker 2

29:58

Now you have decades of research to answer the question, how do you figure?

Speaker 1

30:04

Right, so now there's like good research that gives you intuition about what kind of questions get at it, just like there's something I've done, I've actually used for research in semi-autonomous vehicle, like whether humans are paying attention, there's a body of literature that does like end-back tests, for example, where you have to put workload on the brain to do recall, memory recall, and that helps you kind of put some work onto the brain while the person is doing some other task and does some interesting research with that. But that's loading the memory. So there's like research around stably what that means about the human mind.

Speaker 1

30:54

And here you're saying recall backwards is a good predictor. It's a transformation. Yeah, so you have to do some, like you have to load that into your brain and not just remember it, but do something with it.

Speaker 2

31:11

Right, now here's another example of a different kind of test, called the Hick paradigm, and it's not verbal at all. It's a little box and there are a series of lights arranged in a semi-circle at the top of the box and then there's a home button that you press and when 1 of the lights goes on there's a button next to each of those lights you take your finger off the home button and you just press the button next to the light that goes on and so it's a very simple reaction time light goes on as quick as you can you press the button and you get a reaction time. From the moment you lift your finger off the button to when you press the button with where the light is.

Speaker 2

31:58

That reaction time doesn't really correlate with IQ very much. But if you change the instructions and you say 3 lights are gonna come on simultaneously, I want you to press the button next to the light that's furthest from the other 2. So maybe lights 1 and 2 go on and light 6 goes on. Simultaneously, you take your finger off and you would press the button by light 6.

Speaker 2

32:28

That's, that reaction time to a more complex task, it's not really hard, almost everybody gets it all right, but your reaction time to that is highly correlated with the G factor.

Speaker 1

32:43

This is fascinating. So reaction time, so there's a temporal aspect to this. So what role

Speaker 2

32:49

does time? It's the speed of processing.

Speaker 1

32:52

Is this also true for ones that take longer, like 5, 10, 30 seconds? Is time part of the measure with some of these ideas. Yes,

Speaker 2

33:01

and that is why some of the best IQ tests have a time limit, because if you have no time limit, people can do better, but it doesn't distinguish among people that well. So that adding the time element is important. So speed of information processing, and reaction time is a measure of speed of information processing, turns out to be related to the G factor.

Speaker 1

33:31

But the G factor only accounts for maybe half or some amount on the test performance. For example, I get pretty bad test anxiety. Like I was never, I mean, I just don't enjoy tests.

Speaker 1

33:47

I enjoy going back into my cave and working. Like I've always enjoyed homework way more than tests. No matter how hard the homework is because I can go back to the cave and hide away and think deeply. There's something about being watched and having a time limit that really makes me anxious and I could just see the mind not operating optimally at all, but you're saying underneath there, there's still a G factor, there's still.

Speaker 2

34:15

No question, no question. Boy. And if you get anxious taking the test, many people say, oh, I didn't do well because I'm anxious.

Speaker 2

34:21

Yeah. You know, I hear that a lot. Yeah. They, well, fine.

Speaker 2

34:25

If you're really anxious during the test, the score will be a bad estimate of your G factor.

Speaker 1

34:31

Yeah.

Speaker 2

34:31

It doesn't mean the G factor isn't there.

Speaker 1

34:33

That's right.

Speaker 2

34:34

And by the way, standardized tests like the SAT, they're essentially intelligence tests. They are highly g-loaded. Now the people who make the SAT don't want to mention that.

Speaker 2

34:50

They have enough trouble justifying standardized testing, but to call it an intelligence test is really beyond the pale. But in fact, it's so highly correlated because it's a reasoning test. The SAT is a reasoning test, a verbal reasoning, mathematical reasoning. And if it's a reasoning test, it has to be related to G.

Speaker 2

35:13

But if people go in and take a standardized test, whether it's an IQ test or the SAT, and they happen to be sick that day with 102 fever, the score is not going to be a good estimate of their G. If they retake the test when they're not anxious or less anxious or don't have a fever, the score will go up and that will be a better estimate. But you can't say their G factor increased between the 2 tests.

Speaker 1

35:45

Well, it's interesting. So the question is how wide of a battery of tests is required to estimate the g factor well? Because I'll give you as my personal example, I took the SAT and I think it was called the ACT where I was too, also, I took SAT many times.

Speaker 1

36:02

Every single time I got it perfect on math. And verbal, the time limit on the verbal made me very anxious. I did not, I mean, part of it, I didn't speak English very well, but honestly, it was like, you're supposed to remember stuff, and I was so anxious, and as I'm reading, I'm sweating. I can't, you know that feeling you have when you're reading a book and you just read a page and you know nothing about what you've read because you zoned out, that's the same feeling of like, I can't, I have to, you're like, nope, read and understand, and that anxiety's like, and you start seeing like the typography versus the content of the words.

Speaker 1

36:47

Like that was, I don't, it's interesting because I know that what they're measuring, I could see being correlated with something. But that anxiety or some aspect of the performance sure plays a factor. And I wonder how you sneak up in a stable way. I mean, this is a broader discussion about standardized testing, how you sneak up, how you get at the fact that I'm super anxious and still nevertheless measure some aspect of my intelligence.

Speaker 1

37:23

I wonder, I don't know if you can say to that, that time limit sure is a pain.

Speaker 2

37:28

Well, let me say this. There are 2 ways to approach the very real problem that you say that some people just get anxious or not good test takers. By the way, part of testing is you know the answer, you can figure out the answer, or you can't.

Speaker 2

37:49

If you don't know the answer, there are many reasons you don't know the answer at that particular moment. You may have learned it once and forgotten it. It may be on the tip of your tongue and you just can't get it because you're anxious about the time limit. You may never have learned it.

Speaker 2

38:05

You may never, you may have been exposed to it, but it was too complicated and you couldn't learn it. I mean, there are all kinds of reasons here. But for an individual to interpret your scores as an individual, whoever is interpreting the score has to take into account various things that would affect your individual score. And that's why decisions about college admission or anything else where tests are used are hardly ever the only criterion to make a decision.

Speaker 1

38:42

And I think people are, college admissions letting go of that very much.

Speaker 2

38:46

Oh yes, yeah.

Speaker 1

38:48

But what does that even mean? Because is it possible to design standardized tests that do get, that are useful to college admissions?

Speaker 2

38:58

Well, they already exist. The SAT is highly correlated with many aspects of success at college.

Speaker 1

39:05

Here's the problem. So maybe you could speak to this. The correlation across the population versus individuals.

Speaker 1

39:13

So, You know, our criminal justice system is designed to make sure, well, it's still, there's tragic cases where innocent people go to jail, but you try to avoid that. In the same way with testing, it just, it would suck for an SAT to miss genius.

Speaker 2

39:38

Yes, and it's possible, but it's statistically unlikely. So it really comes down to, do which piece of information maximizes your decision-making ability. So If you just use high school grades, it's okay.

Speaker 2

40:05

But you will miss some people who just don't do well in high school, but who are actually pretty smart, smart enough to be bored silly in high school, and they don't care, and their high school GPA isn't that good. So you will miss them in the same sense that somebody who could be very able and ready for college just doesn't do well on their SAT. This is why you make decisions with taking in a variety of information. The other thing I wanted to say, we talked about when you make a decision for an individual.

Speaker 2

40:43

Statistically, For groups, there are many people who have a disparity between their math score and their verbal score. That disparity, or the other way around, that disparity is called tilt. The score is tilted 1 way or the other. And that tilt has been studied empirically to see what that predicts.

Speaker 2

41:07

And in fact, you can't make predictions about college success based on tilt. And mathematics is a good example. There are many people, especially non-native speakers of English, come to this country, take the SATs, do very well on the math and not so well on the verbal. Well, if they're applying to a math program,

Speaker 1

41:31

the

Speaker 2

41:31

professors there who are making the decision or the admissions officers, don't wait so much to score on verbal, especially if it's a non-native speaker.

Speaker 1

41:41

Well, so yeah, you have to try to, in the admission process, bring in the context. But non-native isn't really the problem. I mean, that was part of the problem for me.

Speaker 1

41:53

But it's the anxiety was, which it's interesting. It's interesting. Oh boy, reducing yourself down to numbers. But it's still true, it's still the truth.

Speaker 1

42:09

It's a painful, that same anxiety that led me to be, to struggle with the SAT verbal tests is still within me in all ways of life. So maybe that's not anxiety. Maybe that's something, you know, like personality is also pretty stable.

Speaker 2

42:32

Personality is stable. Personality does impact the way you navigate life. Yeah.

Speaker 2

42:40

There's no question.

Speaker 1

42:42

Yeah, and we should say that the G factor in intelligence is not just about some kind of number on a paper, it also has to do with how you navigate life, how easy life is for you in this very complicated world. So personality's all tied into that in some deep fundamental way.

Speaker 2

43:05

But now you've hit the key point about why we even want to study intelligence, and personality I think to a lesser extent, but that's my interest, is more on intelligence. I went to graduate school and wanted to study personality, but that's kind of another story how I got kind of shifted from personality research over to intelligence research. Because it's not just a number, Intelligence is not just an IQ score.

Speaker 2

43:32

It's not just an SAT score. It's what those numbers reflect about your ability to navigate everyday life. It has been said that life is 1 long intelligence test. And who can't relate to that?

Speaker 2

43:55

And if you doubt, see, another problem here is a lot of critics of intelligence research, intelligence testing tend to be academics who by and large are pretty smart people. And pretty smart people, by and large, have enormous difficulty understanding what the world is like for people with IQs of 80 or

Speaker 1

44:17

75.

Speaker 2

44:18

It is a completely different everyday experience. Even IQ scores of

Speaker 1

44:25

85, 90,

Speaker 2

44:27

you know, there's a popular television program, Judge Judy, where Judge Judy deals with everyday people with everyday problems, and you can see the full range of problem-solving ability demonstrated there. And sometimes she does it for laughs, But it really isn't funny because people who are... There are people who are very limited in their life navigation, let alone success, by not having good reasoning skills, which cannot be taught.

Speaker 2

45:06

We know this, by the way, because there are many efforts. You know, the United States military, which excels at training people. I mean, I don't know that there's a better organization in the world for training diverse people.

Speaker 1

45:20

And

Speaker 2

45:20

they won't take people with IQs under, I think 83 is the cutoff. Because they have found, they are unable to train people with lower IQs to do jobs in the military.

Speaker 1

45:34

So 1 of the things that g-factor has to do with is learning.

Speaker 2

45:37

Absolutely. Some people learn faster than others. Some people learn more than others. Now faster, by the way, is not necessarily better, as long as you get to the same place eventually.

Speaker 2

45:53

But, you know, there are professional schools that want students who can learn the fastest because they can learn more, or learn deeper, or all kinds of ideas about why you select people with the highest scores. And there's nothing funnier, by the way, to listen to a bunch of academics complain about the concept of intelligence and intelligence testing, and then you go to a faculty meeting where they're discussing who to hire among the applicants. And all they talk about is how smart the person is. We'll get

Speaker 1

46:28

to that, we'll sneak up to that in different ways, but there's something about reducing a person to a number that in part is grounded to the person's genetics that makes people very uncomfortable.

Speaker 2

46:38

But nobody does that. Nobody in the field actually does that. That is a worry that is a worry like, well, I don't want to call it a conspiracy theory.

Speaker 2

46:55

I mean, it's a legitimate worry, but it just doesn't happen. Now I had a professor in graduate school who was the only person I ever knew who considered the students only by their test scores.

Speaker 1

47:12

And

Speaker 2

47:12

later in his life, he kind of backed off that. But... Let me ask

Speaker 1

47:19

you this, so we'll jump around. I'll come back to it. I tend to, I've had political discussions with people and actually my friend Michael Malice, he's an anarchist.

Speaker 1