- The Piraha people do not have words for exact counting, not even for the number 1
- Edward Gibson, a psycholinguistics professor at MIT, discusses his fascination with human language and how he approached it from an engineering perspective
- He finds the structure and generalizations of human language to be beautiful and fascinating, with a focus on word order and minimizing dependencies between words
- He discusses the different word orders in languages and how they are connected to minimizing dependency links in communication

- The piece discusses how word order in language affects the length of dependencies between words
- It explains that all sentences can be broken down into a tree structure, with each word depending on another
- The root of the tree is usually the verb, and the structure can be automatable
- The concept of morphemes, the minimal meaning units within a language, is also discussed
- The differences in word order flexibility and case markers in languages like Russian are highlighted, as well as the use of linguistic terms like agent and patient for meaning

- The video discusses the evolution of language, focusing on the morphology of words and the development of color words in different cultures.
- It explains the concept of morphemes and how they are used to convey meaning in language.
- The discussion also touches on the differences in color vocabulary across cultures, with some cultures having only a few color words while others have a more extensive vocabulary.
- The video suggests that language evolves based on the needs of early tribes and their communication of solutions to problems, whether aesthetic or functional.

- The piece discusses the evolution of language from Old English to Modern English, highlighting the changes in word order and writing systems.
- It mentions the limited evidence of long-term language evolution for most languages, with examples from Mandarin and English.
- It also touches on the evolution of language on the internet, driven by humor and deviation from proper language.
- The discussion then moves to Noam Chomsky's work on syntax and grammar, particularly his proposal of phrase structure grammar and the concept of movement in English language.

- The piece discusses two theories of grammar proposed by Chomsky: one involving movement and the other involving lexical copying
- The movement theory suggests that words and phrases move from one structure to another, while the lexical copying theory suggests that words have two forms for declarative and interrogative sentences
- The lexical copying theory is seen as more advantageous for language learnability and predicts that not all auxiliaries might move
- The piece also touches on the differences between phrase structured grammar and dependency grammar, as well as the distinctions between regular, context-free, and context-sensitive languages
- Chomsky's theories were not proposed specifically for human languages, but they have implications for understanding the structure and learnability of language.

- Noam Chomsky proposed multiple theories on human language in 1957 and 1965, focusing on phrase structure and movement.
- In 1971, he concluded that language learning problems indicated an innate language ability.
- Chomsky's approach to language differs from David Poeppel's, who gathers data from experiments and corpus analyses.
- Poeppel prefers the dependency grammar framework for its transparent representation of word distance in sentences.

- The speaker discusses the complexity of nesting dependencies in language, using English as an example
- Long-distance dependencies are difficult to understand and produce in any language
- Experiments have shown that people struggle with completing sentences with nested dependencies
- Research has shown that all languages have short dependencies, regardless of word order
- The speaker proposes that the difficulty with long dependencies is a universal feature of language, and suggests possible explanations for this phenomenon

- The Piraha people lack words for exact counting
- Edward Gibson discusses his fascination with human language from an engineering perspective
- Human language structure and generalizations are beautiful and fascinating
- Different word orders in languages are connected to minimizing dependency links in communication
- Sentences can be broken down into a tree structure with each word depending on another
- The evolution of language is driven by the needs of early tribes and their communication of solutions to problems
- Noam Chomsky proposed theories on human language focusing on phrase structure and movement
- Long-distance dependencies in language are difficult to understand and produce
- All languages have short dependencies, regardless of word order

- The piece discusses the evolution of language and the purpose of language in terms of information theory.
- The speakers debate whether the primary objective of language optimization is for production or for the listener's understanding.
- They also discuss the relationship between language and thought, using MRI scans to study language networks in the brain.

- The speaker discusses the challenges of using brain scanners, especially with young children
- The left lateralized system for language development is difficult to study in young children
- The language network in the brain is separate from other cognitive tasks and is activated by both spoken and written language
- The language network is also activated by constructed languages and inner voices
- Evidence from stroke patients shows that language is separate from other cognitive abilities such as math and chess

- The piece discusses the relationship between language and symbolic processing, suggesting that language is not necessary for thinking.
- It explores the limitations of large language models in understanding the deeper meaning of language and their susceptibility to being easily fooled.
- The discussion also delves into construction-based theories of language and the form versus meaning aspect of language understanding.

- The discussion is about large language models and their ability to complete nested and center embedded structures.
- Large language models struggle with completing center embedded structures, similar to humans.
- There is a focus on the cognitive cost associated with longer distance connections in language, and how it can be measured and understood.
- The simplicity of the theory of language and its ability to explain complicated phenomena is highlighted.

- The video discusses the complexity of legal language and contracts, particularly the use of center embedding and low-frequency words.
- The researchers found that center embedding in legal language significantly affects comprehension and recall, and that lawyers also prefer simpler, non-center embedded versions of contracts.
- The use of center embedding in legal language may serve a performative function, similar to a "magic spell" that signals the truthfulness of the content.

- The piece discusses the complexity of legal contracts and the incentive for lawyers to make them hard to understand
- It explores the idea of "center embedding" and its impact on language comprehension
- The concept of noisy channels in communication is explained, with a focus on language optimization for passing messages
- The work of Claude Shannon in communication theory and his interest in language is mentioned
- The potential impact of word order and syntax on language robustness in noisy channels is considered, although it is acknowledged as speculative
- The different sources of noise in communication, including background noise and cognitive load, are discussed

- The piece discusses the evolution and purpose of language in terms of information theory
- Debate over whether language optimization is for production or listener's understanding
- MRI scans used to study language networks in the brain, especially in young children
- Evidence that language is separate from other cognitive abilities
- Limitations of large language models in understanding language's deeper meaning
- Complexity of legal language and contracts, including the use of center embedding
- Noisy channels in communication and the impact of word order and syntax on language robustness
- Sources of noise in communication, including background noise and cognitive load

- The speaker discusses various factors that affect language learning, such as age, language structure, and cultural influences
- They question the role of innate language structure and propose that language can be learned from input
- They mention natural experiments where people with brain damage still exhibit normal language abilities
- The speaker also raises the idea that language development may be separate from thought, contrary to Chomsky's belief that language underpins thought

- The speaker discusses the concept of language being separate from thought
- They mention the importance of studying non-industrialized cultures to understand the connection between language and culture
- The speaker has worked with isolated language groups in the Amazon, specifically the Chimani and the Piraha
- The Piraha language lacks words for exact counting, leading to the conclusion that their language is based on approximate quantifiers rather than exact numbers

- The Pirahã people can perform exact matching tasks without the need for counting or words
- They struggle with tasks that require encoding sets, such as counting objects behind an opaque sheet
- The ability to count and have words for exact count is a limiting factor for the Pirahã people
- Language serves a function in a community, and the value of a language is tied to economic opportunities and national identity
- There is a tension between the convenience of language for trade and communication, and the preservation of language as a symbol of identity and culture

- The speaker discusses the challenges of machine translation and the cultural and linguistic barriers it faces.
- They mention the difficulty of translating concepts that exist in one language but not in another.
- The speaker also discusses the potential for communication with animals and plants, and the need for intellectual humility in understanding different forms of communication.
- The conversation concludes with the advice to pursue interests and opportunities as they arise.

- The speaker visited Piedmont and had trouble returning due to corrupt officials in Brazil
- They found the Chamani group and continued their work
- The speaker values ambition and trying new things
- The conversation was enjoyable and the speaker thanks the interviewer
- The podcast is supported by sponsors
- The episode ends with a quote from Wittgenstein

- Factors affecting language learning: age, language structure, and cultural influences
- Questioning the role of innate language structure and proposing language can be learned from input
- Natural experiments showing normal language abilities in people with brain damage
- Language development may be separate from thought, contrary to Chomsky's belief
- Importance of studying non-industrialized cultures to understand language and culture connection
- Work with isolated Amazon language groups, specifically the Chimani and the Piraha
- Piraha language lacks exact counting words, based on approximate quantifiers
- Pirahã people can perform exact matching tasks without counting or words, but struggle with encoding sets
- Language's function in a community, tied to economic opportunities and national identity
- Tension between language convenience for trade and preservation as a symbol of identity and culture
- Challenges of machine translation and cultural/linguistic barriers
- Potential for communication with animals and plants, emphasizing intellectual humility
- Advice to pursue interests and opportunities, based on speaker's experiences in Piedmont and with the Chamani group
- Speaker values ambition and trying new things, thanks interviewer, and ends with a quote from Wittgenstein

Edward Gibson is a psycholinguistics professor at MIT and heads the MIT Language Lab. Please support this podcast by checking out our sponsors: – Yahoo Finance: https://yahoofinance.com – Listening: https://listening.com/lex and use code LEX to get one month free – Policygenius: https://policygenius.com/lex – Shopify: https://shopify.com/lex to get $1 per month trial – Eight Sleep: https://eightsleep.com/lex to get special savings Transcript: https://lexfridman.com/edward-gibson-transcript EPISODE LINKS: Edward’s X: https://x.com/LanguageMIT TedLab: https://tedlab.mit.edu/ Edward’s Google Scholar: https://scholar.google.com/citations?user=4FsWE64AAAAJ TedLab’s YouTube: https://youtube.com/@Tedlab-MIT PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ YouTube Full Episodes: https://youtube.com/lexfridman YouTube Clips: https://youtube.com/lexclips SUPPORT & CONNECT: – Check out

Naively, I certainly thought that all humans would have words for exact counting, and the Piraha don't.

Okay, so they don't have any words for even 1.

There's not a word for 1 in their language.

And so there's certainly not a word for 2, 3, or 4.

So that kind of blows people's minds off.

How are you gonna ask, I want 2 of those?

And so that's just not a thing you can possibly ask in the Pewterhau.

The following is a conversation with Edward Gibson, or Ted, as everybody calls him.

He is a psycholinguistics professor at MIT.

He heads the MIT Language Lab that investigates why human languages look the way they do, the relationship between cultural language and how people represent, process, and learn language.

Also, he should have a book titled Syntax, A Cognitive Approach, published by MIT Press coming out this fall.

To support it, please check out our sponsors in the description.

And now, dear friends, here's Edward Gibson.

When did you first become fascinated with human language?

As a kid in school, when we had to structure sentences in English grammar, I found that process interesting.

I found it confusing as to what it was I was told to do.

I didn't understand what the theory was behind it, but I found it very interesting.

So when you look at grammar, you're almost thinking about like a puzzle, like almost like a mathematical puzzle.

I didn't know I was gonna work on this at all at that point.

I was really just, I was kind of a math geek person, computer scientist, I really liked computer science.

And then I found language as a neat puzzle to work on from an engineering perspective, actually.

That's what I, as a, I sort of accidentally, I decided after I finished my undergraduate degree, which was computer science and math and Canada and Queens University, I decided to go to grad school.

It's like, that's what I always thought I would do.

And I went to Cambridge where they had a master's in, a master's program in computational linguistics.

And I hadn't taken a single language class before.

All I'd taken was CS, computer science, math classes, pretty much, mostly, as an undergrad.

And I just thought this was an interesting thing to do for a year, because it was a single year program.

And then I ended up spending my whole life doing it.

So fundamentally your journey through life was 1 of a mathematician and a computer scientist.

And then you kind of discovered the puzzle, the problem of language, and approached it from that angle, to try to understand it from that angle, almost like a mathematician or maybe even an engineer.

As an engineer, I'd say, I mean, to be frank, I had taken an AI class, I guess it was 83 or 84, 85, somewhere 84 in there a long time ago, and there was a natural language section in there, and it didn't impress me.

I thought there must be more interesting things we can do.

It didn't seem very, it seemed just a bunch of hacks to me.

It didn't seem like a real theory of things in any way.

And so I just thought this seemed like an interesting area where there wasn't enough good work.

Did you ever come across the philosophy angle of logic?

So if you think about the 80s with AI, the expert systems where you try to kind of maybe sidestep the poetry of language and some of the syntax and the grammar and all that kind of stuff and go to the underlying meaning that language is trying to communicate and try to somehow compress that in a computer representable way.

Do you ever come across that in your studies?

I mean, I probably did, but I wasn't as interested in it.

I was trying to do the easier problems first, the ones I could, thought maybe were handleable, which seems like the syntax is easier, like which is just the forms as opposed to the meaning.

Like when you're starting to talk about the meaning, that's a very hard problem, and it still is a really, really hard problem.

But the forms is easier, and so I thought at least figuring out the forms of human language, which sounds really hard, but is actually maybe more tractable.

So it's interesting, you think there is a big divide, there's a gap, there's a distance between form and meaning.

Because that's a question you have discussed a lot with LLMs, because they're damn good at form.

Yeah, I think that's what they're good at, is form.

Exactly, and that's why they're good, because they can do form, meaning's hard.

How close form and meaning are, but we'll discuss it.

But to me, studying form, maybe it's a romantic notion, gives you, form is like the shadow of the bigger meaning thing underlying language.

We communicate with each other using language.

So in understanding the structure of that communication, I think you start to understand the structure of thought and the structure of meaning behind those thoughts and communication to me.

What do you find most beautiful about human language?

Maybe the form of human language, the expression of human language.

What I find beautiful about human language is some of the generalizations that happen across the human languages, within and across a language.

So let me give you an example of something which I find kind of remarkable, that is if a language, if it has a word order such that the verbs tend to come before their objects.

So we have the first, the subject comes first in a simple sentence.

So I say, you know, the dog chased the cat or Mary kicked the ball.

So the subject's first, and then after the subject, there's the verb, and then we have objects.

So it's generally a verb, and most of the stuff that we wanna say comes after the subject.

It's the objects, there's a lot of things we wanna say that come after.

And there's a lot of languages like that.

About 40% of the languages of the world look like that.

And then these languages tend to have prepositions, these little markers on the nouns that connect nouns to other nouns or nouns to verbs.

So a preposition like in or on or of or about, I say I talk about something, the something is the object of that preposition.

We have these little markers come, just like verbs, they come before their nouns.

Okay, and then, so, now we look at other languages, like Japanese or Hindi, or some, these are so-called verb final languages.

Those is about, maybe a little more than 40%, maybe 45% of the world's languages or more, I mean 50% of the world's languages are verb final.

Those markers, the States have the same kinds of markers as we do in English, but they put them after.

So, sorry, they put them first, the markers come first.

So you say, instead of, you know, talk about a book, you say, book about, the opposite order there, in Japanese or in Hindi, You do the opposite.

And the talk comes at the end, so the verb will come at the end as well.

So instead of, Mary kicked the ball, it's Mary ball kicked.

And then it's Mary kicked the ball to John, it's John to.

The to, the marker there, the preposition, it's a postposition in these languages.

And so the interesting thing, a fascinating thing to me is that within a language, this order aligns, it's harmonic.

And so if it's 1 or the other, it's either verb initial or verb final, but then you'll have prepositions, prepositions, or postpositions.

And that's across the languages that we can look at.

We've got around 1,000 languages, There's around 7,000 languages on the earth right now, but we have information about, say, word order on around 1,000 of those, pretty decent amount of information.

And for those 1,000 which we know about, about 95% fit that pattern.

So they will have either verb, it's about half and half, half of verb initial, like English, and half of verb final, like Japanese.

Just to clarify, verb initial is subject-verb-object.

Verb final is still subject, object, verb.

Okay, and it's fascinating that there's a pretty even division in the world amongst those, 45%.

Those 2 word orders, the subject tends to be first.

There's so many interesting things, but these things are, the thing I find so fascinating is there are these generalizations within and across a language.

And not only those, and there's actually a simple explanation, I think, for a lot of that, and that is, you're trying to minimize dependencies between words.

That's basically the story, I think, behind a lot of why word order looks the way it is, is we're always connecting.

I'm talking to you in sentences, you're talking to me in sentences.

These are sequences of words which are connected.

And the connections are dependencies between the words.

And it turns out that what we're trying to do in a language is actually minimize those dependency links.

It's easier for me to say things if the words that are connecting for their meaning are close together.

It's easier for you in understanding if that's also true.

If they're far away, it's hard to produce that and it's hard for you to understand.

And the languages of the world, within a language and across languages, fit that generalization.

So It turns out that having verbs initial and then having prepositions ends up making dependencies shorter.

And having verbs final and having postpositions ends up making dependencies shorter than if you cross them.

If you cross them, it ends up, you just end up, it's possible, you can do it.

It just ends up with longer dependencies than if you didn't.

So it was observed a long time ago, without the explanation, by a guy called Joseph Greenberg, who's a famous typologist from Stanford.

He observed a lot of generalizations about how word order works, and these are some of the harmonic generalizations that he observed.

Harmonic generalizations about word order.

Well, what I mean is, in language, there's kind of 3 structures to, 3 components to the structure of language.

I'm talking, then there's 2 meaning parts, and those are the words, and you were talking about meaning earlier.

So words have a form, and they have a meaning associated with them.

And so cat is a full form in English and it has a meaning associated with whatever a cat is.

And then the combinations of words, that's what I'll call grammar or syntax.

And That's like when I have a combination like the cat or 2 cats, okay?

So where I take 2 different words there and put them together and I get a compositional meaning from putting those 2 different words together.

And In any sentence or utterance, whatever I'm talking to you, you're talking to me, we have a bunch of words and we're putting together in a sequence.

It turns out they are connected so that every word is connected to just 1 other word in that sentence.

And so you end up with what's called technically a tree.

It's a tree structure, so where there's a root of that utterance, of that sentence, and then there's a bunch of dependents, like branches from that root that go down to the words.

The words are the leaves in this metaphor for a tree.

So a tree is also sort of a mathematical construct.

Yeah, yeah, it's a graph theoretical thing.

So, and it's fascinating that you can break down a sentence into a tree, and then 1, every word is hanging on to another, it's depending on it.

That's right, and everyone agrees on that.

No, I think in every language, I think everyone agrees that all sentences are trees at some level.

Because to me, just as a layman, it's surprising that you can break down sentences in most all languages into a tree.

I've never heard of anyone disagreeing with that.

The details of the trees are what people disagree about.

Well, okay, so what's at the root of a tree?

How do you construct, How hard is it, what is the process of constructing a tree from a sentence?

is where, depending on what your, there's different theoretical notions.

I'm gonna say the simplest thing, dependency grammar.

It's like a bunch of people invented this.

Tenier was the first French guy back in, I mean, the paper was published in 1959, but he was working on the 30s and stuff.

And it goes back to, you know, philologist Pignini was doing this in ancient India, okay?

And so, you know, doing something like this.

The simplest thing we can think of is that there's just connections between the words to make the utterance.

And so let's just say I have like 2 dogs entered a room.

And so we're connecting 2 and dogs together.

That's like there's some dependency between those words to make some bigger meaning.

And then we're connecting dogs now to entered, right?

And we connect a room somehow to entered.

And so I'm gonna connect to room and then room back to entered.

That's the tree, is the root is entered, that's the thing is like an entering event, that's what we're saying here, and the subject, which is whatever that dog is, is 2 dogs, it was, and the connection goes back to dogs, which goes back to, then that goes back to 2.

It starts at entered, goes to dogs, down to 2, and on the other side, after the verb, the object, it goes to room, and then that goes back to the determiner or article, whatever you wanna call that word.

So there's a bunch of categories of words here, we're noticing.

Those are these things that typically mark, they refer to events and states in the world.

There are nouns which typically refer to people, places and things is what people say, but they can refer to other more, they can refer to events themselves as well.

They're marked by, you know, how they get used, the category, the part of speech of a word is how it gets used in language.

It's like, that's how you decide what the category of a word is.

Not by the meaning, but how it gets used.

Is it gonna be the verb that defines the event?

I mean, if I don't say a verb, then there won't be a verb, and so it'll be something else.

What if you're messing, are we talking about language that's like correct language?

What if you're doing poetry and messing with stuff?

Is it, then rules go out the window, right?

You're constrained by whatever language you're dealing with.

Probably you have other constraints in poetry, such that you're, like Usually in poetry, there's multiple constraints that you want to, like you want to usually convey multiple meanings is the idea, and maybe you have like a rhythm or a rhyming structure as well.

And depending, but you usually are constrained by the rules of your language for the most part.

You can violate them somewhat, but not too much.

So it has to be recognizable as your language.

Like in English, I can't say, dogs 2 entered room ah.

I mean, I meant that, you know, 2 dogs entered a room.

And I can't mess with the order of the articles, the articles and the nouns, you just can't do that.

In some languages, you can mess around with the order of words much more.

Russian has a much freer word order than English.

And so in fact, you can move around words in, you know, I told you that English has a subject, verb, object, word order, so does Russian, but Russian is much freer than English.

And so you can actually mess around with the word order.

So probably Russian poetry is gonna be quite different from English poetry because the word order is much less constrained.

Yeah, there's a much more extensive culture of poetry throughout the history of the last hundred years in Russia.

But it seems that there's more flexibility in the way the language is used.

You're morphing the language easier by altering the words, altering the order of the words, and messing with it.

Well, you can just mess with different things in each language.

And so in Russian you have case markers, which are these endings on the nouns which tell you how each noun connects to the verb, right?

And so when I say, Mary kissed John, I don't know who the agent or the patient is except by the order of the words, right?

In Russian, you actually have a marker on the end if you're using a Russian name, and each of those names you'll also say is that, you know, it'll be the, you know, nominative, which is marking the subject, or an accusative will mark the object.

And you could put them in the reverse order.

You could put accusative first, you could put subject, you could put the patient first, and then the verb, and then the subject, and that would be a perfectly good Russian sentence, and it would still mean, I could say John kissed Mary, meaning Mary kissed John, as long as I use the case markers in the right way.

I love the terminology of agent and patient.

Those are sort of linguistic terms, correct?

Those are, those are for like kind of meaning.

And subject and object are generally used for position.

So subject is just like the thing that comes before the verb and the object is the 1 that comes after the verb.

The agent is kind of like the thing doing it, that's kind of what that means, right?

The subject is often the person doing the action, right?

So how hard is it to form a tree in general?

See all Unknown transcripts on Generic

#426 – Edward Gibson: Human Language, Psycholinguistics, Syntax, Grammar & LLMs | Lex Fridman Podcast