This is a clip from a conversation with Bjarne Stroustrup from Nov 2019. New full episodes are released once or twice a week and 1-2 new clips or a new non-podcast video is released on all other days. You can watch the full conversation here: https://www.youtube.com/watch?v=uTxRF5ag27A
(more links below)

Podcast full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4

Podcasts clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

Podcast website:
https://lexfridman.com/ai

Podcast on Apple Podcasts (iTunes):
https://apple.co/2lwqZIr

Podcast on Spotify:
https://spoti.fi/2nEwCF8

Podcast RSS:
https://lexfridman.com/category/ai/feed/

Note: I select clips with insights from these much longer conversation with the hope of helping make these ideas more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate YouTube channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the YouTube algorithm makes it very difficult for that channel to grow. So for a little while, I'll keep posting clips on this channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.

Bjarne Stroustrup is the creator of C++, a programming language that after 34 years is still one of the most popular and powerful languages in the world. Its focus on fast, stable, robust code underlies many of the biggest systems in the world that we have come to rely on as a society. If you're watching this on YouTube, many of the critical back-end component of YouTube are written in C++. Same goes for Google, Facebook, Amazon, Twitter, most Microsoft applications, Adobe applications, most database systems, and most physical systems that operate in the real-world like cars, robots, rockets that launch us into space and one day will land us on Mars.

Subscribe to this YouTube channel or connect on:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman

And then there's this idea of concepts that puts some...

was ever available in any form, but it puts some constraints on the stuff you can parameterize, essentially?

We have had versions of it that actually work for the last 4 or 5 years.

It was a design by Gabby Dos Reis, Drew Sutton and me.

We were professors and postdocs in Texas at the time.

And the implementation by Andrew Sutton has been available for that time.

And There's a standard library that uses it.

It's available in Clang and GCC, GCC for a couple of years, and I believe Microsoft is soon going to do it.

Expect all of C++ 20 to be available in all the major compilers in 20.

I'm just saying that because otherwise people might think I was talking about science fiction.

And so what I'm going to say is concrete.

So the basic idea is that when you have a generic component, like a sort function.

The sort function will require at least 2 parameters.

1, a data structure with a given type and a comparison criteria.

And these things are related, but obviously you can't compare things if you don't know what the type of things you compare.

And so, you want to be able to say, I'm going to sort something and it is to be sortable.

It has to have, it has to be a sequence with a beginning and an end.

There has to be random access to that sequence.

And there has to be, the element types has to be comparable.

Basically, what concepts are, they're compile time predicates.

They're predicates you can ask, are you a sequence?

Is your element type something that has a less than?

Yes, I have a less than, it's, and so basically that's the system.

And so instead of saying, I will take a parameter of any type, it'll say, I'll take something that's sortable.

And so we say, okay, you can sort with less than, I don't want less than, I want greater than or something I invent.

So you have 2 parameters, the sortable thing and the comparison criteria.

And the comparison criteria will say, well, you can write it saying it should operate on the element type and it has the comparison operations.

So it specifies the requirements of the code on the parameters that it gets.

The word concept was used by Alex Stefanov, who is sort of the father of generic programming in the context of C++.

There's other places that use that word, but the way we call generic programming is Alexis.

And he called them concepts because he said they're the sort of the fundamental concepts of an area, so they should be called concepts.

If you look at the K&R book about C, C has arithmetic types and it has integral types.

And then it lists what they are and they have certain properties.

The difference today is that we can actually write a concept that will ask a type, are you an integral type?

Do you have the properties necessary to be an integral type?

So maybe the story of concepts, because I thought it might be part of C++11, C-O-X, whatever it was at the time.

What was the, why didn't it, what, like what, we'll talk a little bit about this fascinating process of standards, because I think it's really interesting for people.

It's interesting for me, but why did it take so long?

What shapes did the idea of concepts take?

In 1987 or thereabouts, when I was designing templates, obviously I wanted to express the notion of what is required by a template of its arguments.

And basically for templates, I wanted 3 properties.

It had to be able to express things I couldn't imagine because I know I can't imagine everything and I've been suffering from languages that try to constrain you to only do what the designer thought good.

Secondly, it had to run faster, as fast or faster than handwritten code.

So basically, if I have a vector of T and I take a vector of char, it should run as fast as you build a vector of char yourself without parameterization.

And thirdly, I wanted to be able to express the constraints of the arguments, have proper type checking of the interfaces.

And neither I nor anybody else at the time knew how to get all 3.

And I thought for C++, I must have the 2 first.

And It bothered me for another couple of decades that I couldn't solve the third 1.

I mean, I was the 1 that put function argument type checking into C.

I didn't invent that idea, it's very common, but I did it.

And I wanted to do the same for templates, of course, and I couldn't.

Gabi Des Reys and I started analyzing the problem, explained possible solutions.

A group in University of Indiana, an old friend of mine, They started a project at Indiana and we thought we could get a good system of concepts in another 2 or 3 years.

Well it turned out that I think we got a lot of the fundamental ideas wrong.

Didn't serve implicit conversions very well.

It didn't serve mixed type arithmetic, mixed type computations very well.

A lot of stuff came out of the functional community and that community didn't deal with multiple types in the same way as C++ does, had more constraints on what you could express, and didn't have the Draconian performance requirements.

And basically we tried, we tried very hard.

We had some successes, but it just in the end wasn't, didn't compile fast enough, was too hard to use and didn't run fast enough unless you had optimizers that was beyond the state of the art, they still are.

Basically, it was the idea that a set of parameters has defined a set of operations and you go through an indirection table, just like for virtual functions, and then you try to optimize the indirection away to get performance.


Speaker 0: And then there's this idea of concepts that puts some... Now I've never even... I don't know if it

Speaker 1: was ever available in any form, but it puts some constraints on the stuff you can parameterize, essentially?

Speaker 2: Let me try and explain. Yes. So, yes, it wasn't there 10 years ago. We have had versions of it that actually work for the last 4 or 5 years. It was a design by Gabby Dos Reis, Drew Sutton and me.

We were professors and postdocs in Texas at the time. And the implementation by Andrew Sutton has been available for that time. And it is part of C++

Speaker 0: 20.

Speaker 2: And There's a standard library that uses it. So this is becoming really very real. It's available in Clang and GCC, GCC for a couple of years, and I believe Microsoft is soon going to do it. Expect all of C++ 20 to be available in all the major compilers in 20. But this kind of stuff is available now.

I'm just saying that because otherwise people might think I was talking about science fiction. And so what I'm going to say is concrete. You can run it today. And there's production uses of it. So the basic idea is that when you have a generic component, like a sort function.

The sort function will require at least 2 parameters. 1, a data structure with a given type and a comparison criteria. And these things are related, but obviously you can't compare things if you don't know what the type of things you compare. And so, you want to be able to say, I'm going to sort something and it is to be sortable. What does it mean to be sortable?

You look it up in the standard. It has to have, it has to be a sequence with a beginning and an end. There has to be random access to that sequence. And there has to be, the element types has to be comparable. By default, by less than.

Which means

Speaker 1: less than operator can

Speaker 2: operate on. Yes. Less than logical

Speaker 1: operator can operate.

Speaker 2: Basically, what concepts are, they're compile time predicates. They're predicates you can ask, are you a sequence? Yes, I have a begin and end. Are you a random exit sequence? Yes, I have a subscripting and plus.

Is your element type something that has a less than? Yes, I have a less than, it's, and so basically that's the system. And so instead of saying, I will take a parameter of any type, it'll say, I'll take something that's sortable. And it's well-defined. And so we say, okay, you can sort with less than, I don't want less than, I want greater than or something I invent.

So you have 2 parameters, the sortable thing and the comparison criteria. And the comparison criteria will say, well, you can write it saying it should operate on the element type and it has the comparison operations. So that's simply the fundamental thing. It's compile-time predicates. Do you have the properties I need?

So it specifies the requirements of the code on the parameters that it gets. It's very similar to types, actually.

Speaker 1: But operating in the space of concepts.

Speaker 2: Concepts? The word concept was used by Alex Stefanov, who is sort of the father of generic programming in the context of C++. There's other places that use that word, but the way we call generic programming is Alexis. And he called them concepts because he said they're the sort of the fundamental concepts of an area, so they should be called concepts. And we've had concepts all the time.

If you look at the K&R book about C, C has arithmetic types and it has integral types. It says so in the book. And then it lists what they are and they have certain properties. The difference today is that we can actually write a concept that will ask a type, are you an integral type? Do you have the properties necessary to be an integral type?

Do you have plus minus, divide and such?

Speaker 1: So maybe the story of concepts, because I thought it might be part of C++11, C-O-X, whatever it was at the time. What was the, why didn't it, what, like what, we'll talk a little bit about this fascinating process of standards, because I think it's really interesting for people. It's interesting for me, but why did it take so long? What shapes did the idea of concepts take? What were the challenges?

Back

Speaker 2: in 87 or thereabouts.

Speaker 0: 1987?

Speaker 2: In 1987 or thereabouts, when I was designing templates, obviously I wanted to express the notion of what is required by a template of its arguments. And so I looked at this. And basically for templates, I wanted 3 properties. I wanted to be very flexible. It had to be able to express things I couldn't imagine because I know I can't imagine everything and I've been suffering from languages that try to constrain you to only do what the designer thought good.

Didn't want to do that. Secondly, it had to run faster, as fast or faster than handwritten code. So basically, if I have a vector of T and I take a vector of char, it should run as fast as you build a vector of char yourself without parameterization. And thirdly, I wanted to be able to express the constraints of the arguments, have proper type checking of the interfaces. And neither I nor anybody else at the time knew how to get all 3.

And I thought for C++, I must have the 2 first. Otherwise it's not C++. And It bothered me for another couple of decades that I couldn't solve the third 1. I mean, I was the 1 that put function argument type checking into C. I know the value of good interfaces.

I didn't invent that idea, it's very common, but I did it. And I wanted to do the same for templates, of course, and I couldn't. So it bothered me. Then we tried again,

Speaker 0: 2002, 2003.

Speaker 2: Gabi Des Reys and I started analyzing the problem, explained possible solutions. It was not a complete design. A group in University of Indiana, an old friend of mine, They started a project at Indiana and we thought we could get a good system of concepts in another 2 or 3 years. That would have made C++ 11 to C++

Speaker 0: 06

Speaker 2: or

Speaker 0: 07.

Speaker 2: Well it turned out that I think we got a lot of the fundamental ideas wrong. They were too conventional. They didn't quite fit C++ in my opinion. Didn't serve implicit conversions very well. It didn't serve mixed type arithmetic, mixed type computations very well.

A lot of stuff came out of the functional community and that community didn't deal with multiple types in the same way as C++ does, had more constraints on what you could express, and didn't have the Draconian performance requirements. And basically we tried, we tried very hard. We had some successes, but it just in the end wasn't, didn't compile fast enough, was too hard to use and didn't run fast enough unless you had optimizers that was beyond the state of the art, they still are. So we had to do something else. Basically, it was the idea that a set of parameters has defined a set of operations and you go through an indirection table, just like for virtual functions, and then you try to optimize the indirection away to get performance.

And we just couldn't do all of that. You

See all Lex Fridman transcripts on Youtube

Bjarne Stroustrup: C++ Concepts - Constraints on Template Parameters