Twitter Space participated by Elon Musk, Kim Dotcom, Brian Krassenstein, ALX 🇺🇸, Whole Mars Catalog, Walter Isaacson, Greg Yang, Christian Szegedy, Yuhuai (Tony) Wu, Guodong Zhang, Toby Pohlen, Zihang Dai, Jimmy Ba, Manuel Kroiss, Kyle Kosic, Robert Scoble

You you you you you you you You You You You You You You You You You You You Silence.

We're just waiting for everyone who wants to join the space to join.

We need to tweak the algorithm a little bit.

Before you recommendation, spaces needs to have higher immediacy in recommendations for obvious reasons.

So We're just giving everyone a minute to be aware of the space.

And we're going to just adjust the four-year algorithm to have higher immediacy for spaces, especially large spaces.

So we're just going to probably start in about 2 minutes.

I'll make sure I don't get any dust on my hands.

So let's see, I'll just do a brief introduction of the company and then the founding team will I think just say a few words about their background, things they've worked on, whatever they'd like to talk about really.

But I think it's helpful to hear from people in their own words, the various things they've worked on and what they wanna do with XAI.

So I guess the overarching goal of XAI is to build a A good AGI with the overarching purpose of just trying to understand the universe.

I think the safest way to build an AI is actually make 1 that is maximally curious and truth-seeking.

So you go for trying to aspire to the truth with with acknowledged error.

So like, you know, this will never actually get fully to the truth.

It's not clear, but you want to always aspire to that and try to minimize the error between what it you know, what you think is true and what is actually true.

My sort of theory behind the maximally curious, maximally truthful, as being probably the safest approach is that I think to a superintelligence, humanity is much more interesting than not humanity.

You know, 1 can look at the various planets in our solar system, the moons and the asteroids, and really probably all of them combined are not as interesting as humanity.

I mean, as people know, I'm a huge fan of Mars, next level.

I mean, the middle name of 1 of my kids is basically the Greek word for Mars.

So I'm a huge fan of Mars, but Mars is just much less interesting than Earth with humans on it.

And so I think that that kind of approach to growing an A.I.

And I think that is the right word for it.

Safety and I've been 1 of the strongest voices calling for AI regulation or oversight to just have some kind of oversight, some kind of referee, so that it's not just up to companies to decide what they want to do.

I think there's also a lot to be done with AI safety, with industry cooperation, kind of like motion pictures association.

So I think there's value to that as well.

But I do think there's gotta be some, like in any kind of situation that is, even if it's a game, they have referees.

So I think it is important for there to be regulation.

And then, like I said, my view on safety is try to make it maximally curious, maximally truth-seeking.

And I think this is important to avoid the inverse morality problem.

Like if you try to program a certain morality, you can basically invert it and get the opposite what is sometimes called the Waluigi problem.

If you make Luigi, you risk creating Waluigi at the same time.

So I think that's a metaphor that a lot of people can appreciate.

So that's what we're going to try to do here.

With that, I think, let me turn it over to Igor.

I was actually originally a physicist, so I studied physics at university.

And I briefly worked at the Large Hadron Collider at CERN.

So understanding the universe is something I've always been very passionate about.

And once some of these really impressive results from deep learning came out, like AlphaGo, for example, I got really interested in machine learning and AI and decided to make a switch into that field.

Then I joined DeepMind, worked on various projects, including AlphaStar.

So that's where we tried to teach a machine learning agent to play the game StarCraft

through self-play, which was a really, really fun project.

Then later on, I joined OpenAI, worked on various projects there, including GPD

So I was very, very passionate about language models, making them do impressive things.

Yeah, now I've teamed up with Elon to see if we can actually deploy these new technologies to really make a dent in our understanding of the universe and progress our collective knowledge.

Yeah, actually, I had a similar background.

My 2 best subjects were computer science and physics.

And I actually thought about a career in physics for a while because physics is really just trying to understand the fundamental truths of the universe.

And then I was a little concerned that I would get stuck at a collider and then the collider might get cancelled because of some arbitrary government decision.

So that's actually why I decided not to pursue a career in physics.

So focused initially more on computer science and then obviously later got back into physical objects with SpaceX and Tesla.

So I'm a big believer in pursuing physics and information theory as the sort of 2 areas that really help you understand the nature of reality.

Before joining XAI, I was previously at Diepmann for the past 6 years, where I worked on the reinforcement learning team, and I mostly focused on the engineering side of building these large reinforcement learning agents, like for example, AlphaStar together with Igor.

In general, I've been excited about AI for a long time.

For me, it has the potential to be the ultimate tool to solve the hardest problems.

I first studied bioinformatics, but then became also more excited about the AI, because if you have a tool that can solve all the problems, to me that's just much more exciting.

And with XAI in particular, I'm excited about doing this in a way where we built tools for people and we share them with everybody so that people can do their own research and understand things.

And my hope is that it was like a new wave of researchers that wasn't there before.

Yeah, so I'm Christian, I mean, Christian Szegedi, so we decided to switch places with Tony because I wanted to talk a bit about the role of mathematics in understanding the universe.

So I have worked for the past 7 years on trying to create an AI that is as good in mathematics as any human.

the reason for that is that mathematics is the language of, is basically the language of pure logics.

And I think that mathematics and logical reasoning at a high level would demonstrate that the AI is really understanding things not just emulating humans.

It would be instrumental for programming and physics in the long run.

So I think as AI that starts to show real understanding of deep reasoning is crucial for our first steps to understand the universe.

Same to Christian, my dream has been to tackle the most difficult problems in mathematics with artificial intelligence.

That's why we became such a cool friends and long-term collaborators.

So achieving that is definitely a very ambitious goal.

Last year, We've been making some really interesting breakthroughs, which made us really convinced that we're not far from our dream.

So I believe with such a talented team and abundant resources, I'm super hopeful that we will get there.

it's worth just mentioning, I think generally people are reluctant to be self-promotional, but I think it is important that people hear, like, what are the things that you've done that are noteworthy?

So basically, brag a little is what I'm saying.

Yeah, so last year, I think we've made some really interesting progress in the field of AI for math.

Specifically, with some team at Google, we built this agent called Minerva, which is actually able to achieve very high scores in high school exams, actually higher than average high school students.

So that actually is a very big motivation for us to push this research forward.

Another piece of work that we've done is also to convert natural language mathematics into formalized mathematics, which gives you a very grounding of the facts and reasoning.

And last year, we also made very interesting progress in that direction as well.

So now we are pushing almost a hybrid approach of these 2 in this new organization.

And we are very hopeful we will make our dream

So I taught at University of Toronto, and some of you probably have taken my course last couple of months.

I've been a C4AI chair and Sloan Fellow in Computer Science.

So I guess my research pretty much have touched on every aspect of deep learning left every stones turn, and has been pretty lucky to come with a lot of fundamental building blocks for the modern transformers and empowering the new wave of deep learning revolution.

My long-term research ambition, very fortunately, aligns with this very strong XAI team very well.

That is, how can we build a general purpose problem-solving machines to help all of us, the humanity, to overcome some of the most challenging and ambitious problems out there?

How can we use this tool to augment ourself and empower everyone.

So I'm very excited to embark on this new journey.

I started coding at a very young age when my dad taught me some basic and then throughout my youth I continued coding.

When I got to uni I got really into mathematics and machine learning.

Initially my research focused mostly on computer vision.

And then I joined DeepMind 6 years ago, where I worked on imitation learning and reinforcement learning, and learned a lot about distributed systems and research at scale.

Now I'm really looking forward to implementing products and features that bring the benefits of this technology to really all members of society.

And I really believe that having the AI nice and accessible and useful will be a benefit to all of us.

I'm a Distributed Systems Engineer at XAI.

Like some of my colleagues here, I started off my career in math and applied physics as well.

And gradually found myself working through some tech startups.

I worked at a startup a couple years ago called OnScale where we did physics simulations on HPCs.

And then most recently, I was at OpenAI working on HPC problems there as well.

Specifically, I worked on the GPT-4 project.

And the reason I'm particularly excited about XAI is that I think that the biggest danger of AI really is monopolization by a couple of entities.

I think that when you involve the amount of capital that's required to train these massive AI models that the incentives are not necessarily aligned with the rest of humanity.

And I think that the chief way of really addressing that issue is introducing competition.

And so I think that XAI really provides a unique opportunity for engineers to focus on the science, the engineering, and the safety issues directly without really getting as involved and sidetracked by political and social trends du jour.

So that's why I'm excited by XAI and I'm gonna go ahead and hand it off now to my colleague Greg who should be on the line as well.

I work on the mathematics and science of deep learning.

So my journey really started 10 years ago.

And so I was pretty good at math and took math 55 and, you know, did all kinds of stuff.

But after 2 years of college, I was just kind of like tired of being in the hamster wheel of, you know, taking the path that everybody else has taken.

So I did something unimaginable before, which was I took some time off from school and became a DJ and producer.

OK, so the side effect of taking some time off from school was that I was able to think a bit more about myself, to understand myself and to understand the world at large.

So, you know, I was grappling with questions like, what is free will?

You know, what does quantum physics have to do with the reality of the universe, and so on and so forth.

What does the Gerdau's incompleteness theorem says?

And after this period of intense self-introspection, I figured out what I want to do in life is not to be a DJ necessarily, maybe that's the second dream, but first and foremost, I wanted to make AGI happen.

I wanted to make something smarter than myself and kind of like, and be able to iterate on that and contribute and see so much more of our fundamental reality than I can in my current form.

And then I started, you know, and then I, you know, I realized that mathematics is the language underlying all of our reality and all of our science.

And to make fundamental progress, it really pays to know math as well as possible.

So I essentially started learning math from the very beginning just by reading from the textbooks.

Like in the first, some of the first few books I read, kind of going, restarting from scratch, is like Naive Set Theory by Helmholtz, or, you know, linear algebra done right by Ackler.

And then slowly I scaled up to algebraic geometry, algebraic topology, category theory, real analysis, measure theory, so on and so forth.

At the end, I think my goal at the time was I should be able to speak with any mathematician in the world and be able to hold a conversation to understand their contributions for 30 minutes.

from school and then somehow from there, I got a job at Microsoft Research.

And for the past 5 and a half years, I worked at Microsoft Research, which was amazing environment that enabled me to make a lot of foundational contribution toward the understanding of large scale neural networks.

In particular, I think my most well known work nowadays are about really wide neural networks and how we should think about them.

And so this is the framework called tensor programs.

And from there, I was able to derive this thing called MUP that perhaps the large language model builders know about, which allows 1 to extrapolate the optimal hyper parameters for a large model from understanding or the tuning of small neural networks.

And, and this is able to, you know, create create a lot of ensure the quality of the model is very good as we scale up.

Yeah, so looking forward, I'm really, really excited about XAI and also about the time that we're in right now, where I think, not only are we approaching AGI, but from a scientific perspective, we're also approaching a time where, like, neural networks, the science and mathematics and neural networks feels just like the turn of the 20th century in the history of physics, where we suddenly discover quantum physics and general relativity, which has some beautiful mathematics and science behind it.

And I'm really excited to be in the middle of everything.

And like Christian and Tony said, I am also very excited about creating in AI that is as good as myself or even better at creating new mathematics and new science that helps all achieve and see further into our fundamental reality.

And I work on large neural network training.

So this is also my kind of focus at XAI as well.

And before that, I was at a team mind working on the German projects and the leading the automation part.

And also I did my PhD at the University of Toronto.

Right now, teaming up with other Lectifunny members, I'm so excited about this effort.

So without doubt, AI is clearly a defining technology for our generation.

So I think it's important for us to make sure it ends up being a net of quality for humanity.

So at SAI, I not only want to train good models, but also understand how they behave and how they scales, and then use them to solve some of the hardest problems humanity has.

So actually I started in business school for my undergrad And I spent 10 years to get where I am now.

I got my PhD at Carnegie Mellon, and I was in Google before joining the team.

My first work was mostly about how to better utilize unlabeled data, how to improve transformer architecture, and how to really push the best technology into real-world usage.

So I believe in hard work and consistency.

So with XII, I'll be digging into the deepest details of some of the most challenging problems.

For myself, there are so many interesting things I don't understand, but I wanna understand.

So I will build something to help people who just share that dream or that feeling.

So I've worked on building and scaling large scale distributed systems for most of my life, starting out at National Labs and then kind of moving on to Palantir, Tesla, and a brief stint at Twitter.

And now I'm really excited about working on doing the same thing at XAI.

So mostly experience, you know, scaling large GPU clusters, custom basics, data centers, high speed networks, file systems, power cooling, manufacturing, pretty much all the things.

I'm basically a generalist that really loves learning, you know, physics, science fiction, math, science, cosmology.

I'm kind of looking to really, I guess really excited about the mission that XAI has and basically solving the most fundamental questions in science and engineering and also kind of helping us create tools to ask the right questions in the Douglas Adams mindset.

Is there anything anyone would like to add or kick off the discussion with?

There was like a lot of discussion around the vision statement that is a bit vague.

vague and ambitious and not concrete enough.

Well, I don't disagree with that position, obviously.

I mean, I understand the universe is the entire purpose of physics.

That's just so much that we don't understand right now.

Or we think we understand, but actually we don't in reality.

So there's still a lot of unresolved questions that are extremely fundamental.

This whole dark matter, dark energy thing is really, I think, an unresolved question.

We have the standard model, which is proved to be extremely good at predicting things, very robust, but still many questions remaining about the nature of gravity, for example.

There's the Fermi paradox of where are the aliens?

billion years old, why is there not massive evidence of aliens?

And people often ask me, since I am obviously deeply involved in space that, you know, if anyone would know about, would have seen evidence of aliens, it's probably me.

And yet I have not seen even 1 tiny shred of evidence for aliens, nothing, 0.

And I would jump on it in a second if I saw it.

There are many explanations for the Fermi paradox, but which 1 is actually true?

Or Maybe none of the current theories are true.

So I mean, the furry paradox is, which is really just like, where the hell are the aliens, is Part of what gives me concern about the fragility of civilization and consciousness as we know it, since we see no evidence thus far of it anywhere, and we've tried hard to find it, we may actually be the only thing, at least in this galaxy or this part of the galaxy.

If so, it suggests that what we have is extremely rare.

And I think it would be wise to assume that consciousness is extremely rare.

It's worth noting for the evolution of consciousness on Earth that we're about, Earth is about 4.5 billion years old.

It will expand to heat up Earth to the point where it will effectively boil the oceans.

You'll get a runaway, you know, next level greenhouse effect.

And Earth will become like Venus, which really cannot support life as we know it.

So the sun doesn't need to expand to envelop Earth, it just needs to make things hot enough to increase the water vapor in the air to the point where you get a runaway greenhouse effect.

So for argument's sake, it could be that if life, if consciousness had taken 10% longer than Earth's current existence, it wouldn't have developed at all.

So on a cosmic scale, this is a very narrow window.

Anyway, so there are all these fundamental questions.

I don't think you can call anything AGI until it has solved at least 1 fundamental question.

Because humans have solved many fundamental questions or substantially solved them.

And so if the computer can't solve even 1 of them, I'm like, okay, it's not as good as humans.

You know, where's that RIMA and Hypothesis solution?

It would be great to know what the hell is really going on, essentially.

So I guess you could reformulate the XAI mission statement as what the hell is really going on?

I think there's also, at least for me, a nice aspirational aspect to the mission statement, namely that of course in the short run we're working on more well understood like deep learning technologies, but I think in everything we do we should always bear in mind that we aren't just supposed to build, we're also supposed to understand.

So pursuing the science of it is really fundamental to what we do.

And this is also encompassed in this mission statement of understanding.

Yeah, I want to also add that, you know, we've essentially been mostly talking about creating a really smart agent that can help us understand the universe better.

But also from my viewpoint, my vantage point, when I'm doing the, I'm discovering the mathematics of, you know, large new networks, I can also see that there are, the mathematics here can actually also open up new ways of thinking about fundamental physics or about other kinds of reality because the, you know, for example, like a, you know, a large neural network with no nonlinearities, roughly like classical random matrix theory, and that has a lot of connections with gauge theory in high energy physics.

So in other words, as we're trying to understand neural networks better from a mathematical point of view, that can also lead to really good, very interesting perspectives on some existing questions, like, you know, the theory, everything, what is quantum gravity, so on and so forth.

But of course this is all speculative right now.

I see some patterns but I don't have anything concrete to say.

But again, this is like another perspective to understanding the universe.

By the way, by understand the universe, we don't just mean that we want to understand the universe.

We also want to make it easy for you to understand the universe.

Get a better sense of reality and to learn and take advantage of the internet or the knowledge that's out there.

So we are pretty passionate about actually releasing tools and products pretty early, involving the public.

We're not gonna understand the universe and not tell anyone.

I mean, when I think about neural networks today, it's currently the case that if you have 10 megawatts of GPUs, which really should be renamed something else because there's no graphics there.

But if you get 10 megawatts of GPUs cannot currently write a better novel than a good human.

That any good Humans using roughly 10 watts of higher order brain power.

So not counting the basic stuff to, you know, operate your body.

So, so there we got, we got a 6 order magnitude difference.

See all xAI transcripts on Twitterspaces

xAI