0
SPEAKER_01
The following is a conversation with Chris Lattner, his third time on this podcast.
0
SPEAKER_01
As I ve said many times before, he s one of the most brilliant engineers in modern computing, having created LLM Compiler Infrastructure Project, the Clang Compiler, the Swift Programming Language, a lot of key contributions to TensorFlow and TPUs as part of Google.
0
SPEAKER_01
He served as Vice President of Autopilot Software at Tesla.
0
SPEAKER_01
a software innovator and leader at Apple, and now he co-created a new full-stack AI infrastructure for distributed training, inference, and deployment on all kinds of hardware called Modular, and a new programming language called Mojo that is a superset of Python, giving you all the usability of Python, but with the performance of C, C++.
0
SPEAKER_01
In many cases, Mojo code has demonstrated over 30,000x speed up over Python.
0
SPEAKER_01
If you love machine learning, if you love Python, you should definitely give Mojo a try.
0
SPEAKER_01
This programming language, this new AI framework and infrastructure, and this conversation with Chris is mind blowing.
0
SPEAKER_01
I love it.
0
SPEAKER_01
It gets pretty technical at times, so I hope you hang on for the ride.
0
SPEAKER_01
And now, a quick few second mention of each sponsor.
0
SPEAKER_01
Check them out in the description.
0
SPEAKER_01
It s the best way to support this podcast.
0
SPEAKER_01
We got iHerb for health, Numeri for world s hardest data science tournament, and Insight Tracker for tracking your biological data.
0
SPEAKER_01
Choose wisely, my friends.
0
SPEAKER_01
Also, if you want to work with our team, our amazing team, we re always hiring, go to lexfriedman.com slash hiring.
0
SPEAKER_01
And now, onto the full ad reads.
0
SPEAKER_01
As always, no ads in the middle.
0
SPEAKER_01
I try to make this interesting, but if you skip them, if you must, my friends, please still check out the sponsors.
0
SPEAKER_01
I enjoy their stuff, maybe you will too.
0
SPEAKER_01
This show is brought to you by iHerb.
0
SPEAKER_01
a platform, a website, a place where you can go and get high quality, selected just for you, health and wellness products for great value, inexpensive, affordable.
0
SPEAKER_01
I get fish oil over there.
0
SPEAKER_01
It s one of the main supplements I ve taken for a long, long, long time in pill form.
0
SPEAKER_01
Makes me feel like I m oiling the machine that is the human body.
0
SPEAKER_01
and the human mind.
0
SPEAKER_01
Even just saying that makes me wonder, what is the power of the placebo effect in all of this?
0
SPEAKER_01
I m actually a big believer in the power of the human mind, coupled with the effectiveness of medication and supplements and nutrition and diet and exercise, all of it.
0
SPEAKER_01
If you couple the belief that the thing will work with stuff that actually works,
0
SPEAKER_01
it s like a supercharge.
0
SPEAKER_01
There s something about the mind allowing the thing to work and maybe the belief that it works reduces stress and has kind of secondary and tertiary effects that you can t even comprehend on the entirety of the biological system that is the human body.
0
SPEAKER_01
It s so fascinating.
0
SPEAKER_01
And it s so difficult to do good studies on that.
0
SPEAKER_01
Because the whole point is you want to study the effect of the entirety of the lifestyle and diet decisions you make on the entirety of the human organism.
0
SPEAKER_01
The billions of organisms that make up a single organism that is you.
0
SPEAKER_01
Anyway, get 22% off with promo code Lex when you go to iherb.com slash Lex.
0
SPEAKER_01
This show is also brought to you by Numerai, a hedge fund that uses AI and machine learning to make investment decisions.
0
SPEAKER_01
They created a tournament, a challenge for all machine learning gurus to come and to compete against each other.
0
SPEAKER_01
to build the best predictive models for financial markets.
0
SPEAKER_01
The stakes are high.
0
SPEAKER_01
This is the kind of problems in the machine learning space that I really care about.
0
SPEAKER_01
Real world problems with high stakes, not toy problems, not ImageNet.
0
SPEAKER_01
Now, ImageNet and all that kind of stuff is good for exploring little ideas, the nuances of architectures, training procedures,
0
SPEAKER_01
of cool little ideas of the entirety of the pipeline on how to do machine learning, or for education purposes.
0
SPEAKER_01
But if you want to really develop ideas that work in the real world, you should be working on real world data where the stakes are high.
0
SPEAKER_01
And this is probably one of the hardest tournaments for machine learning in the world.
0
SPEAKER_01
Head over to Numeri.com slash Lex to sign up for a tournament and hone your machine learning skills.
0
SPEAKER_01
That s Numeri.com slash Lex for a chance to play against me and win a share of the tournament prize pool.
0
SPEAKER_01
This show is also brought to you by Insight Tracker, a service I use to track biological data.
0
SPEAKER_01
As I always say, the entirety of the biological organism that is you, the individual cells, all the life force that makes up the details of the cells and the proteins and the blood and the organs and the individual systems that are interconnected to each other in these incredible complex ways.
0
SPEAKER_01
that are all asynchronously communicating with each other through chemistry, through physics, through electrical signals, through mechanical signals, through whatever the hell signals that I can t even find the right words for.
0
SPEAKER_01
It s an incredible system.
0
SPEAKER_01
How is it possible that this thing works at all?
0
SPEAKER_01
I m looking out into the distance now as I say these words and my visual cortex is processing all of it and somehow makes sense of all of it.
0
SPEAKER_01
This is incredible.
0
SPEAKER_01
Anyway, measuring data on this incredible organism
0
SPEAKER_01
is good, and companies that allow you to measure it in order to make lifestyle decisions is obviously the future.
0
SPEAKER_01
That s why I support Insight Tracker.
0
SPEAKER_01
You can get special savings for a limited time when you go to insighttracker.com slash Lex.
0
SPEAKER_01
This is the Lex Friedman Podcast.
0
SPEAKER_01
To support it, please check out our sponsors in the description.
0
SPEAKER_01
And now, dear friends, here s Chris Latner.
0
SPEAKER_01
It s been, I think, two years since we last talked, and in that time, you somehow went and co-created a new programming language called Mojo.
0
SPEAKER_01
So it s optimized for AI, it s a superset of Python.
0
SPEAKER_01
Let s look at the big picture.
0
SPEAKER_01
What is the vision for Mojo?
0
SPEAKER_00
Well so I mean I think you have to zoom out so I ve been working on a lot of related technologies for many many years so I ve worked on LVM and a lot of things and mobile and servers and things like this but the world s changing and what s happened with AI is we have new GPUs and new.
0
SPEAKER_00
machine learning accelerators and other ASICs and things like that that make AI go real fast.
0
SPEAKER_00
At Google, I worked on TPUs.
0
SPEAKER_00
That s one of the biggest, largest scale deployed systems that exist for AI.
0
SPEAKER_00
And really what you see is if you look across all of the things that are happening in the industry, there s this new compute platform coming.
0
SPEAKER_00
And it s not just about
0
SPEAKER_00
CPUs or GPUs or TPUs or NPUs or IPUs or whatever, all the PUs, right?
0
SPEAKER_00
It s about how do we program these things, right?
0
SPEAKER_00
And so for software folks like us, right, it doesn t do us any good if there s this amazing hardware that we can t use.
0
SPEAKER_00
And one of the things you find out really quick is that
0
SPEAKER_00
having the theoretical capability of programming something and then having the world s power and the innovation of all the all the smart people in the world get unleashed on something can be quite different.
0
SPEAKER_00
And so really where Mojo came from was starting from a problem of we need to be able to take machine learning, take the infrastructure underneath it and make it way more accessible, way more usable, way more understandable by
0
SPEAKER_00
normal people and researchers and other folks that are not themselves like experts in GPUs and things like this.
0
SPEAKER_00
And then through that journey, we realized, hey, we need syntax for this, we need to do a programming language.
0
SPEAKER_01
So one of the main features of the language, I say so fully in jest, is that it allows you to have the file extension to be an emoji, or the fire emoji, which is one of the first
0
SPEAKER_01
emojis used as a file extension I ve ever seen in my life.
0
SPEAKER_01
And then you ask yourself the question, why in the 21st century are we not using Unicode for file extensions?
0
SPEAKER_01
I mean, it s an epic decision.
0
SPEAKER_01
I think clearly the most important decision you ve made, but you could also just use M-O-J-O as the file extension.
0
SPEAKER_00
Okay, so take a step back.
0
SPEAKER_00
I mean, come on, Lex, do you think that the world s ready for this?
0
SPEAKER_00
This is a big moment in the world, right?
0
SPEAKER_01
We re releasing this onto the world.
0
SPEAKER_00
This is innovation.
0
SPEAKER_01
I mean, it really is kind of brilliant.
0
SPEAKER_01
Emojis are such a big part of our daily lives.
0
SPEAKER_01
Why isn t it not in programming?
0
SPEAKER_00
Well, and like you take a step back and look at what file extensions are, right?
0
SPEAKER_00
They re basically metadata.
0
SPEAKER_00
And so why are we spending all the screen space on them and all this stuff?
0
SPEAKER_00
Also, you have them stacked up next to text files and PDF files and whatever else.
0
SPEAKER_00
If you re going to do something cool, you want to stand out.
0
SPEAKER_00
Emojis are colorful.
0
SPEAKER_00
They re visual.
0
SPEAKER_00
They re beautiful.
0
SPEAKER_00
What s been the response so far from
0
SPEAKER_01
Is there support on Windows on operating systems in displaying File Explorer?
0
SPEAKER_00
The one problem I ve seen is that Git doesn t escape it right.
0
SPEAKER_00
And so it thinks that the fire emoji is unprintable, and so it prints out weird hex things if you use the command line Git tool.
0
SPEAKER_00
But everything else, as far as I m aware, works fine.
0
SPEAKER_00
And I have faith that Git can be improved.
0
SPEAKER_00
And so GitHub is fine?
0
SPEAKER_00
GitHub is fine.
0
SPEAKER_00
Yeah, GitHub is fine, Visual Studio Code, Windows, like all this stuff, totally ready, because people have internationalization in their normal part of their paths.
0
SPEAKER_00
So this is just like taking the next step, right?
0
SPEAKER_00
somewhere between, oh, wow, that makes sense.
0
SPEAKER_00
Cool.
0
SPEAKER_00
I like new things, too.
0
SPEAKER_00
Oh, my God, you re killing my baby.
0
SPEAKER_00
Like, what are you talking about?
0
SPEAKER_00
This can never be like I can never handle this.
0
SPEAKER_00
How am I going to type this like all these things?
0
SPEAKER_00
And so this is something where I think that the world will get there.
0
SPEAKER_00
We don t have to bet the whole farm on this.
0
SPEAKER_00
I think we can provide both paths, but I think it ll be great.
0
SPEAKER_01
When can we have emojis as part of the code?
0
SPEAKER_00
Uh, yeah.
0
SPEAKER_00
So, I mean, lots of languages provide that.
0
SPEAKER_00
So, um, I think that we have partial support for that.
0
SPEAKER_00
It s probably not fully done yet, but, but yeah, you can, you can do that.
0
SPEAKER_00
For example, in Swift, you can do that for sure.
0
SPEAKER_00
So an example we gave at Apple was the dog cow.
0
SPEAKER_00
So that s a classical Mac heritage thing.
0
SPEAKER_00
And so you use the dog and the cow emoji together, and that could be your variable name.
0
SPEAKER_00
But of course, the internet went and made pile of poop for everything.
0
SPEAKER_00
So if you want to name your function pile of poop, then you can totally go to town and see how that gets through code review.
0
SPEAKER_01
Okay, so let me just ask a bunch of random questions.
0
SPEAKER_01
So is Mojo primarily designed for AIs or is it a general purpose program?
0
SPEAKER_00
Yeah, good question.
0
SPEAKER_00
So it s AI first.
0
SPEAKER_00
And so AI is driving a lot of the requirements.
0
SPEAKER_00
And so Modular is building and designing and driving Mojo forward.
0
SPEAKER_00
And it s not because it s an interesting project theoretically to build, it s because we need it.
0
SPEAKER_00
And so at Modular, we re really tackling the AI infrastructure landscape and the big problems in AI and the reasons that it is so difficult to use and scale and adopt and deploy and all these big problems in AI.
0
SPEAKER_00
And so we re coming at it from that perspective.
0
SPEAKER_00
Now, when you do that, when you start tackling these problems, you realize that the solution to these problems isn t actually an AI-specific solution.
0
SPEAKER_00
And so while we re doing this, we re building Mojo to be a fully general programming language.
0
SPEAKER_00
And that means that you can obviously tackle GPUs and CPUs and these AI things, but it s also a really great way to build NumPy and other things like that.
0
SPEAKER_00
If you look at what many Python libraries are today, often they re a layer of Python for the API, and they end up being C and C++ code underneath them.
0
SPEAKER_00
That s very true in AI.
0
SPEAKER_00
That s true in lots of other domains as well.
0
SPEAKER_00
And so anytime you see this pattern, that s an opportunity for Mojo to help simplify the world and help people have one thing.
0
SPEAKER_01
So optimize through simplification by having one thing.
0
SPEAKER_01
So you mentioned modular.
0
SPEAKER_01
Mojo s the programming language.
0
SPEAKER_01
Modular s the whole software stack.
0
SPEAKER_00
So just over a year ago, we started this company called Modular.
0
SPEAKER_00
What Modular s about is it s about taking AI and up-leveling it into the next generation.
0
SPEAKER_00
If you take a step back, what s gone on in the last five, six, seven, eight years is that we ve had things like TensorFlow and PyTorch and these other systems come in.
0
SPEAKER_00
You ve used them, you know this.
0
SPEAKER_00
And what s happened is these things have grown like crazy.
0
SPEAKER_00
And they get tons of users.
0
SPEAKER_00
It s in production deployment scenarios.
0
SPEAKER_00
It s being used to power so many systems.
0
SPEAKER_00
I mean, AI is all around us now.
0
SPEAKER_00
It used to be controversial years ago, but now it s a thing.
0
SPEAKER_00
But the challenge with these systems is that they haven t always been thought out with current demands in mind.
0
SPEAKER_00
And so you think about it, where were LLMs eight years ago?
0
SPEAKER_00
Well, they didn t exist, right?
0
SPEAKER_00
AI has changed so much.
0
SPEAKER_00
And a lot of what people are doing today are very different than when these systems were built.
0
SPEAKER_00
And meanwhile, the hardware side of this has gotten into a huge mess.
0
SPEAKER_00
There s tons of new chips and accelerators, and every big company s announcing a new chip every day, it feels like.
0
SPEAKER_00
And so between that, you have this moving system on one side, moving system on the other side, and it just turns into this gigantic mess, which makes it very difficult for people to actually use AI, particularly in production deployment scenarios.
0
SPEAKER_00
And so what Modular s doing is we re helping build out that software stack to help solve some of those problems, so then people can be more productive and get more AI research into production.
0
SPEAKER_00
Now, what Mojo does is it s a really, really, really important piece of that.
0
SPEAKER_00
And so that is part of that engine and part of the technology that allows us to solve these problems.
0
SPEAKER_01
So Mojo is a programming language that allows you to do the higher level programming, the low level programming.
0
SPEAKER_01
They do all kinds of programming in that spectrum that gets you closer and closer to the hardware.
0
SPEAKER_00
So take a step back.
0
SPEAKER_00
So Lex, what do you love about Python?
0
SPEAKER_01
Oh boy, where do I begin?
0
SPEAKER_01
What is love?
0
SPEAKER_01
What do I love about Python?
0
SPEAKER_00
You re a guy who knows love.
0
SPEAKER_00
I know this.
0
SPEAKER_01
Yes.
0
SPEAKER_01
How intuitive it is.
0
SPEAKER_01
How it feels like I m writing natural language English.
0
SPEAKER_01
Yeah.
0
SPEAKER_01
how when I can not just write, but read other people s code, somehow I can understand it faster.
0
SPEAKER_01
It s more condensed than other languages, like ones I m really familiar with, like C++ and C. There s a bunch of sexy little features.
0
SPEAKER_01
We ll probably talk about some of them, but list comprehensions and stuff like this.
0
SPEAKER_00
And don t forget the entire ecosystem of all the packages.
0
SPEAKER_00
There s always something.
0
SPEAKER_00
If you want to do anything, there s always a package.
0
SPEAKER_01
Yeah, so it s not just the ecosystem of the packages and the ecosystem of the humans that do it.
0
SPEAKER_01
That s a really
0
SPEAKER_01
That s an interesting dynamic.
0
SPEAKER_01
I think something about the usability and the ecosystem makes the thing viral, it grows, and it s a virtuous cycle.
0
SPEAKER_00
I think.
0
SPEAKER_00
Well, and there s many things that went into that.
0
SPEAKER_00
So I think that ML was very good for Python.
0
SPEAKER_00
And so I think that TensorFlow and PyTorch and these systems embracing Python really took and helped Python grow.
0
SPEAKER_00
But I think that the major thing underlying it is that Python s like the universal connector.
0
SPEAKER_00
It really helps bring together lots of different systems so you can compose them and build out larger systems without having to understand how it works.
0
SPEAKER_00
But then, what is the problem with Python?
0
SPEAKER_01
Well, I guess you could say several things, but probably that it s slow.
0
SPEAKER_00
I think that s usually what people complain about.
0
SPEAKER_00
And so, slow, I mean, other people complain about tabs and spaces versus curly braces or whatever, but I mean, those people are just wrong, because it is actually just better to use indentation.
0
SPEAKER_00
Wow, strong words.
0
SPEAKER_00
So actually, on a small tangent, let s actually take that.
0
SPEAKER_00
Let s take all kinds of tangents.
0
SPEAKER_00
Oh, come on, Lex, you can push me on it.
0
SPEAKER_00
I can take it.
0
SPEAKER_01
Design, design.
0
SPEAKER_01
Listen, I ve recently left Emacs for VS Code.
0
SPEAKER_01
Okay.
0
SPEAKER_01
The kind of hate mail I had to receive because on the way to doing that, I also said I ve considered Vim.
0
SPEAKER_01
Yep.
0
SPEAKER_01
And chose not to and went with VS Code.
0
SPEAKER_01
You re touching on deep religions, right?
0
SPEAKER_01
Anyway, Tabs is an interesting design decision, and so you ve really written a new programming language here.
0
SPEAKER_01
Yes, it is a superset of Python, but you can make a bunch of different interesting decisions here.
0
SPEAKER_01
And you chose actually to stick with Python in terms of some of the syntax.
0
SPEAKER_00
Well, so let me explain why.
0
SPEAKER_00
I mean, you can explain this in many rational ways.
0
SPEAKER_00
I think that the annotation
1
SPEAKER_00
beautiful, but that s not a rational explanation, but I can defend it rationally.
1
SPEAKER_00
First of all, Python 1 has millions of programmers.
1
SPEAKER_00
It s huge.
1
SPEAKER_00
It s everywhere.
1
SPEAKER_00
It owns machine learning.
1
SPEAKER_00
Factually, it is the thing.
1
SPEAKER_00
Second of all, if you look at it, C code, C++ code, Java, whatever, Swift, curly-braced languages also run through formatting tools and get indented.
1
SPEAKER_00
And so if they re not indented correctly, first of all, it will twist your brain around.
1
SPEAKER_00
It can lead to bugs.
1
SPEAKER_00
There s notorious bugs that have happened across time where the indentation was wrong or misleading and it wasn t formatted right, and so it turned into an issue, right?
1
SPEAKER_00
And so what ends up happening in modern large-scale code bases is people run automatic formatters.
1
SPEAKER_00
So now what you end up with is indentation and curly braces.
1
SPEAKER_00
Well, if you re gonna have
1
SPEAKER_00
you know, the notion of grouping, why not have one thing, right?
1
SPEAKER_00
And get rid of all the clutter and have a more beautiful thing, right?
1
SPEAKER_00
Also, you look at many of these languages, it s like, okay, well you can have curly braces or you can omit them if there s one statement or you just like enter this entire world of complicated design space that objectively you don t need if you have Python style indentation, so.
1
SPEAKER_01
Yeah, I would love to actually see statistics on errors made because of indentation.
1
SPEAKER_01
Like how many errors are made in Python versus in C++ that have to do with basic formatting, all that kind of stuff.
1
SPEAKER_00
I would love to see.
1
SPEAKER_00
I think it s probably pretty minor because once you get, like you use VS Code, I do too.
1
SPEAKER_00
So if you get VS Code set up, it does the indentation for you generally, right?
1
SPEAKER_00
And so you don t, you know, it s actually really nice to not have to fight it.
1
SPEAKER_00
And then what you can see is the editor s telling you how your code will work by indenting it, which I think is pretty cool.
1
SPEAKER_01
I honestly don t think
1
SPEAKER_01
I ve ever, I don t remember having an error in Python because I intended stuff wrong.
1
SPEAKER_00
So, I mean, I think that there s, again, this is a religious thing, and so I can joke about it, and I love to kind of, you know, I realize that this is such a polarizing thing, and everybody wants to argue about it, and so I like poking at the bear a little bit, right?
1
SPEAKER_00
But frankly, right, come back to the first point, Python 1.
1
SPEAKER_00
Like, it s huge, it s in AI, it s the right thing.
1
SPEAKER_00
For us, we see Mojo as being an incredible part of the Python ecosystem.
1
SPEAKER_00
We re not looking to break Python or change it or quote-unquote fix it.
1
SPEAKER_00
We love Python for what it is.
1
SPEAKER_00
Our view is that Python is just not done yet.
1
SPEAKER_00
And so if you look at, you mentioned Python being slow.
1
SPEAKER_00
Well, there s a couple of different things that go into that, which we can talk about if you want.
1
SPEAKER_00
But one of them is that it just doesn t have those features that you would use to do C-like programming.
1
SPEAKER_00
And so if you say, OK, well, I m forced out of Python into C for certain use cases,
1
SPEAKER_00
well, then what we re doing is we re saying, okay, well, why is that?
1
SPEAKER_00
Can we just add those features that are missing from Python back up to Mojo?
1
SPEAKER_00
And then you can have everything that s great about Python, all the things that you re talking about that you love, plus not be forced out of it when you do something a little bit more computationally intense or weird or hardware-y or whatever it is that you re doing.
1
SPEAKER_01
Well, a million questions I wanna ask, but high level again, is it compiled or is it an interpreted language?
1
SPEAKER_01
So Python is just-in-time compilation, what s Mojo?
1
SPEAKER_00
So Mojo, the complicated answer, does all the things.
1
SPEAKER_00
So it s interpreted, it s JIT compiled, and it s statically compiled.
1
SPEAKER_00
And so this is for a variety of reasons.
1
SPEAKER_00
So one of the things that makes Python beautiful is that it s very dynamic.
1
SPEAKER_00
And because it s dynamic, one of the things they added is that it has this powerful metaprogramming feature.
1
SPEAKER_00
And so if you look at something like PyTorch or TensorFlow or, I mean, even a simple use case, like you define a class that has the plus method, right?
1
SPEAKER_00
You can overload the dunder methods like dunder add, for example, and then the plus method works on your class.
1
SPEAKER_00
And so it has very nice and very expressive dynamic metaprogramming features.
1
SPEAKER_00
In Mojo, we want all those features to come in.
1
SPEAKER_00
We don t want to break Python.
1
SPEAKER_00
We want it all to work.
1
SPEAKER_00
But the problem is you can t run those super dynamic features on an embedded processor or on a GPU.
1
SPEAKER_00
Or if you could, you probably don t want to just because of the performance.
1
SPEAKER_00
And so we entered this question of saying, OK, how do you get the power of this dynamic metaprogramming
1
SPEAKER_00
into a language that has to be super efficient in specific cases.
1
SPEAKER_00
And so what we did was we said, okay, we ll take that interpreter.
1
SPEAKER_00
Python has an interpreter in it, right?
1
SPEAKER_00
Take that interpreter and allow it to run at compile time.
1
SPEAKER_00
And so now what you get is you get compile time metaprogramming.
1
SPEAKER_00
And so this is super interesting and super powerful because one of the big advantages you get is you get Python-style expressive APIs.
1
SPEAKER_00
You get the ability to have overloaded operators.
1
SPEAKER_00
And if you look at what happens inside of like PyTorch, for example, with automatic differentiation and eager mode and like all these things, they re using these really dynamic and powerful features at runtime.
1
SPEAKER_00
But we can take those features and lift them so that they run at compile time.
1
SPEAKER_01
So you re, because C++ has metaprogramming with templates.
1
SPEAKER_01
But it s really messy.
1
SPEAKER_00
It s super messy.
1
SPEAKER_00
It s always, it was accidentally, I mean, different people have different interpretations.
1
SPEAKER_00
My interpretation is that it was made accidentally powerful.
1
SPEAKER_00
It was not designed to be Turing complete, for example, but that was discovered kind of along the way, accidentally.
1
SPEAKER_00
And so there have been a number of languages in the space.
1
SPEAKER_00
And so they usually have templates or code instantiation, code copying features of various sorts.
1
SPEAKER_00
Some more modern languages or some more newer languages, let s say, like, you know, they re fairly unknown, like Zig, for example, says, okay, well, let s take all of those types that you can run it, all those things you can do at runtime and allow them to happen at compile time.
1
SPEAKER_00
And so, one of the problems with C++, I mean, which is, one of the problems with C++ is... Here we go.
1
SPEAKER_00
Wrong words.
1
SPEAKER_00
We re going to offend everybody today.
1
SPEAKER_00
Oh, it s okay.
1
SPEAKER_00
I mean, everybody hates me for a variety of reasons anyways, I m sure, right?
1
SPEAKER_00
It s just the way they show love.
1
SPEAKER_00
I have written enough C++ code to earn a little bit of grumpiness with C++.
1
SPEAKER_00
But one of the problems with it is that the metaprogramming system templates is just a completely different universe from the normal runtime programming world.
1
SPEAKER_00
And so if you do metaprogramming and programming, it s just like a different universe, different syntax, different concepts, different stuff going on.
1
SPEAKER_00
And so again, one of our goals with Mojo is to make things really easy to use, easy to learn.
1
SPEAKER_00
And so there s a natural stepping stone.
1
SPEAKER_00
And so as you do this, you say, okay, well, I have to do programming at runtime.
1
SPEAKER_00
I have to do programming at compile time.
1
SPEAKER_00
Why are these different things?
1
SPEAKER_01
How hard is that to pull it off?
1
SPEAKER_01
Because that sounds, to me, as a fan of metaprogramming in C++ even, how hard is it to pull that off?
1
SPEAKER_01
That sounds really, really exciting, because you can do the same style programming at compile time and at runtime.
1
SPEAKER_00
That s really, really exciting.
1
SPEAKER_00
Yep, yep.
1
SPEAKER_00
And so, I mean, in terms of the compiler implementation details, it s hard.
1
SPEAKER_00
I won t be shy about that.
1
SPEAKER_00
It s super hard.
1
SPEAKER_00
It requires, I mean, what Mojo has underneath the covers is a completely new approach to the design of the compiler itself.
1
SPEAKER_00
And so this builds on these technologies like MLIR that you mentioned, but it also includes other like caching and other interpreters and JIT compilers and other stuff like that.
1
SPEAKER_00
So you have like an interpreter inside the compiler?
1
SPEAKER_00
Within the compiler, yes.
1
SPEAKER_00
And so it really takes the standard model of programming languages and kind of twists it and unifies it with the runtime model, which I think is really cool.
1
SPEAKER_00
And to me, the value of that is that, again, many of these languages have metaprogramming features, like they grow macros or something, right?
1
SPEAKER_00
Lisp, right?
1
SPEAKER_00
I know your roots, right?
1
SPEAKER_00
And this is a powerful thing, right?
1
SPEAKER_00
And so if you go back to Lisp, one of the most powerful things about it is that it said that the metaprogramming and the programming are the same.
1
SPEAKER_00
Right, and so that made it way simpler, way more consistent, way easier to understand, reason about, and it made it more composable, so if you build a library, you can use it both at runtime and compile time, which is pretty cool.
1
SPEAKER_01
Yeah, and for machine learning, I think metaprogramming, I think we could generally say is extremely useful.
1
SPEAKER_01
And so you get features, I mean, I ll jump around, but there s the feature of auto-tuning and adaptive compilation just blows my mind.
1
SPEAKER_00
Yeah, well, so okay, so let s come back to that.
1
SPEAKER_00
Okay, all right.
1
SPEAKER_00
So what is machine learning?
1
SPEAKER_00
Or what is a machine learning model?
1
SPEAKER_00
Like you take a PyTorch model off the internet, right?
1
SPEAKER_00
It s really interesting to me, because what PyTorch and what TensorFlow and all these frameworks are kind of pushing compute into, is they re pushing into this abstract specification of a compute problem, which then gets mapped in a whole bunch of different ways.
1
SPEAKER_00
And so this is why it became a metaprogramming problem.
1
SPEAKER_00
You want to be able to say, cool, I have this neural net, now run it with batch size 1,000.
1
SPEAKER_00
Do a mapping across batch.
1
SPEAKER_00
Or, okay, I want to take this problem, now run it across 1,000 CPUs, or GPUs.
1
SPEAKER_00
And so this problem of describe the compute and then map it and do things and transform it are, actually it s very profound and that s one of the things that makes machine learning systems really special.
1
SPEAKER_01
Maybe can you describe autotuning and how do you pull off, I mean I guess adaptive compilation is what we re talking about as metaprogramming.
1
SPEAKER_01
How do you pull off autotune?
1
SPEAKER_01
Is that as profound as I think it is?
1
SPEAKER_01
It seems like a really, we ll mention list comprehensions, to me,
1
SPEAKER_01
from a quick glance at Mojo, which, by the way, I have to absolutely dive in.
1
SPEAKER_01
As I realize how amazing this is, I absolutely must dive in.
1
SPEAKER_01
That looks like just an incredible feature for machine learning people.
1
SPEAKER_00
Yeah, well, so what is auto-tuning?
1
SPEAKER_00
So take a step back.
1
SPEAKER_00
Auto-tuning is a feature in Mojo.
1
SPEAKER_00
It s not, so very little of what we re doing is actually research.
1
SPEAKER_00
Like many of these ideas have existed in other systems and other places, and so what we re doing is we re pulling together good ideas, remixing them, and making them into hopefully a beautiful system, right?
1
SPEAKER_00
And so auto-tuning, the observation is that
1
SPEAKER_00
Turns out hardware systems algorithms are really complicated.
1
SPEAKER_00
Turns out maybe you don t actually want to know how the hardware works, right?
1
SPEAKER_00
A lot of people don t, right?
1
SPEAKER_00
And so there are lots of really smart hardware people, I know a lot of them, where they know everything about, okay, the cache size is this and the number of registers is that.
1
SPEAKER_00
And if you use this length of vector, it s going to be super efficient because it maps directly onto what it can do.
1
SPEAKER_00
And like,
1
SPEAKER_00
all this kind of stuff where the GPU has SMs and it has a warp size of whatever, right?
1
SPEAKER_00
All the stuff that goes into these things where the tile size of a TPU is 128, like these factoids, right?
1
SPEAKER_00
My belief is that most normal people, and I love hardware people also, I m not trying to offend literally everybody in the internet, but most programmers actually don t want to know this stuff, right?
1
SPEAKER_00
And so if you come at it from the perspective of how do we allow people to build both more abstracted but also more portable code,
1
SPEAKER_00
Because, you know, it could be that the vector length changes, or the cache size changes, or it could be that the tile size of your matrix changes, or the number, you know, an A100 versus an H100 versus a Volta versus a whatever GPU have different characteristics, right?
1
SPEAKER_00
A lot of the algorithms that you run are actually the same, but the parameters, these magic numbers you have to fill in, end up being really fiddly numbers that an expert has to go figure out.
1
SPEAKER_00
And so what autotuning does is says, okay, well, guess what?
1
SPEAKER_00
There s a lot of compute out there.
1
SPEAKER_00
Right?
1
SPEAKER_00
So instead of having humans go randomly try all the things or do a grid search or go search some complicated multidimensional space, how about we have computers do that?
1
SPEAKER_00
Right?
1
SPEAKER_00
And so what autotuning does is you can say, hey, here s my algorithm.
1
SPEAKER_00
If it s a matrix operation or something like that, you can say, okay, I m gonna carve it up into blocks.
1
SPEAKER_00
I m gonna do those blocks in parallel.
1
SPEAKER_00
And I want this with 128 things that I m running on, I want to cut it this way or that way or whatever.
1
SPEAKER_00
And you can say, hey, go see which one s actually empirically better on the system.
1
SPEAKER_00
And then the result of that, you cache for that system.
1
SPEAKER_00
You save it.
1
SPEAKER_00
And so come back to twisting your compiler brain, right?
1
SPEAKER_00
So not only does the compiler have an interpreter that you use to do metaprogramming, that compiler, that interpreter, that metaprogramming now has to actually take your code and go run it on a target machine.
1
SPEAKER_00
See which one it likes the best and then stitch it in and then keep going, right?
1
SPEAKER_00
So part of the compilation is machine specific.
1
SPEAKER_00
Yeah, well, so I mean, this is an optional feature, right?
1
SPEAKER_00
So you don t have to use it for everything.
1
SPEAKER_00
But yeah, if you re, so one of the things that we re in the quest of is ultimate performance.
1
SPEAKER_00
And ultimate performance is important for a couple of reasons, right?
1
SPEAKER_00
So if you re an enterprise, you re looking to save costs and compute and things like this, ultimate performance translates to fewer servers.
1
SPEAKER_00
Like if you care about the environment, hey, better performance leads to more efficiency, right?
1
SPEAKER_00
I mean, you could joke and say like, Python s bad for the environment.
1
SPEAKER_00
And so if you move to Mojo, it s like at least 10x better, just out of the box and keep going.
1
SPEAKER_00
But performance is also interesting because it leads to better products.
1
SPEAKER_00
And so in the space of machine learning, if you reduce the latency of a model so that it runs faster, so every time you query the server running the model, it takes less time, well then the product team can go and make the model bigger.
1
SPEAKER_00
Well, that actually makes it so you have a better experience as a customer.
1
SPEAKER_00
And so a lot of people care about that.
1
SPEAKER_01
So for auto tuning, for like tile size, you mentioned 128 for TPU, you would specify like a bunch of options to try just in the code, just simple statement.
1
SPEAKER_01
And then you can just set and forget and know, depending on wherever it compiles, it ll actually be the fastest.
1
SPEAKER_00
And yeah, exactly.
1
SPEAKER_00
And the beauty of this is that it helps you in a whole bunch of different ways.
1
SPEAKER_00
Right.
1
SPEAKER_00
So if you re building
1
SPEAKER_00
So often what will happen is that you ve written a bunch of software yourself, right?
1
SPEAKER_00
You wake up one day, you say, I have an idea.
1
SPEAKER_00
I m going to go cut up some code.
1
SPEAKER_00
I get to work.
1
SPEAKER_00
I forget about it.
1
SPEAKER_00
I move on with life, I come back six months, or a year, or two years, or three years later, you dust it off, and you go use it again in a new environment.
1
SPEAKER_00
And maybe your GPU s different, maybe you re running on a server instead of a laptop, maybe, whatever, right?
1
SPEAKER_00
And so, the problem now is you say, okay, well, I mean, again, not everybody cares about performance, but if you do, you say, okay, well, I wanna take advantage of all these new features.
1
SPEAKER_00
I don t wanna break the old thing, though.
1
SPEAKER_00
Right, and so the typical way of handling this kind of stuff before is, you know, if you re talking about C++ templates, or you re talking about C with macros, you end up with if defs, you get like all these weird things get layered in, make the code super complicated, and then how do you test it?
1
SPEAKER_00
Right, it becomes this crazy complexity, multidimensional space that you have to worry about.
1
SPEAKER_00
And, you know, that just doesn t scale very well.
1
SPEAKER_01
Actually, let me just jump around before I go to some specific features, like the increase in performance here that we re talking about can be just insane.
1
SPEAKER_01
You write that Mojo can provide a 35,000x speedup over Python.
1
SPEAKER_01
How does it do that?
1
SPEAKER_00
Yeah, so it can even do more, but we ll get to that.
1
SPEAKER_00
So first of all, when we say that, we re talking about what s called CPython.
1
SPEAKER_00
It s the default Python that everybody uses.
1
SPEAKER_00
When you type Python 3, that s typically the one you use, right?
1
SPEAKER_00
CPython is an interpreter.
1
SPEAKER_00
And so interpreters, they have an extra layer of byte codes and things like this that they have to go read, parse, interpret, and it makes them kind of slow from that perspective.
1
SPEAKER_00
And so one of the first things we do is we move to a compiler.
1
SPEAKER_00
And so just moving to a compiler, getting the interpreter out of the loop is two to five to 10x speed up, depending on the code.
1
SPEAKER_00
So just out of the gate, just using more modern techniques, right?
1
SPEAKER_00
Now, if you do that, one of the things you can do is you can start to look at how CPython started to lay out data.
1
SPEAKER_00
And so one of the things that CPython did, and this isn t part of the Python spec necessarily, but this is just sets of decisions, is that,
1
SPEAKER_00
If you take an integer, for example, it ll put it in an object.
1
SPEAKER_00
Because in Python, everything s an object.
1
SPEAKER_00
And so they do the very logical thing of keeping the memory representation of all objects the same.
1
SPEAKER_00
So all objects have a header, they have like payload data, and what this means is that every time you pass around an object, you re passing around a pointer to the data.
1
SPEAKER_00
Well, this has overhead.
1
SPEAKER_00
It turns out that modern computers don t like chasing pointers very much and things like this.
1
SPEAKER_00
It means that you have to allocate the data.
1
SPEAKER_00
It means you have to reference count it, which is another way that Python uses to keep track of memory.
1
SPEAKER_00
And so this has a lot of overhead.
1
SPEAKER_00
And so if you say, okay, let s try to get that out of the heap, out of a box, out of an indirection and into the registers.
1
SPEAKER_00
That s another 10x.
1
SPEAKER_01
So it adds up if you re reference counting every single thing you create, that adds up.
1
SPEAKER_00
Yep, and if you look at, you know, people complain about the Python gil.
1
SPEAKER_00
This is one of the things that hurts parallelism.
1
SPEAKER_00
That s because of the reference counting.
1
SPEAKER_00
Right, and so the gil and reference counting are very tightly intertwined in Python.
1
SPEAKER_00
It s not the only thing, but it s very tightly intertwined.
1
SPEAKER_00
And so then you lean into this and you say, okay, cool, well, modern computers, they can do more than one operation at a time.
1
SPEAKER_00
so they have vectors what is a vector well a vector allows you to take one instead of taking one piece of data doing an add or multiply and then picking up the next one you can now do 4 or 8 or 16 or 32 at a time right well python doesn t expose that because of reasons and so now you can say okay well you can adopt that now you have threads now you have like additional
2
SPEAKER_01
Like, you can control memory hierarchy.
2
SPEAKER_01
And so what Mojo allows you to do is it allows you to start taking advantage of all these powerful things that have been built into the hardware over time.
2
SPEAKER_01
And it gives, the library gives very nice features.
2
SPEAKER_01
So you can say, just parallelize this.
2
SPEAKER_01
Do this in parallel, right?
2
SPEAKER_01
So it s very, very powerful weapons against slowness, which is why people have been, I think, having fun, like, just taking code and making it go fast, because it s just kind of an adrenaline rush to see, like, how fast you can get things.
2
SPEAKER_00
Before I talk about some of the interesting stuff with parallelization and all that, let s first talk about the basics.
2
SPEAKER_00
We talked indentation, right?
2
SPEAKER_00
So this thing looks like Python.
2
SPEAKER_00
It s sexy and beautiful like Python, as we mentioned.
2
SPEAKER_00
Is it a typed language?
2
SPEAKER_00
So what s the role of types?
2
SPEAKER_01
Yeah, good question.
2
SPEAKER_01
So Python has types.
2
SPEAKER_01
It has strings, it has integers, it has dictionaries, and all that stuff.
2
SPEAKER_01
But they all live at runtime.
2
SPEAKER_01
Because all those types live at runtime in Python, you don t have to spell them.
2
SPEAKER_01
Python also has this whole typing thing going on now, and a lot of people use it.
2
SPEAKER_01
I m not talking about that.
2
SPEAKER_01
That s kind of a different thing.
2
SPEAKER_01
We can go back to that if you want.
2
SPEAKER_01
But typically, you just say, I have a def, and my def takes two parameters.
2
SPEAKER_01
I m going to call them a and b, and I don t have to write a type.
2
SPEAKER_01
is great, but what that does is that forces what s called a consistent representation.
2
SPEAKER_01
So these things have to be a pointer to an object with the object header, and they all have to look the same.
2
SPEAKER_01
And then when you dispatch a method, you go through all the same different paths, no matter what the receiver, whatever that type is.
2
SPEAKER_01
So what Mojo does is it allows you to have more than one kind of type.
2
SPEAKER_01
And so what it does is allows you to say, OK, cool, I have an object, and objects behave like Python does.
2
SPEAKER_01
And so it s fully dynamic, and that s all great.
2
SPEAKER_01
And for many things, classes, that s all very powerful and very important.
2
SPEAKER_01
But if you want to say, hey, it s an integer, and it s 32 bits, or it s 64 bits, or whatever it is, or it s a floating point value,
2
SPEAKER_01
and it s 64 bits.
2
SPEAKER_01
Well, then the compiler can take that and it can use that to do way better optimization.
2
SPEAKER_01
It turns out, again, getting rid of the indirections is huge.
2
SPEAKER_01
It means you can get better code completion because the compiler knows what the type is, and so it knows what operations work on it.
2
SPEAKER_01
And so that s actually pretty huge.
2
SPEAKER_01
And so what Mojo does is it allows you to progressively adopt types into your program.
2
SPEAKER_01
And so you can start, again, it s compatible with Python.
2
SPEAKER_01
And so then you can add however many types you want, wherever you want them.
2
SPEAKER_01
And if you don t want to deal with it, you don t have to deal with it.
2
SPEAKER_01
And so one of our opinions on this is that it s not that types are the right thing or the wrong thing.
2
SPEAKER_01
It s that they re a useful thing.
2
SPEAKER_00
Which is kind of optional, it s not strict typing.
2
SPEAKER_00
You don t have to specify a type.
2
SPEAKER_00
Okay, so it s starting from the thing that Python s kind of reaching towards right now with trying to inject types into it.
2
SPEAKER_01
Yeah, with a very different approach, but yes.
2
SPEAKER_00
What s the different approach?
2
SPEAKER_00
I m actually one of the people
2
SPEAKER_00
They have not been using types very much in Python.
2
SPEAKER_00
So I haven t- That s okay.
2
SPEAKER_00
Why did you say?
2
SPEAKER_00
It s just, well, because I know the importance.
2
SPEAKER_00
It s like adults use strict typing.
2
SPEAKER_00
And so I refuse to grow up in that sense.
2
SPEAKER_00
It s a kind of rebellion.
2
SPEAKER_00
But I just know that...
2
SPEAKER_00
it probably reduces the amount of errors, even just for, forget about performance improvements, it probably reduces errors when you do strict typing.
2
SPEAKER_01
Yeah, so I mean, I think it s interesting if you look at that, right?
2
SPEAKER_01
And the reason I m giving you a hard time, Dan, is that there s this cultural norm, this pressure, this like, there has to be a right way to do things.
2
SPEAKER_01
You know, grownups only do it one way, and if you don t do that, you should feel bad, right?
2
SPEAKER_01
Like, some people feel like Python s a guilty pleasure or something, and it s like, when it gets serious, I need to go rewrite it, right?
2
SPEAKER_01
Yeah, exactly.
2
SPEAKER_01
I mean, cool, I understand history and I understand kind of where this comes from, but I don t think it has to be a guilty pleasure.
2
SPEAKER_00
Yeah.
2
SPEAKER_01
So if you look at that, you say, why do you have to rewrite it?
2
SPEAKER_01
Well, you have to rewrite it to deploy.
2
SPEAKER_01
Well, why do you want to deploy?
2
SPEAKER_01
Well, you care about performance, you care about predictability, or you want, you know, a tiny thing on the server that has no dependencies or, you know, you have objectives that you re trying to attain.
2
SPEAKER_01
So what if Python can achieve those objectives?
2
SPEAKER_01
So if you want types, well, maybe you want types because you want to make sure you re passing the right thing.
2
SPEAKER_01
Sure, you can add a type.
2
SPEAKER_01
If you don t care, you re prototyping some stuff, you re hacking some things out, you re pulling some RAM code off the internet, it should just work.
2
SPEAKER_01
And you shouldn t be pressured.
2
SPEAKER_01
You shouldn t feel bad about doing the right thing or the thing that feels good.
2
SPEAKER_01
Now, if you re in a team, you re working at some massive internet company and you have 400 million lines of Python code, well, they may have a house rule that you use types because it makes it easier for different humans to talk to each other and understand what s going on and bugs at scale.
2
SPEAKER_01
And so there are lots of good reasons why you might want to use types.
2
SPEAKER_01
But that doesn t mean that everybody should use them all the time, right?
2
SPEAKER_01
So what Mojo does is it says, cool, well, allow people to use types.
2
SPEAKER_01
And if you use types, you get nice things out of it, right?
2
SPEAKER_01
You get better performance and things like this, right?
2
SPEAKER_01
But Mojo is a full compatible superset of Python.
2
SPEAKER_01
And so that means it has to work without types.
2
SPEAKER_01
It has to support all the dynamic things, it has to support all the packages, it has to support list comprehensions and things like this, right?
2
SPEAKER_01
And so that starting point, I think, is really important.
2
SPEAKER_01
And I think that, again, you can look at why I care so much about this, and there s many different aspects of that, one of which is the world went through a very challenging migration from Python 2 to Python 3.
2
SPEAKER_01
This migration took many years, and it was very painful for many teams.
2
SPEAKER_01
There s a lot of things that went on in that.
2
SPEAKER_01
I m not an expert in all the details, and I honestly don t want to be.
2
SPEAKER_01
I don t want the world to have to go through that.
2
SPEAKER_01
People can ignore Mojo, and if it s not their thing, that s cool.
2
SPEAKER_01
But if they want to use Mojo, I don t want them to have to rewrite all their code.
2
SPEAKER_00
The superset part is...
2
SPEAKER_00
It s just, I mean, there s so much brilliant stuff here.
2
SPEAKER_00
That definitely is incredible.
2
SPEAKER_00
We ll talk about that.
2
SPEAKER_00
First of all, how s the typing implemented differently in Python versus Mojo?
2
SPEAKER_00
So this heterogeneous flexibility you said is definitely implemented.
2
SPEAKER_01
Yeah, so I m not a full expert in the whole backstory and types in Python.
2
SPEAKER_01
So I ll give you, I ll give you that I can give you my understanding.
2
SPEAKER_01
My understanding is basically like many dynamic languages, the ecosystem went through a phase where people went from writing scripts to writing large scale, huge code bases in Python.
2
SPEAKER_01
And at scale,
2
SPEAKER_01
It kind of helps to have types.
2
SPEAKER_01
People want to be able to reason about interfaces.
2
SPEAKER_01
Do you expect a string or an int?
2
SPEAKER_01
These basic things, right?
2
SPEAKER_01
And so what the Python community started doing is it started saying, OK, let s have tools on the side.
2
SPEAKER_01
checker tools that go and enforce invariants, check for bugs, try to identify things.
2
SPEAKER_01
These are called static analysis tools, generally.
2
SPEAKER_01
And so these tools run over your code and try to look for bugs.
2
SPEAKER_01
What ended up happening is there s so many of these things, so many different weird patterns and different approaches on specifying the types and different things going on that the Python community realized and recognized, hey, hey, hey, there s a thing here.
2
SPEAKER_01
And so what they started to do is they started to standardize the syntax for adding types to Python.
2
SPEAKER_01
Now, one of the challenges that they had is that they re coming from kind of this fragmented world where there s lots of different tools, they have different trade-offs and interpretations, and the types mean different things.
2
SPEAKER_01
And so, if you look at types in Python, according to the Python spec, the types are ignored.
2
SPEAKER_01
So according to the Python spec, you can write pretty much anything in a type position.
2
SPEAKER_01
Technically, you can write any expression.
2
SPEAKER_01
Now, that s beautiful because you can extend it, you can do cool things, you can build your own tools, you can build your own house linter or something like that.
2
SPEAKER_01
But it s also a problem because any
2
SPEAKER_01
Existing python program may be using different tools and they have different interpretations And so if you adopt somebody s package into your ecosystem try around the tool you prefer It may throw out tons of weird errors and warnings and problems just because it s incompatible with how these things work Also because they re added late and they re not checked by the python interpreter is always kind of more of a hint than it is a requirement
2
SPEAKER_01
Also, the CPython implementation can t use them for performance, and so it s really... That s the big one, right?
2
SPEAKER_00
So you can t utilize for the compilation, for the just-in-time compilation.
2
SPEAKER_01
Exactly.
2
SPEAKER_01
And this all comes back to the design principle of it s... They re kind of hints.
2
SPEAKER_01
They re kind of... The definition s a little bit murky.
2
SPEAKER_01
It s unclear exactly the interpretation in a bunch of cases, and so because of that, you can t actually... Even if you want to, it s really difficult to use them to say, like, it is going to be an int, and if it s not, it s a problem, right?
2
SPEAKER_01
A lot of code would break if you did that.
2
SPEAKER_01
So in Mojo, right, so you can still use those kind of type annotations, it s fine.
2
SPEAKER_01
But in Mojo, if you declare a type and you use it, then it means it is going to be that type.
2
SPEAKER_01
And the compiler helps you check that and enforce it and it s safe.
2
SPEAKER_01
And it s not a best effort hint kind of a thing.
2
SPEAKER_00
So if you try to shove a string type thing into an integer.
2
SPEAKER_00
You get an error from the compiler.
2
SPEAKER_00
From the compiler compile time.
2
SPEAKER_00
Nice.
2
SPEAKER_00
Okay, what kind of basic types are there?
2
SPEAKER_01
Yeah, so Mojo is pretty hardcore in terms of what it tries to do in the language, which is the philosophy there is that we
2
SPEAKER_01
Again, if you look at Python, right?
2
SPEAKER_01
Python s a beautiful language because it s so extensible, right?
2
SPEAKER_01
And so all of the different things in Python, like for loops and plus and like all these things can be accessed through these underbar arm bar methods, okay?
2
SPEAKER_01
So you have to say, okay, if I make something that is super fast, I can go all the way down to the metal.
2
SPEAKER_01
Why do I need to have integers built into the language?
2
SPEAKER_01
And so what Mojo does is it says, okay, well we can have this notion of structs.
2
SPEAKER_01
So you have classes in Python, now you can have structs.
2
SPEAKER_01
Classes are dynamic, structs are static.
2
SPEAKER_01
Cool, we can get high performance, we can write C++ kind of code with structs if you want.
2
SPEAKER_01
These things mix and work beautifully together.
2
SPEAKER_01
But what that means is that you can go and implement strings and ints and floats and arrays and all that kind of stuff in the language.
2
SPEAKER_01
And so that s really cool because
2
SPEAKER_01
to me as a idealizing compiler language type of person, what I want to do is I want to get magic out of the compiler and put it in the libraries.
2
SPEAKER_01
Because if somebody can, if we can build an integer that s beautiful and has an amazing API and does all the things you d expect an integer to do, but you don t like it, maybe you want a big integer, maybe you want a sideways integer, I don t know, like what all the space of integers are, then you can do that and it s not a second class citizen.
2
SPEAKER_01
And so if you look at certain other languages, like C++, one I also love and use a lot, Int is hard-coded in the language, but Complex is not.
2
SPEAKER_01
And so isn t it kind of weird that you have this STD complex class, but you have int, and complex tries to look like a natural numeric type and things like this, but integers and floating point have these special promotion rules and other things like that that are magic, and they re hacked into the compiler.
2
SPEAKER_01
And because of that, you can t actually make something that works like the built-in types.
2
SPEAKER_00
Is there something provided as a standard?
2
SPEAKER_00
Because it s AI first, numerical types are so important here.
2
SPEAKER_00
So is there something like a nice standard implementation of integer and float?
2
SPEAKER_01
Yeah, so we re still building all that stuff out.
2
SPEAKER_01
So we provide integers and floats and all that kind of stuff.
2
SPEAKER_01
We also provide buffers and tensors and things like that that you d expect in an ML context.
2
SPEAKER_01
Honestly, we need to keep designing and redesigning and working with the community to build that out and make that better.
2
SPEAKER_01
That s not our strength right now.
2
SPEAKER_01
give us six months or a year, and I think it ll be way better.
2
SPEAKER_01
But the power of putting it in the library means that we can have teams of experts that aren t compiler engineers that can help us design and refine and drive us forward.
2
SPEAKER_00
So one of the exciting things we should mention here is that this is new and fresh.
2
SPEAKER_00
This cake is unbaked.
2
SPEAKER_00
It s almost baked.
2
SPEAKER_00
You can tell it s delicious.
2
SPEAKER_00
but it s not fully ready to be consumed.
2
SPEAKER_01
Yep, that s very fair.
2
SPEAKER_01
It is very useful, but it s very useful if you re a super low-level programmer right now.
2
SPEAKER_01
And what we re doing is we re working our way up the stack.
2
SPEAKER_01
And so the way I would look at Mojo today in May in 2023 is that it s like a 0.1.
2
SPEAKER_01
So I think that a year from now, it s going to be way more interesting to a variety of people.
2
SPEAKER_01
But what we re doing is we decided to release it early so that people can get access to it and play with it, and we can build it with the community.
2
SPEAKER_01
We have a big roadmap fully published, being transparent about this, and a lot of people are involved in this stuff.
2
SPEAKER_01
And so what we re doing is we re really optimizing for building this thing the right way.
2
SPEAKER_01
And building it the right way is kind of interesting, working with the community, because everybody wants it yesterday.
2
SPEAKER_01
And so sometimes it s kind of, you know, there s some dynamics there, but I think it s the right thing.
2
SPEAKER_00
So there s a discord also.
2
SPEAKER_00
So the dynamics is pretty interesting.
2
SPEAKER_00
Sometimes the community probably can be very chaotic and introduce a lot of stress.
2
SPEAKER_00
Guido famously quit over the stress of the walrus operator.
2
SPEAKER_00
I mean, it broke.
2
SPEAKER_00
His father broke the camel s back.
2
SPEAKER_00
Exactly.
2
SPEAKER_00
And so like, it can be very stressful to develop, but can you just,
2
SPEAKER_00
a tangent upon a tangent.
2
SPEAKER_00
Is it stressful to work through the design of various features here given that the community is so richly involved?
2
SPEAKER_01
So I ve been doing open development and community stuff for decades now.
2
SPEAKER_01
Somehow this has happened to me.
2
SPEAKER_01
So I ve learned some tricks, but the thing that always gets me is I want to make people happy, right?
2
SPEAKER_01
And so this is maybe not all people all happy all the time, but generally I want people to be happy, right?
2
SPEAKER_01
And so the challenge is that
2
SPEAKER_01
again, we re tapping into some long, some deep seated, long tensions and pressures, both in the Python world, but also in the AI world, in the hardware world and things like this.
2
SPEAKER_01
And so people just want us to move faster.
2
SPEAKER_01
Right.
2
SPEAKER_01
And so, again, our decision was, let s release this early.
2
SPEAKER_01
Let s get people used to it or access to it and play with it.
2
SPEAKER_01
And like, let s, let s build in the open, which we could have
2
SPEAKER_01
you know, had the language monk sitting in the cloister up on the hilltop, like, beavering away, trying to build something.
2
SPEAKER_01
But in my experience, you get something that s way better if you work with the community.
2
SPEAKER_01
And so yes, it can be frustrating, can be challenging for lots of people involved.
2
SPEAKER_01
And, you know, if you, I mean, if you mentioned our Discord, we have over 10,000 people on the Discord, 11,000 people or something.
2
SPEAKER_01
Keep in mind, we released Mojo like two weeks ago.
2
SPEAKER_01
So it s very cool.
2
SPEAKER_01
But what that means is that 10,000, 11,000 people all will want something different.
2
SPEAKER_01
And so what we ve done is we ve tried to say, OK, cool, here s our roadmap.
2
SPEAKER_01
And the roadmap isn t completely arbitrary.
2
SPEAKER_01
It s based on here s the logical order in which to build these features or add these capabilities and things like that.
2
SPEAKER_01
And what we ve done is we ve spun really fast some bug fixes.
2
SPEAKER_01
And so we actually have very few bugs, which is
2
SPEAKER_01
Cool, I mean, actually for a project in the state, but then what we re doing is we re dropping in features very deliberately.
2
SPEAKER_00
I mean, this is fun to watch, because you got the two gigantic communities of like hardware, like systems engineers, and then you have the machine learning Python people that are like higher level.
2
SPEAKER_00
Yeah.
2
SPEAKER_00
And it s just two, like army.
2
SPEAKER_00
They ve been at war, yeah.
2
SPEAKER_01
They ve been at war.
2
SPEAKER_01
Right?
2
SPEAKER_01
And so here s a test.
2
SPEAKER_01
It s like a Tolkien novel or something.
2
SPEAKER_00
Okay.
2
SPEAKER_01
So here s a test.
2
SPEAKER_01
And again, like, it s super funny for something that s only been out for two weeks, right?
2
SPEAKER_01
People are so impatient, right?
2
SPEAKER_01
But okay, cool.
2
SPEAKER_01
Let s fast forward a year.
2
SPEAKER_01
Like in a year s time, Mojo will be actually quite amazing and solve tons of problems and be very good.
2
SPEAKER_01
People still have these problems, right?
2
SPEAKER_01
And so you look at this and you say, and the way I look at this, at least, is to say, okay, well, we re solving big, longstanding problems.
2
SPEAKER_01
To me, I, again, working on many different problems, I wanna make sure we do it right.
2
SPEAKER_01
There s a responsibility, you feel, because if you mess it up, there s very few opportunities to do projects like this and have them really have impact on the world.
2
SPEAKER_01
If we do it right, then maybe we can take those feuding armies and actually heal some of those wounds.
2
SPEAKER_00
This feels like a speech by George Washington or Abraham Lincoln or something.
2
SPEAKER_01
And you look at this and it s like, okay, well, how different are we?
2
SPEAKER_01
We all want beautiful things.
2
SPEAKER_01
We all want something that s nice.
2
SPEAKER_01
We all want to be able to work together.
2
SPEAKER_01
We all want our stuff to be used, right?
2
SPEAKER_01
And so if we can help heal that.
2
SPEAKER_01
Now, I m not optimistic that all people will use Mojo and they ll stop using C++.
2
SPEAKER_01
That s not my goal, right?
2
SPEAKER_01
But if we can heal some of that, I think that d be pretty cool.
2
SPEAKER_00
Yeah, and we start by putting the people who like braces into the Gulag.
2
SPEAKER_00
No.
2
SPEAKER_01
So there are proposals for adding braces to Mojo, and we just tell them no.
2
SPEAKER_01
Oh, OK.
2
SPEAKER_00
Anyway, so there s a lot of amazing features on the roadmap and those already implemented.
2
SPEAKER_00
It d be awesome if I could just ask you a few things.
2
SPEAKER_00
Yeah, go for it.
3
SPEAKER_01
this improvement comes from immutability.
3
SPEAKER_01
So what s this var and this let thing that we got going on?
3
SPEAKER_01
And what s immutability?
3
SPEAKER_00
Yeah, so one of the things that is useful, and it s not always required, but it s useful, is knowing whether something can change out from underneath you.
3
SPEAKER_00
And so in Python, you have a pointer to an array.
3
SPEAKER_00
And so you pass that pointer to an array around to things.
3
SPEAKER_00
if you pass it into a function, they may take that and scroll it away in some other data structure.
3
SPEAKER_00
And so you get your array back and you go to use it, and now somebody else is like putting stuff in your array.
3
SPEAKER_00
How do you reason about that?
3
SPEAKER_00
It gets to be very complicated and leads to lots of bugs, right?
3
SPEAKER_00
And so one of the things that, again, this is not something Mojo forces on you, but something that Mojo enables is a thing called value semantics.
3
SPEAKER_00
And what value semantics do is they take collections like arrays, like dictionaries, also tensors and strings and things like this that are much higher level and make them behave like proper values.
3
SPEAKER_00
And so it makes it look like if you pass these things around, you get a logical copy of all the data.
3
SPEAKER_00
And so if I pass you an array, it s your array.
3
SPEAKER_00
You can go do what you want to it.
3
SPEAKER_00
You re not going to hurt my array.
3
SPEAKER_00
Now, that is an interesting and very powerful design principle.
3
SPEAKER_00
It defines away a ton of bugs.
3
SPEAKER_00
You have to be careful to implement it in an efficient way.
3
SPEAKER_01
Is there a performance hit that s significant?
3
SPEAKER_00
Generally not, if you implement it the right way.
3
SPEAKER_00
But it requires a lot of very low-level getting the language right bits.
3
SPEAKER_01
I assume there d be a huge performance hit, because the benefit is really nice, because you don t get into the complex.
3
SPEAKER_01
Absolutely.
3
SPEAKER_00
Well, the trick is you can t do copies.
3
SPEAKER_00
So you have to provide the behavior of copying without doing the copy.
3
SPEAKER_00
Yeah, how do you do that?
3
SPEAKER_00
How do you do that?
3
SPEAKER_00
It s not magic, it s just, it s actually pretty cool.
3
SPEAKER_00
Well, so first, before we talk about how that works, let s talk about how it works in Python, right?
3
SPEAKER_00
So in Python, you define a person class, or maybe a person class is a bad idea.
3
SPEAKER_00
You define a database class, right?
3
SPEAKER_00
And a database class has an array of records, something like that, right?
3
SPEAKER_00
And so the problem is that if you pass in a record or a class instance into the database, it ll take a hold of that object, and then it assumes it has it.
3
SPEAKER_00
And if you re passing an object in, you have to know that that database is gonna take it, and therefore you shouldn t change it after you put it in the database, right?
3
SPEAKER_00
This is the problem.
3
SPEAKER_00
You just kind of have to know that.
3
SPEAKER_00
You just have to kind of know that, right?
3
SPEAKER_00
And so you roll out version one of the database, you just kind of have to know that.
3
SPEAKER_00
Of course, Lex uses his own database, right?
3
SPEAKER_01
Yeah.
3
SPEAKER_00
Right, because you built it.
3
SPEAKER_00
You understand how this works, right?
3
SPEAKER_00
Somebody else joins the team.
3
SPEAKER_00
They don t know this.
3
SPEAKER_00
Yes, right.
3
SPEAKER_00
And so now they suddenly get bugs You re having to maintain the database you shake your fist You argue the tenth time this happens.
3
SPEAKER_00
You re like, okay, we have to do something different, right?
3
SPEAKER_00
And so what you do is you go change your Python code and you change your database class to copy the record every time you add it and so what ends up happening is you say okay, I will do what s called a defensive copy and
3
SPEAKER_00
inside the database, and then that way, if somebody passes something in, I will have my own copy of it, and they can go do whatever, and they re not gonna break my thing.
3
SPEAKER_00
Okay, this is usually the two design patterns.
3
SPEAKER_00
If you look in PyTorch, for example, this is cloning a tensor, like there s a specific thing, and you have to know where to call it, and if you don t call it in the right place, you get these bugs, and this is state-of-the-art, right?
3
SPEAKER_00
So a different approach, so it s used in many languages, so I work with it in Swift, is you say, OK, well, let s provide value semantics.
3
SPEAKER_00
And so we want to provide the view that you get a logically independent copy, but we want to do that lazily.
3
SPEAKER_00
And so what we do is we say, OK, if you pass something into a function, it doesn t actually make a copy.
3
SPEAKER_00
What it actually does is it just increments a reference to it.
3
SPEAKER_00
And if you pass it around, you stick it in your database, it can go in the database you own it.
3
SPEAKER_00
and then you come back out of the stack, nobody s copied anything, you come back out of the stack and then the caller lets go of it, well then you ve just handed it off to the database, you ve transferred it, and there s no copies made.
3
SPEAKER_00
Now on the other hand, if your coworker goes and hands you a record and you pass it in, you stick it in the database, and then you go to town and you start modifying it, what happens is you get a copy lazily on demand.
3
SPEAKER_00
And so what this does is it gives you copies only when you need them, and it also, so it defines away the bugs, but it also generally reduces the number of copies in practice.
3
SPEAKER_01
And so it s- But the implementation details are tricky here.
3
SPEAKER_01
Yeah, so this is, yes.
3
SPEAKER_01
Something with reference counting, but to make it performant across a number of different kinds of objects.
3
SPEAKER_00
Yeah, so you need a couple of things.
3
SPEAKER_00
So this concept has existed in many different worlds, and so again, it s not novel research at all, right?
3
SPEAKER_00
The magic is getting the design right so that you can do this in a reasonable way, right?
3
SPEAKER_00
And so there s a number of components that go into this.
3
SPEAKER_00
One is when you re passing around, so we re talking about Python and reference counting at the expense of doing that,
3
SPEAKER_00
When you re passing values around, you don t want to do extra reference counting for no good reason.
3
SPEAKER_00
And so you have to make sure that you re efficient and you transfer ownership instead of duplicating references and things like that, which is a very low level problem.
3
SPEAKER_00
You also have to adopt this and you have to build these data structures.
3
SPEAKER_00
And so if you say,
3
SPEAKER_00
You know, Mojo has to be compatible with Python, so of course the default list is a reference semantic list that works the way you d expect in Python, but then you have to design a value semantic list.
3
SPEAKER_00
And so you just have to implement that, and then you implement the logic within.
3
SPEAKER_00
And so the role of the language here is to provide all the low-level hooks that allow the author of the type to be able to get and express this behavior without forcing it into all cases or hard-coding this into the language itself.
3
SPEAKER_01
But there s an ownership, so you re constantly tracking who owns the thing.
3
SPEAKER_00
Yes, and so there s a whole system called ownership, and so this is related to work done in the Rust community.
3
SPEAKER_00
Also the Swift community has done a bunch of work, and there s a bunch of different other languages that have all kind of, C++ actually has copy constructors and destructors and things like that, and so, and I mean, C++ has everything.
3
SPEAKER_00
So it has move constructors, it has like this whole world of things, and so,
3
SPEAKER_00
This is a body of work that s kind of been developing for many, many years now.
3
SPEAKER_00
And so Mojo takes some of the best ideas out of all these systems and remixes it in a nice way so that you get the power of something like the Rust programming language, but you don t have to deal with it when you don t want to, which is a major thing in terms of teaching and learning and being able to use and scale these systems.
3
SPEAKER_01
How does that play with argument conventions?
3
SPEAKER_01
What are they, why are they important?
3
SPEAKER_01
How does the value semantics, how does the transfer ownership work with the arguments when they re passed into functions?
3
SPEAKER_00
Yeah, so if you go deep into systems programming land, this isn t, again, this is not something for everybody, but if you go deep into systems programming land,
3
SPEAKER_00
what you encounter is you encounter these types that get weird.
3
SPEAKER_00
So if you re used to Python, you think about everything, I can just copy it around, I can go change it and mutate it and do these things and it s all cool.
3
SPEAKER_00
If you get into systems programming land, you get into these things like I have an atomic number, or I have a mutex, or I have a uniquely owned database handle, things like this, right?
3
SPEAKER_00
So these types you can t necessarily copy.
3
SPEAKER_00
Sometimes you can t necessarily even move them to a different address.
3
SPEAKER_00
And so what Mojo allows you to do is it allows you to express, hey, I don t want to get a copy of this thing.
3
SPEAKER_00
I want to actually just get a reference to it.
3
SPEAKER_00
And by doing that, what you can say is you can say, OK, if I m defining something weird, like an atomic number or something, it s like,
3
SPEAKER_00
It has to be, so an atomic number is an area in memory that multiple threads can access at a time without locks.
3
SPEAKER_00
And so the definition of an atomic number is multiple different things have to be poking at that, therefore they have to agree on where it is.
3
SPEAKER_00
And so you can t just like move it out from underneath one because it kind of breaks what it means.
3
SPEAKER_00
And so that s an example of a type that you can t even, you can t copy, you can t move it.
3
SPEAKER_00
Like once you create it, it has to be where it was, right?
3
SPEAKER_00
Now, if you look at many other examples, like a database sample, right?
3
SPEAKER_00
So, okay, well, what happens?
3
SPEAKER_00
How do you copy a database handle?
3
SPEAKER_00
Do you copy the whole database?
3
SPEAKER_00
That s not something you necessarily want to do.
3
SPEAKER_00
There s a lot of types like that where you want to be able to say that they are uniquely owned.
3
SPEAKER_00
So there s always one of this thing, or if I create a thing, I don t copy it.
3
SPEAKER_00
And so what Mojo allows you to do is it allows you to say,
3
SPEAKER_00
hey I want to pass around a reference to this thing without copying it and so it has borrowed conventions so you can say you can use it but you don t get to change it you can pass it by mutable reference and so if you do that then you can you get a reference to it but you can change it and so it manages all that kind of stuff.
3
SPEAKER_01
So it s just a really nice implementation of, like, C++ has, you know, the different kinds of pointers.
3
SPEAKER_01
Different kinds of implementations of smart pointers that you can explicitly define.
3
SPEAKER_01
But you re saying that s more like the weird case versus the common case.
3
SPEAKER_00
Well, it depends on where, I mean, I don t think I m a normal person, so, I mean, I m not one to call other people weird.
3
SPEAKER_00
But if you talk to a typical Python programmer, you re typically not thinking about this.
3
SPEAKER_00
This is a lower level of abstraction.
3
SPEAKER_00
Now, if you talk to a C++ programmer, certainly if you talk to a Rust programmer, again, they re not weird, they re delightful.
3
SPEAKER_00
These are all good people.
3
SPEAKER_00
Those folks will think about all the time.
3
SPEAKER_00
And so I look at this as there s a spectrum between very deep, low-level systems, I m gonna go poke the bits and care about how they re laid out in memory, all the way up to application and scripting and other things like this.
3
SPEAKER_00
And so it s not that anybody s right or wrong, it s about how do we build one system that scales.
3
SPEAKER_01
By the way, the idea of an atomic number has been something that always brought me deep happiness.
3
SPEAKER_01
Because the flip side of that, the idea that
3
SPEAKER_01
threads can just modify stuff asynchronously.
3
SPEAKER_01
Just the whole idea of concurrent programming is a source of infinite stress for me.
3
SPEAKER_00
Well, so this is where you jump into, you know, again, you zoom out and get out of programming languages or compilers and you just look at what the industry has done.
3
SPEAKER_00
my mind is constantly blown by this right and you look at what you know moore s law moore s law has this idea that like computers for a long time single thread performance just got faster and faster and faster and faster for free but then physics
3
SPEAKER_00
and other things intervened, and power consumption, other things started to matter.
3
SPEAKER_00
And so what ended up happening is we went from single-core computers to multi-core.
3
SPEAKER_00
Then we went to accelerators.
3
SPEAKER_00
This trend towards specialization of hardware is only going to continue.
3
SPEAKER_00
For years, us programming language nerds and compiler people have been saying, OK, well, how do we tackle multi-core?
3
SPEAKER_00
For a while, it was like, multi-core is the future.
3
SPEAKER_00
We have to get on top of this thing.
3
SPEAKER_00
Then it was multi-core is the default.
3
SPEAKER_00
What are we doing with this thing?
3
SPEAKER_00
And then it s like, there s chips with hundreds of cores in them.
3
SPEAKER_00
What will happen?
3
SPEAKER_00
I m super inspired by the fact that in the face of this, those machine learning people invented this idea of a tensor.
3
SPEAKER_00
And what is a tensor?
3
SPEAKER_00
A tensor is an arithmetic and algebraic concept.
3
SPEAKER_00
It s like an abstraction around a gigantic, parallelizable data set.
3
SPEAKER_00
And because of that, and because of things like TensorFlow and PyTorch, we re able to say, okay, we ll express the math
3
SPEAKER_00
of the system.
3
SPEAKER_00
This enables you to do automatic differentiations, enables you to do all these cool things.
3
SPEAKER_00
And it s an abstract representation.
3
SPEAKER_00
Well, because you have that abstract representation, you can now map it onto these parallel machines without having to control, OK, put that byte here, put that byte there, put that byte there.
3
SPEAKER_00
And this has enabled an explosion in terms of AI, compute,
3
SPEAKER_00
accelerators, like all the stuff, and so that s super, super exciting.
3
SPEAKER_01
What about the deployment and the execution across multiple machines?
3
SPEAKER_01
So you write that the modular compute platform dynamically partitions models with billions of parameters and distributes their execution across multiple machines, enabling unparalleled efficiency.
3
SPEAKER_01
By the way, the use of unparalleled in that sentence, anyway, enabling unparalleled efficiency scale and reliability for the largest workloads.
3
SPEAKER_01
So how do you do this abstraction of distributed deployment of large
3
SPEAKER_00
Yeah, so one of the really interesting tensions, so there s a whole bunch of stuff that goes into that.
3
SPEAKER_00
I ll pick a random walkthrough.
3
SPEAKER_00
If you go back and replay the history of machine learning, right?
3
SPEAKER_00
I mean, the brief, the most recent history of machine learning, because this is, as you know, very deep.
3
SPEAKER_00
I knew Lex when he had an AI podcast.
3
SPEAKER_00
So if you look at just TensorFlow and PyTorch, which is pretty recent history in the big picture, right?
3
SPEAKER_00
But
3
SPEAKER_00
TensorFlow is all about graphs.
3
SPEAKER_00
PyTorch, I think, pretty unarguably ended up winning.
3
SPEAKER_00
And why did it win?
3
SPEAKER_00
Mostly because of usability, right?
3
SPEAKER_00
And the usability of PyTorch is, I think, huge.
3
SPEAKER_00
And I think, again, that s a huge testament to the power of taking abstract, theoretical, technical concepts and bringing it to the masses, right?
3
SPEAKER_00
Now, the challenge with what the TensorFlow versus the PyTorch design points was that TensorFlow is kind of difficult to use for researchers, but it was actually pretty good for deployment.
3
SPEAKER_00
PyTorch is really good for researchers.
3
SPEAKER_00
It kind of is not super great for deployment, right?
3
SPEAKER_00
And so I think that we as an industry have been struggling.
3
SPEAKER_00
And if you look at what deploying a machine learning model today means, is that you ll have researchers who are, I mean, wicked smart, of course, but they re wicked smart at
3
SPEAKER_00
model architecture and data and calculus and they re wicked smart in various domains, they don t want to know anything about the hardware deployment or C++ or things like this, right?
3
SPEAKER_00
And so what s happened is you get people who train the model, they throw it over the fence, and then you have people that try to deploy the model.
3
SPEAKER_00
Well, every time you have a team A does X, they throw it over the fence, and team B does Y, you have a problem, because of course it never works the first time.
3
SPEAKER_00
And so you throw it over the fence, they figure out, okay, it s too slow, it won t fit, doesn t use the right operator, the tool crashes, whatever the problem is, then they have to throw it back over the fence.
3
SPEAKER_00
And every time you throw a thing over a fence, it takes three weeks of project managers and meetings and things like this.
3
SPEAKER_00
And so what we ve seen today is that getting models in production can take weeks or months.
3
SPEAKER_00
It s not atypical.
3
SPEAKER_00
I talk to lots of people, and you talk about a VP of software at some internet company trying to deploy a model.
3
SPEAKER_00
And they re like, why do I need a team of 45 people?
3
SPEAKER_00
It s so easy to train a model.
3
SPEAKER_00
Why can t I deploy it?
3
SPEAKER_00
And if you dig into this,
3
SPEAKER_00
Every layer is problematic.
3
SPEAKER_00
So if you look at the language piece, I mean, this is tip of the iceberg.
3
SPEAKER_00
It s a very exciting tip of the iceberg for folks, but you ve got Python on one side and C++ on the other side.
3
SPEAKER_00
Python doesn t really deploy.
3
SPEAKER_00
I mean, it can theoretically, technically in some cases, but often a lot of production teams will want to get things out of Python because they get better performance and control and whatever else.
3
SPEAKER_00
So Mojo can help with that.
3
SPEAKER_00
If you look at serving, so you talk about gigantic models, well, a gigantic model won t fit on one machine, right?
3
SPEAKER_00
And so now you have this model, it s written in Python, it has to be rewritten in C++.
3
SPEAKER_00
Now it also has to be carved up so that half of it runs on one machine, half of it runs on another machine.
3
SPEAKER_00
Or maybe it runs on 10 machines.
3
SPEAKER_00
Well, so now suddenly the complexity is exploding, right?
3
SPEAKER_00
And the reason for this is that if you look into TensorFlow, PyTorch, these systems, they weren t really designed for this world, right?
3
SPEAKER_00
They were designed for, you know, back in the day when we were
3
SPEAKER_00
starting and doing things, where it was a different, much simpler world.
3
SPEAKER_00
Like, you want to run ResNet-50 or some ancient model architecture like this.
3
SPEAKER_00
It was just a, it was a completely different world.
3
SPEAKER_00
Train on one GPU.
3
SPEAKER_00
Exactly.
3
SPEAKER_00
AlexNet.
3
SPEAKER_00
Yeah, AlexNet, right?
3
SPEAKER_00
The major breakthrough.
3
SPEAKER_00
And, um,
3
SPEAKER_00
And the world has changed, right?
3
SPEAKER_00
And so now the challenge is that TensorFlow, PyTorch, these systems, they weren t actually designed for LLMs.
3
SPEAKER_00
That was not a thing.
3
SPEAKER_00
And so where TensorFlow actually has amazing power in terms of scale and deployment and things like that, and I think Google is
3
SPEAKER_00
I mean, maybe not unmatched, but they re like incredible in terms of their capabilities and gigantic scale.
3
SPEAKER_00
Many researchers using PyTorch, right?
3
SPEAKER_00
And so PyTorch doesn t have those same capabilities.
3
SPEAKER_00
And so what Modular can do is it can help with that.
3
SPEAKER_00
Now, if you take a step back and say like, what is Modular doing, right?
3
SPEAKER_00
So Modular has like a bitter enemy that we re fighting against in the industry.
3
SPEAKER_00
And it s one of these things where everybody knows it, but nobody is usually willing to talk about it.
3
SPEAKER_00
the bitter enemy, the bitter thing that we have to destroy that we re all struggling with.
3
SPEAKER_00
And it s like all around.
3
SPEAKER_00
It s like fish can t see water.
3
SPEAKER_00
It s complexity.
3
SPEAKER_00
Sure.
4
SPEAKER_01
Yes, it s complexity.
4
SPEAKER_01
That was very philosophical of you, as you said.
4
SPEAKER_01
And so if you look at it, yes, it is on the hardware side.
4
SPEAKER_01
Yes.
4
SPEAKER_01
All these accelerators, all these software stacks that go with the accelerator, all these massive complexity over there.
4
SPEAKER_01
What s happening on the modeling side?
4
SPEAKER_01
Massive amount of complexity.
4
SPEAKER_01
Things are changing all the time.
4
SPEAKER_01
People are inventing.
4
SPEAKER_01
Turns out the research is not done.
4
SPEAKER_01
And so people want to be able to move fast.
4
SPEAKER_01
Transformers are amazing, but there s a ton of diversity even within Transformers.
4
SPEAKER_01
And what s the next Transformer?
4
SPEAKER_01
And you look into serving.
4
SPEAKER_01
Also, huge amounts of complexity.
4
SPEAKER_01
It turns out that all the cloud providers have all their very weird but very cool hardware for networking and all this kind of stuff, and it s all very complicated.
4
SPEAKER_01
People aren t using that.
4
SPEAKER_01
You look at classical serving.
4
SPEAKER_01
There s this whole world of people who know how to write high-performance servers with zero copy networking and all this fancy asynchronous I.O.
4
SPEAKER_01
and all these fancy things in the serving community.
4
SPEAKER_01
Very little that has pervaded into the machine learning world Right and why is that well?
4
SPEAKER_01
It s because again these systems have been built up over many years They haven t been rethought there hasn t been a first principles approach to this and so what modular is doing is we re saying okay?
4
SPEAKER_01
We ve built many of these things.
4
SPEAKER_01
So I ve worked on TensorFlow and TPUs and things like that.
4
SPEAKER_01
Other folks on our team have worked on PyTorch Core.
4
SPEAKER_01
We ve worked on Audix Runtime.
4
SPEAKER_01
We ve worked on many of these other systems.
4
SPEAKER_01
And so built systems like the Apple accelerators and all that kind of stuff.
4
SPEAKER_01
Our team is quite amazing.
4
SPEAKER_01
And so one of the things that roughly everybody at Modular is grumpy about is that when you re working on one of these projects, you have a first order goal.
4
SPEAKER_01
get the hardware to work, get the system to enable one more model, get this product out the door, enable the specific workload or make it, solve this problem for this product team, right?
4
SPEAKER_01
And nobody s been given a chance to actually do that step back.
4
SPEAKER_01
And so we as an industry, we didn t take two steps forward, we took like 18 steps forward in terms of all this really cool technology across compilers and systems and runtimes and heterogeneous computing, like all this kind of stuff.
4
SPEAKER_01
And like all this technology has been
4
SPEAKER_01
I wouldn t say beautifully designed, but it s been proven in different quadrants.
4
SPEAKER_01
You look at Google with TPUs.
4
SPEAKER_01
Massive, huge exaflops of compute strapped together into machines that researchers are programming in Python in a notebook.
4
SPEAKER_01
That s huge, that s amazing.
4
SPEAKER_01
It s incredible.
4
SPEAKER_01
And so you look at the technology that goes into that, and the algorithms are actually quite general.
4
SPEAKER_01
And so lots of other hardware out there and lots of other teams out there don t have the sophistication or maybe the years working on it or the budget or whatever that Google does, right?
4
SPEAKER_01
And so they should be getting access to the same algorithms, but they just don t have that, right?
4
SPEAKER_01
And so what Modular s doing is we re saying,
4
SPEAKER_01
Cool, this is not research anymore.
4
SPEAKER_01
We ve built auto-tuning in many systems.
4
SPEAKER_01
We ve built programming languages, right?
4
SPEAKER_01
And so have implemented C++, have implemented Swift, have implemented many of these things.
4
SPEAKER_01
And so it s hard, but it s not research.
4
SPEAKER_01
And you look at accelerators.
4
SPEAKER_01
Well, we know there s a bunch of different weird kind of accelerators, but they actually cluster together.
4
SPEAKER_01
And you look at GPUs.
4
SPEAKER_01
Well, there s a couple of major vendors of GPUs, and they maybe don t always get along, but their architectures are very similar.
4
SPEAKER_01
You look at CPUs.
4
SPEAKER_01
CPUs are still super important for the deployment side of things.
4
SPEAKER_01
You see new architectures coming out from all the cloud providers and things like this, and they re all super important.
4
SPEAKER_01
to the world, but they don t have the 30 years of development that the entrenched people do.
4
SPEAKER_01
And so what Modular can do is we re saying, okay, all this complexity, it s not bad complexity, it s actually innovation.
4
SPEAKER_01
And so it s innovation that s happening, and it s for good reasons, but have sympathy for the poor software people.
4
SPEAKER_01
I mean, again, I m generally a software person too.
4
SPEAKER_01
I love hardware.
4
SPEAKER_01
But software people want to build applications and products and solutions that scale over many years.
4
SPEAKER_01
They don t want to build a solution for one generation of hardware with one vendor s tools.
4
SPEAKER_01
And because of this, they need something that scales with them.
4
SPEAKER_01
They need something that works on cloud and mobile.
4
SPEAKER_01
right, because their product manager said, hey, I want it to have lower latency and it s better for personalization or whatever they decide, right?
4
SPEAKER_01
Products evolve.
4
SPEAKER_01
And so the challenge with the machine learning technology and the infrastructure that we have today in the industry is that it s all these point solutions.
4
SPEAKER_01
And because there are all these point solutions, it means that as your product evolves, you have to switch different technology stacks or switch to a different vendor.
4
SPEAKER_01
And what that does is that slows down progress.
4
SPEAKER_00
So basically, a lot of the things we ve developed in those little silos for machine learning tasks, you want to make that the first class citizen of a general purpose programming language that can then be compiled across all these kinds of hardware.
4
SPEAKER_01
Well, so it s not really about a programming language.
4
SPEAKER_01
I mean, the programming language is a component of the mission, right?
4
SPEAKER_01
And the mission is, or not literal, but our joking mission is to save the world from terrible AI software.
4
SPEAKER_01
Excellent, I love this.
4
SPEAKER_01
So, you know, if you look at this mission, you need a syntax.
4
SPEAKER_01
So that s so yes, you need a program language, right?
4
SPEAKER_01
And and like we wouldn t have to build the programming language if one existed Right, so if Python was already good enough, then cool.
4
SPEAKER_01
We would just used it, right?
4
SPEAKER_01
We re not just doing very large-scale expensive engineering projects for the sake of it.
4
SPEAKER_01
Like it s to solve a problem, right?
4
SPEAKER_01
It s also about
4
SPEAKER_01
accelerators.
4
SPEAKER_01
It s also about exotic numerics and bfloat16 and matrix multiplications and convolutions and this kind of stuff.
4
SPEAKER_01
Within the stack, there are things like kernel fusion.
4
SPEAKER_01
That s an esoteric but really important thing that leads to much better performance and much more general research hackability together.
4
SPEAKER_01
Right, and so.
4
SPEAKER_00
And that s enabled by the ASICs.
4
SPEAKER_00
That s enabled by certain hardware.
4
SPEAKER_00
So it s like, where s the dance between
4
SPEAKER_00
I mean, there s several questions here, like how do you add a piece of hardware to the stack?
4
SPEAKER_00
Like if I have this genius invention of a specialized accelerator, how do I add that to the modular framework?
4
SPEAKER_00
And also, how does modular as a standard start to define the kinds of hardware that should be developed?
4
SPEAKER_01
Yeah, so let me take a step back and talk about status quo.
4
SPEAKER_01
And so if you go back to TensorFlow 1, PyTorch 1, this kind of time frame, and these have all evolved and gotten way more complicated.
4
SPEAKER_01
So let s go back to the glorious simple days.
4
SPEAKER_01
These things basically were CPUs and CUDA.
4
SPEAKER_01
and so what you do is you say go do a dense layer and a dense layer has a matrix multiplication in it right and so when you say that you say go do this big operation of matrix multiplication and if it s on a gpu kick off a cuda kernel if it s on a cpu go do
4
SPEAKER_01
like an Intel algorithm or something like that with the Intel MKL, okay?
4
SPEAKER_01
Now, that s really cool if you re either NVIDIA or Intel, right?
4
SPEAKER_01
But then more hardware comes in, right?
4
SPEAKER_01
And on one axis you have more hardware coming in, on the other hand you have an explosion of innovation in AI.
4
SPEAKER_01
And so what happened with both TensorFlow and PyTorch is that the explosion of innovation in AI has led to, it s not just about matrix multiplication and convolution, these things have now like 2,000 different operators.
4
SPEAKER_01
And on the other hand, you have, I don t know how many pieces of hardware there are out there, it s a lot.
4
SPEAKER_01
It s not even hundreds, it s probably thousands.
4
SPEAKER_01
And across all of Edge and across all the different things.
4
SPEAKER_00
That are used at scale.
4
SPEAKER_01
Yeah, exactly.
4
SPEAKER_00
Also, it s not just like AI is everywhere.
4
SPEAKER_00
It s not a handful of TPU alternatives.
4
SPEAKER_01
It s every phone, often with many different chips inside of it from different vendors.
4
SPEAKER_01
AI is everywhere, it s a thing.
4
SPEAKER_00
Why are they all making their own chips?
4
SPEAKER_00
Why is everybody making their own thing?
4
SPEAKER_00
Is that a good thing, for sure?
4
SPEAKER_01
So Chris s philosophy on hardware?
4
SPEAKER_01
Yeah.
4
SPEAKER_01
Right, so my philosophy is that there isn t one right solution.
4
SPEAKER_01
Right, and so I think that, again, we re at the end of Moore s Law, specialization happens.
4
SPEAKER_01
If you re building, if you re training GPT-5, you want some crazy supercomputer data center thingy, if you re making a smart camera that runs on batteries, you want something that looks very different.
4
SPEAKER_01
If you re building a phone, you want something that looks very different.
4
SPEAKER_01
If you have something like a laptop, you want something that looks,
4
SPEAKER_01
maybe similar, but a different scale, right?
4
SPEAKER_01
And so AI ends up touching all of our lives, robotics, right?
4
SPEAKER_01
And like lots of different things.
4
SPEAKER_01
And so as you look into this, these have different power envelopes.
4
SPEAKER_01
There s different trade-offs in terms of the algorithms.
4
SPEAKER_01
There s new innovations in sparsity and other data formats and things like that.
4
SPEAKER_01
And so hardware innovation, I think, is a really good thing, right?
4
SPEAKER_01
And what I m interested in is unlocking that innovation.
4
SPEAKER_01
There s also like analog and quantum and like all the,
4
SPEAKER_01
The really weird stuff, right?
4
SPEAKER_01
And so if somebody can come up with a chip that uses analog computing and it s 100x more power efficient, think what that would mean in terms of the daily impact on the products we use.
4
SPEAKER_01
That d be huge.
4
SPEAKER_01
Now, if you re building an analog computer, you may not be a compiler specialist.
4
SPEAKER_01
These are different skill sets, right?
4
SPEAKER_01
And so you can hire some compiler people if you re running a big company maybe, but it turns out,
4
SPEAKER_01
these are really exotic new generation of compilers.
4
SPEAKER_01
This is a different thing.
4
SPEAKER_01
So if you take a step back out and come back to what is the status quo, the status quo is that
4
SPEAKER_01
If you re Intel or you re NVIDIA, you keep up with the industry and you chase, and okay, there s 1900 now, there s 2000 now, there s 2100, and you have a huge team of people that are trying to keep up and tune and optimize, and even when one of the big guys comes out with a new generation of their chip, they have to go back and rewrite all these things.
4
SPEAKER_01
So really, it s only powered by having hundreds of people that are all frantically trying to keep up, and what that does is that keeps out the little guys.
4
SPEAKER_01
And sometimes they re not so little guys, the big guys that are also just not in those dominant positions.
4
SPEAKER_01
And so what has been happening, and so a lot of, you talk about the rise of new exotic crazy accelerators, is people have been trying to turn this from a let s go write lots of special kernels problem into a compiler problem.
4
SPEAKER_01
And so we and I contributed to this as well.
4
SPEAKER_01
We as an industry went into it like, let s go make this compiler problem phase, let s call it.
4
SPEAKER_01
And much of the industry is still in this phase, by the way.
4
SPEAKER_01
So I wouldn t say this phase is over.
4
SPEAKER_01
And so the idea is to say, look, OK, what a compiler does is it provides a much more general, extensible,
4
SPEAKER_01
a hackable interface for dealing with the general case.
4
SPEAKER_01
And so within machine learning algorithms, for example, people figured out that, hey, if I do a matrix multiplication and I do a ReLU, the classic activation function, it is way faster to do one pass over the data and then do the ReLU on the output where I m writing out the data, because ReLU is just a maximum operation, max of zero.
4
SPEAKER_01
It s an amazing optimization, take Matmul, ReLU, squish together in one operation, now we have Matmul ReLU.
4
SPEAKER_01
Well, wait a second.
4
SPEAKER_01
If I do that, now I just went from having two operators to three.
4
SPEAKER_01
But now I figure out, okay, well, there s a lot of activation functions.
4
SPEAKER_01
What about leaky value?
4
SPEAKER_01
What about, like a million things that are out there, right?
4
SPEAKER_01
And so as I start fusing these in, now I get permutations of all these algorithms, right?
4
SPEAKER_01
And so what the compiler people said is they said, hey, cool, well, I will go enumerate all the algorithms, and I will enumerate all the pairs, and I will actually generate a kernel for you.
4
SPEAKER_01
And I think that this has been very, very useful for the industry.
4
SPEAKER_01
This is one of the things that powers Google TPUs.
4
SPEAKER_01
PyTorch 2 is rolling out really cool compiler stuff with Triton and other technology and things like this.
4
SPEAKER_01
And so the compiler people are kind of coming into their 4 and saying like, awesome, this is a compiler problem, we ll compiler it.
4
SPEAKER_01
Here s the problem.
4
SPEAKER_01
Not everybody s a compiler person.
4
SPEAKER_01
I love compiler people, trust me, right?
4
SPEAKER_01
But not everybody can or should be a compiler person.
4
SPEAKER_01
It turns out that there are people that know analog computers really well, or they know some GPU internal architecture thing really well, or they know some crazy, sparse, numeric, interesting algorithm that is the cusp of research, but they re not compiler people.
4
SPEAKER_01
And so one of the challenges with this new wave of technology trying to turn everything into a compiler
4
SPEAKER_01
Because again, it has excluded a ton of people.
4
SPEAKER_01
And so you look at what does Mojo do?
4
SPEAKER_01
What does the modular stack do?
4
SPEAKER_01
It brings programmability back into this world.
4
SPEAKER_01
It enables, I wouldn t say normal people, but a different kind of delightful nerd that cares about numerics, or cares about hardware, or cares about things like this, to be able to express that in the stack and extend the stack without having to
4
SPEAKER_01
actually go hack the compiler itself.
4
SPEAKER_00
So extend the stack on the algorithm side, and then on the hardware side.
4
SPEAKER_01
Yeah, so again, go back to like the simplest example of int, right?
4
SPEAKER_01
And so what both Swift and Mojo and other things like this did is we said, okay, pull magic out of the compiler and put it in the standard library.
4
SPEAKER_01
And so what Modular s doing with the engine that we re providing and this very deep technology stack, which goes into heterogeneous runtimes and a whole bunch of really cool things, this whole stack allows that stack to be extended and hacked and changed by researchers and by hardware innovators and by people who know things that we don t know.
4
SPEAKER_01
Because Modular has some smart people, but we don t have all the smart people, it turns out.
4
SPEAKER_00
What are heterogeneous runtimes?
4
SPEAKER_01
Yeah, so what is heterogeneous, right?
4
SPEAKER_01
So heterogeneous just means many different kinds of things together.
4
SPEAKER_01
And so the simplest example you might come up with is a CPU and a GPU.
4
SPEAKER_01
And so it s a simple heterogeneous computer to say, I will run my data loading and preprocessing and other algorithms on the CPU.
4
SPEAKER_01
And then once I get it into the right shape, I shove it into the GPU.
4
SPEAKER_01
I do a lot of matrix multiplications and convolutions and things like this.
4
SPEAKER_01
And then I get it back out, and I
4
SPEAKER_01
do some reductions and summaries and they shove it across the wire, across the network to another machine, right?
4
SPEAKER_01
And so you ve got now what are effectively two computers, a CPU and a GPU talking to each other, working together in a heterogeneous system.
4
SPEAKER_01
But that was 10 years ago.
4
SPEAKER_01
You look at a modern cell phone,
4
SPEAKER_01
Modern cell phone, you ve got CPUs, and they re not just CPUs, there s like big dot little CPUs, and so there s multiple different kinds of CPUs that are kind of working together that are multi-core.
4
SPEAKER_01
You ve got GPUs, you ve got neural network accelerators, you ve got dedicated hardware blocks for media, so for video decode and JPEG decode and things like this.
4
SPEAKER_01
And so you ve got this massively complicated system.
4
SPEAKER_01
And this isn t just cell phones, every laptop these days is doing the same thing.
4
SPEAKER_01
And all of these blocks can run at the same time.
4
SPEAKER_01
and need to be choreographed, right?
4
SPEAKER_01
And so, again, one of the cool things about machine learning is it s moving things to like data flow graphs and higher level abstractions and tensors and these things that it doesn t specify, here s how to do the algorithm.
4
SPEAKER_01
It gives the system a lot more flexibility in terms of how to translate or map or compile it onto the system that you have.
4
SPEAKER_01
And so what you need, you know, the bottom-est part of the layer there is a way for all these devices to talk to each other.
4
SPEAKER_01
And so this is one thing that I m very passionate about.
4
SPEAKER_01
I mean, you know I m a nerd.
4
SPEAKER_01
But all these machines and all these systems are effectively parallel computers running at the same time, sending messages to each other.
4
SPEAKER_01
And so they re all fully asynchronous.
4
SPEAKER_01
Well, this is actually a small version of the same problem you have in a data center.
4
SPEAKER_01
In a data center, you now have multiple different machines, sometimes very specialized, sometimes with GPUs or TPUs in one node, and sometimes with disks in another node.
4
SPEAKER_01
And so you get a much larger scale heterogeneous computer.
4
SPEAKER_01
And so what ends up happening is you have this multi-layer abstraction of hierarchical parallelism, hierarchical
4
SPEAKER_01
asynchronous communication and making that, again, my enemy is complexity.
4
SPEAKER_01
By getting that away from being different specialized systems at every different part of the stack and having more consistency and uniformity, I think we can help lift the world and make it much simpler and actually get used.
4
SPEAKER_00
But how do you leverage the strength of the different specialized systems?
4
SPEAKER_00
So looking inside the smartphone, there s just, I don t know, five, six computers essentially inside the smartphone.
4
SPEAKER_00
How do you
4
SPEAKER_00
without trying to minimize the explicit, making it explicit which computer is supposed to be used for which operation.
4
SPEAKER_01
Yeah, so there s a pretty well-known algorithm, and what you re doing is you re looking at two factors.
4
SPEAKER_01
You re looking at the factor of sending data from one thing to another, right?
4
SPEAKER_01
So it takes time to get it from that side of the chip to that side of the chip, and things like this.
4
SPEAKER_01
And then you re looking at what is the time it takes to do an operation on a particular block.
4
SPEAKER_01
So take CPUs.
4
SPEAKER_01
CPUs are,
4
SPEAKER_01
fully general.
4
SPEAKER_01
They can do anything.
4
SPEAKER_01
But then you have a neural net accelerator that s really good at matrix multiplications.
4
SPEAKER_01
And so you say, OK, well, if my workload is all matrix multiplications, I start
5
SPEAKER_00
up i send the data over the neural net thing it goes and does matrix multiplications when it s done it sends me back the result all is good right and so the simplest thing is just saying do matrix do matrix operations over there right
5
SPEAKER_00
But then you realize you get a little bit more complicated, because you can do matrix multiplications on a GPU.
5
SPEAKER_00
You can do it on a neural net accelerator.
5
SPEAKER_00
You can do it on a CPU.
5
SPEAKER_00
And they ll have different trade-offs and costs.
5
SPEAKER_00
And it s not just matrix multiplication.
5
SPEAKER_00
And so what you actually look at is you look at, I have generally a graph of compute.
5
SPEAKER_00
I want to do a partitioning.
5
SPEAKER_00
I want to look at the communication
5
SPEAKER_00
the bisection bandwidth and the overhead and the sending of all these different things and build a model for this and then decide, okay, it s an optimization problem, where do I want to place this compute?
5
SPEAKER_01
So it s the old school theoretical computer science problem of scheduling.
5
SPEAKER_01
And then how does, presumably it s possible to somehow magically include autotune into this?
5
SPEAKER_00
Absolutely.
5
SPEAKER_00
So, I mean, in my opinion, this is an opinion, this is not, not everybody would agree with this, but in my opinion, the world benefits from simple and predictable systems at the bottom that you can control.
5
SPEAKER_00
But then once you have a predictable execution layer, you can build lots of different policies on top of it, right?
5
SPEAKER_00
And so one policy can be that
5
SPEAKER_00
the human programmer says, do that here, do that here, do that here, do that here, and like fully manually controls everything, and the system should just do it.
5
SPEAKER_00
Then you quickly get in the mode of like, I don t wanna have to tell it to do it.
5
SPEAKER_00
And so the next logical step that people typically take is they write some terrible heuristic.
5
SPEAKER_00
Oh, if it s a major small location, do it over there.
5
SPEAKER_00
Or if it s floating point, do it on the GPU.
5
SPEAKER_00
If it s integer, do it on the CPU.
5
SPEAKER_00
Like something like that, right?
5
SPEAKER_00
And then you then get into this mode of like people care more and more and more.
5
SPEAKER_00
And you say, OK, well, let s actually like make the heuristic better.
5
SPEAKER_00
Let s get into auto-tuning.
5
SPEAKER_00
Let s actually do a search of the space to decide, well, what is actually better, right?
5
SPEAKER_00
Well, then you get into this problem where you realize this is not a small space.
5
SPEAKER_00
This is a many dimensional.
5
SPEAKER_00
hyperdimensional space that you cannot exhaustively search.
5
SPEAKER_00
So, do you know of any algorithms that are good at searching very complicated spaces for?
5
SPEAKER_00
Don t tell me you re gonna turn this into a machine learning problem.
5
SPEAKER_00
So then you turn it into a machine learning problem and then you have a space of genetic algorithms and reinforcement learning and like all these, all these cool things.
5
SPEAKER_01
But can you include that into the stack, into the module stack?
5
SPEAKER_00
Yeah, yeah.
5
SPEAKER_01
Where does it sit, where does it live?
5
SPEAKER_01
Is it a separate thing or is it part of the compilation?
5
SPEAKER_00
So, you start from simple and predictable models.
5
SPEAKER_00
And so you can have full control, and you can have coarse-grained knobs that, like, nudge the system so you don t have to do this.
5
SPEAKER_00
But if you really care about getting the best, you know, the last ounce out of a problem, then you can use additional tools.
5
SPEAKER_00
And there, the cool thing is you don t want to do this every time you run a model.
5
SPEAKER_00
You want to figure out the right answer and then cache it.
5
SPEAKER_00
Once you do that, you can say, okay, cool.
5
SPEAKER_00
I can get up and running very quickly.
5
SPEAKER_00
I can get good execution out of my system.
5
SPEAKER_00
I can decide if something s important, and if it s important, I can go through a bunch of machines at it and do a big expensive search over the space using whatever technique I feel like it s really up to the problem.
5
SPEAKER_00
Then when I get the right answer, cool, I can just start using it.
5
SPEAKER_00
And so you can get out of this trade-off between, okay, am I going to spend forever doing a thing, or do I get up and running quickly and it s a quality result?
5
SPEAKER_00
These are actually not in contention with each other if the system s designed to scale.
5
SPEAKER_01
You started and did a little bit of a whirlwind overview of how you get 35,000x speed up or more over Python.
5
SPEAKER_01
Jeremy Howard did a really great presentation about sort of the basic, like, look at the code, here s how you get the speed up.
5
SPEAKER_01
Like you said, that s something we could...
5
SPEAKER_01
probably developers can do for their own code to see how you can get these gigantic speedups.
5
SPEAKER_01
But can you maybe speak to the machine learning tasks in general?
5
SPEAKER_01
How do you make some of this code fast and specific?
5
SPEAKER_01
What would you say is the main bottleneck for machine learning tasks?
5
SPEAKER_01
So are we talking about matmul, matrix multiplication, how do you make that fast?
5
SPEAKER_00
So, I mean, if you just look at the Python problem, right, you can say, how do I make Python faster?
5
SPEAKER_00
And there s been a lot of people that have been working on the, okay, how do I make Python 2X faster, 10X faster, or something like that, right?
5
SPEAKER_00
And there ve been a ton of projects in that vein, right?
5
SPEAKER_00
Mojo started from the, what can the hardware do?
5
SPEAKER_00
Like, what is the limit of physics?
5
SPEAKER_00
What is the speed of light?
5
SPEAKER_00
What is the, like, how fast can this thing go?
5
SPEAKER_00
And then, how do I express that?
5
SPEAKER_00
Right, and so it wasn t anchored relatively on make Python a little bit faster, it s saying,
5
SPEAKER_00
Cool, I know what the hardware can do, let s unlock that, right?
5
SPEAKER_01
Now, when you, when- You can just say how gutsy that is to be in the meeting, and as opposed to trying to see how do we get the improvement, it s like, what can the physics do?
5
SPEAKER_00
I mean, maybe I m a special kind of nerd, but you look at that, what is the limit of physics, how fast can these things go, right?
5
SPEAKER_00
when you start looking at that, typically it ends up being a memory problem.
5
SPEAKER_00
And so today, particularly with these specialized accelerators, the problem is that you can do a lot of math within them, but you get bottleneck sending data back and forth to memory, whether it be local memory or distant memory or disk or whatever it is, and that bottleneck, particularly as the training sizes get large, as you start doing tons of inferences all over the place, that becomes a huge bottleneck for people.
5
SPEAKER_00
So again, what happened is we went through a phase of many years where people took the special case and hand-tuned it, and tweaked it and tricked it out, and they knew exactly how the hardware worked, and they knew the model, and they made it fast.
5
SPEAKER_00
didn t generalize.
5
SPEAKER_00
And so you can make, you know, Resonant 50 or some, or AlexNet or something, Inception V1, like you can do that, right, because the models are small, they fit in your head, right?
5
SPEAKER_00
But as the models get bigger, more complicated, as the machines get more complicated, it stops working, right?
5
SPEAKER_00
And so this is where things like kernel fusion come in.
5
SPEAKER_00
So what is kernel fusion?
5
SPEAKER_00
This is this idea of saying, let s avoid going to memory, and let s do that by building a new hybrid
5
SPEAKER_00
kernel and a numerical algorithm that actually keeps things in the accelerator instead of having to write all the way out to memory.
5
SPEAKER_00
What s happened with these accelerators now is you get multiple levels of memory.
5
SPEAKER_00
Like in a GPU, for example, you ll have global memory and local memory and like all these things.
5
SPEAKER_00
If you zoom way into how hardware works, the register file is actually a memory.
5
SPEAKER_00
So the registers are like an L0 cache.
5
SPEAKER_00
And so a lot of taking advantage of the hardware ends up being fully utilizing the full power in all of its capability.
5
SPEAKER_00
And this has a number of problems, right?
5
SPEAKER_00
One of which is, again, the complexity of disaster, right?
5
SPEAKER_00
There s too much hardware.
5
SPEAKER_00
Even if you just say, let s look at the chips from one line of vendor, like Apple or Intel or whatever it is,
5
SPEAKER_00
Each version of the chip comes out with new features, and they change things so that it takes more time or less time to do different things.
5
SPEAKER_00
And you can t rewrite all the software whenever a new chip comes out, right?
5
SPEAKER_00
And so this is where you need a much more scalable approach.
5
SPEAKER_00
And this is what Mojo and what the modular stack provides, is it provides
5
SPEAKER_00
this infrastructure and the system for factoring all this complexity and then allowing people to express algorithms.
5
SPEAKER_00
You talk about autotuning, for example, express algorithms in a more portable way so that when a new chip comes out, you don t have to rewrite it all.
5
SPEAKER_00
So to me, I kind of joke, what is a compiler?
5
SPEAKER_00
Well, there s many ways to explain that.
5
SPEAKER_00
You convert thing A into thing B, and you convert source code to machine code.
5
SPEAKER_00
You can talk about many, many things that compilers do.
5
SPEAKER_00
But to me, it s about a bag of tricks.
5
SPEAKER_00
It s about a system and a framework that you can hang complexity.
5
SPEAKER_00
It s a system that can then generalize and it can work on problems that are bigger than fit in one human s head, right?
5
SPEAKER_00
And so what that means, what a good stack and what the modular stack provides is the ability to walk up to it with a new problem and it ll generally work quite well.
5
SPEAKER_00
And that s something that a lot of machine learning infrastructure and tools and technologies don t have.
5
SPEAKER_00
Typical state of the art today is you walk up, particularly if you re deploying, if you walk up with a new model, you try to push it through the converter and the converter crashes.
5
SPEAKER_00
That s crazy.
5
SPEAKER_00
The state of ML tooling today is not anything that a C programmer would ever accept, right?
5
SPEAKER_00
And it s always been this kind of flaky set of tooling that s never been integrated well, and it s never worked together because it s not designed together.
5
SPEAKER_00
It s built by different teams, it s built by different hardware vendors, it s built by different systems, it s built by different internet companies that are trying to solve their problems, right?
5
SPEAKER_00
And so that means that we get this fragmented, terrible mess of complexity.
5
SPEAKER_01
So, I mean, the specifics of, I mean, Jeremy showed this.
5
SPEAKER_01
There s the vectorize function, which I guess is built into Mojo.
5
SPEAKER_00
Vectorize, as he showed, is built into the library.
5
SPEAKER_01
Into the library, it s in a library.
5
SPEAKER_01
Vectorize, parallelize, which vectorizes more low level, parallelizes higher level.
5
SPEAKER_01
There s the tiling thing, which is how he demonstrated the,
5
SPEAKER_00
Autotune, I think.
5
SPEAKER_00
So think about this in hierarchical levels of abstraction.
5
SPEAKER_00
If you zoom all the way into a compute problem, you have one floating point number.
5
SPEAKER_00
So then you say, okay, I can do things one at a time in an interpreter.
5
SPEAKER_00
It s pretty slow.
5
SPEAKER_00
So I can get to doing one at a time in a compiler, I can see.
5
SPEAKER_00
Then I can get to doing four or eight or 16 at a time with vectors.
5
SPEAKER_00
That s called vectorization.
5
SPEAKER_00
Then you can say, hey, I have a whole bunch of different, you know, what a multi-core computer is, is it s basically a bunch of computers, right?
5
SPEAKER_00
So they re all independent computers that can talk to each other and they share memory.
5
SPEAKER_00
And so now what Parallelize does is it says, okay, run multiple instances on different computers.
5
SPEAKER_00
and now they can all work together on a problem, right?
5
SPEAKER_00
And so what you re doing is you re saying, keep going out to the next level out, and as you do that, how do I take advantage of this?
5
SPEAKER_00
So tiling is a memory optimization, right?
5
SPEAKER_00
It says, okay, let s make sure that we re keeping the data close to the compute part of the problem, instead of sending it all back and forth through memory every time I load a block.
5
SPEAKER_00
And the size of the block size is all, that s how you get to the autotune to make sure it s optimized.
5
SPEAKER_00
Yeah, well, so all of these, the details matter so much to get good performance.
5
SPEAKER_00
This is another funny thing about machine learning and high-performance computing that is very different than C compilers we all grew up with, where if you get a new version of GCC or a new version of Clang or something like that, maybe something will go 1% faster.
5
SPEAKER_00
Right?
5
SPEAKER_00
And so compiler engineers will work really, really, really hard to get half a percent out of your C code, something like that.
5
SPEAKER_00
But when you re talking about an accelerator or an AI application, or you re talking about these kinds of algorithms, and these are things people used to write in Fortran, for example, right?
5
SPEAKER_00
If you get it wrong, it s not 5% or 1%.
5
SPEAKER_00
It could be 2x or 10x, right?
5
SPEAKER_00
If you think about it, you really want to make use of the full memory you have, the cache, for example.
5
SPEAKER_00
But if you use too much space, it doesn t fit in the cache, now you re gonna be thrashing all the way back out to main memory.
5
SPEAKER_00
And these can be 2x, 10x, major performance differences.
5
SPEAKER_00
And so this is where getting these magic numbers and these things right is really actually quite important.
5
SPEAKER_01
So you mentioned that Moji is a superset of Python.
5
SPEAKER_01
Can you run Python code as if it s Mojo code?
5
SPEAKER_00
Yes, yes.
5
SPEAKER_00
And this has two sides of it.
5
SPEAKER_00
So Mojo s not done yet, so I ll give you a disclaimer.
5
SPEAKER_00
Mojo s not done yet, but already we see people that take small pieces of Python code, move it over, they don t change it, and you can get 12x speedups.
5
SPEAKER_00
Like somebody was just tweeting about that yesterday, which is pretty cool, right?
5
SPEAKER_00
And again, interpreters, compilers, right?
5
SPEAKER_00
And so without changing any code, also this is not JIT compiling or doing anything fancy, this is just,
5
SPEAKER_00
basic stuff and move it straight over now mojo will continue to grow out and as it grows out it will have more and more and more features and Our North Star is to be a full superset of Python and so you can bring over Basically arbitrary Python code and have it just work and it may not always be 12x faster But but it should be at least as fast and way faster in many cases.
5
SPEAKER_00
This is the goal.
5
SPEAKER_00
I
5
SPEAKER_00
Now it ll take time to do that, and Python is a complicated language.
5
SPEAKER_00
There s not just the obvious things, but there s also non-obvious things that are complicated, like we have to be able to talk to CPython packages that talk to the CAPI, and there s a bunch of pieces to this.
5
SPEAKER_01
So you have to, I mean, just to make explicit the obvious,
5
SPEAKER_01
it may not be so obvious until you think about it.
5
SPEAKER_01
So to run Python code, that means you have to run all the Python packages and libraries.
5
SPEAKER_01
So that means what?
5
SPEAKER_01
What s the relationship between Mojo and CPython, the interpreter that s...
5
SPEAKER_01
presumably would be tasked with getting those packages to work.
5
SPEAKER_00
Yep.
5
SPEAKER_00
So in the fullness of time, Mojo will solve for all the problems and you ll be able to move Python packages over and run them in Mojo.
5
SPEAKER_00
Without the CPython.
5
SPEAKER_00
Without CPython.
5
SPEAKER_00
Someday.
5
SPEAKER_00
Not today, but someday.
5
SPEAKER_00
And that ll be a beautiful day because then you ll get a whole bunch of advantages and you ll get massive speed ups and things like this.
5
SPEAKER_01
But you can do that one at a time, right?
5
SPEAKER_01
You can move packages one at a time.
5
SPEAKER_00
Exactly.
5
SPEAKER_00
But we re not willing to wait for that.
5
SPEAKER_00
Python is too important.
5
SPEAKER_00
The ecosystem is too broad.
5
SPEAKER_00
We want to both be able to build Mojo out.
5
SPEAKER_00
We also want to do it the right way without time, like without intense time pressure.
5
SPEAKER_00
We re obviously moving fast.
5
SPEAKER_00
And so what we do is we say, OK, well, let s make it so you can import an arbitrary existing package.
5
SPEAKER_00
arbitrary, including like you write your own on your local disk or whatever.
5
SPEAKER_00
It s not, it s not like a standard pack, like an arbitrary package and import that using CPython.
5
SPEAKER_00
But CPython already runs all the packages, right?
5
SPEAKER_00
And so what we do is we built an integration layer where we can actually use CPython.
5
SPEAKER_00
Again, I m practical.
5
SPEAKER_00
And to actually just load and use all the existing packages as they are.
5
SPEAKER_00
The downside of that is you don t get the benefits of Mojo for those packages, right?
5
SPEAKER_00
And so they ll run as fast as they do in the traditional CPython way.
5
SPEAKER_00
But what that does is that gives you an incremental migration path.
5
SPEAKER_00
And so if you say, hey, cool, well, the Python ecosystem is vast.
5
SPEAKER_00
I want all of it to just work.
5
SPEAKER_00
But there s certain things that are really important.
5
SPEAKER_00
And so if I m doing weather forecasting or something, well, I want to be able to load all the data.
5
SPEAKER_00
I want to be able to work with it.
5
SPEAKER_00
And then I have my own crazy algorithm inside of it.
5
SPEAKER_00
Well, normally, I d write that in C++.
5
SPEAKER_00
If I can write in Mojo and have one system that scales, well, that s way easier to work with.
5
SPEAKER_01
Is it hard to do that, to have that layer that s running CPython?
5
SPEAKER_01
Because is there some communication back and forth?
5
SPEAKER_00
Yes, it s complicated.
5
SPEAKER_00
I mean, this is what we do.
5
SPEAKER_00
So, I mean, we make it look easy, but it is complicated.
5
SPEAKER_00
But what we do is we use the CPython existing interpreter.
5
SPEAKER_00
So it s running its own bytecodes, and that s how it provides full compatibility.
5
SPEAKER_00
And then it gives us CPython objects.
5
SPEAKER_00
And we use those objects as is.
5
SPEAKER_00
And so that way we re fully compatible with all the CPython objects and all the, you know, it s not just the Python part, it s also the C packages, the C libraries underneath them, because they re often hybrid.
5
SPEAKER_00
And so we can fully run and we re fully compatible with all that.
5
SPEAKER_00
And the way we do that is that we have to play by the rules, right?
5
SPEAKER_00
And so we keep objects in that representation when they re coming from that world.
5
SPEAKER_00
What s the representation that s being used?
5
SPEAKER_00
In memory.
5
SPEAKER_00
You d have to know a lot about how the CPython interpreter works.
5
SPEAKER_00
It has,
5
SPEAKER_00
for example, reference counting, but also different rules on how to pass pointers around and things like this, super low-level fiddly, and it s not like Python, it s like how the interpreter works, okay?
5
SPEAKER_00
And so that gets all exposed out, and then you have to define wrappers around the low-level C code, right?
5
SPEAKER_00
And so what this means is you have to know not only C,
5
SPEAKER_00
which is a different role from Python, obviously.
5
SPEAKER_00
Not only Python, but the wrappers.
5
SPEAKER_00
But the interpreter and the wrappers and the implementation details and the conventions, and it s just this really complicated mess.
5
SPEAKER_00
And when you do that, now suddenly you have a debugger that debugs Python, they can t step into C code.
5
SPEAKER_00
So you have this two-world problem, right?
5
SPEAKER_00
And so by pulling this all into Mojo, what you get is you get one world.
5
SPEAKER_00
You get the ability to say, cool, I have untyped, very dynamic, beautiful, simple code.
5
SPEAKER_00
Okay, I care about performance for whatever reason, right?
5
SPEAKER_00
There s lots of reasons you might care.
5
SPEAKER_00
And so then you add types, you can parallelize things, you can vectorize things, you can use these techniques.
6
SPEAKER_00
which are general techniques to solve a problem, and then you can do that by staying in the system.
6
SPEAKER_00
And if you have that one Python package that s really important to you, you can move it to Mojo, you get massive performance benefits on that, and other advantages.
6
SPEAKER_00
If you like stack types, it s nice if they re enforced.
6
SPEAKER_00
Some people like that, right?
6
SPEAKER_00
Rather than being hints, so there s other advantages too.
6
SPEAKER_00
And then you can do that incrementally as you go.
6
SPEAKER_01
So one different perspective on this would be, why Mojo instead of making CPython faster or redesigning CPython?
6
SPEAKER_00
Yeah, I mean, you could argue Mojo is redesigning CPython, but why not make CPython faster and better and other things like that?
6
SPEAKER_00
There s lots of people working on that.
6
SPEAKER_00
So actually, there s a team at Microsoft that is really improving.
6
SPEAKER_00
I think CPython 3.11 came out in October or something like that, and it was 15% faster, 20% faster across the board.
6
SPEAKER_00
which is pretty huge given how mature Python is and things like this.
6
SPEAKER_00
And so that s awesome, I love it.
6
SPEAKER_00
Doesn t run on GPU.
6
SPEAKER_00
It doesn t do AI stuff, like it doesn t do vectors, doesn t do things.
6
SPEAKER_00
20% is good, 35,000 times is better.
6
SPEAKER_00
I m a huge fan of that work, by the way, and it composes well with what we re doing, and so it s not like we re fighting or anything like that.
6
SPEAKER_00
It s actually just goodness for the world, but it s just a different path.
6
SPEAKER_00
And again, we re not working forwards from making Python a little bit better.
6
SPEAKER_00
We re working backwards from what is the limit of physics.
6
SPEAKER_01
What s the process of porting Python code to Mojo?
6
SPEAKER_01
What s involved in that process?
6
SPEAKER_00
Is there tooling for that?
6
SPEAKER_00
Not yet.
6
SPEAKER_00
So we re missing some basic features right now.
6
SPEAKER_00
And so we re continuing to drop out new features like on a weekly basis.
6
SPEAKER_00
But you know, at the fullness of time, give us a year and a half, maybe two years.
6
SPEAKER_00
Is it an automatable process?
6
SPEAKER_00
So when we re ready, it will be very automatable.
6
SPEAKER_01
Yes.
6
SPEAKER_01
Is it possible to automate
6
SPEAKER_01
in the general case, the Python to Mojo conversion.
6
SPEAKER_01
You re saying it s possible.
6
SPEAKER_00
Well, so, and this is why, I mean, among other reasons why we use tabs.
6
SPEAKER_01
Yes.
6
SPEAKER_00
Right, so, first of all, by being a superset, it s like C versus C++.
6
SPEAKER_00
Can you move C code to C++?
6
SPEAKER_00
Yes, yeah, right?
6
SPEAKER_00
And you move.
6
SPEAKER_00
You can move C code to C++ and then you can adopt classes.
6
SPEAKER_00
You can adopt templates.
6
SPEAKER_00
You can adopt other references or whatever C++ features you want after you move seed C code to C++ like you can t use templates and see.
6
SPEAKER_00
Right and so if you leave it a C fine you can t use the cool features But it still works right and C and C++ could work together, and so that s the analogy right now Here right you you you There s not a Python is bad and the mojo is good
6
SPEAKER_00
Right, Mojo just gives you superpowers, right?
6
SPEAKER_00
And so if you wanna stay with Python, that s cool.
6
SPEAKER_00
But the tooling should be actually very beautiful and simple because we re doing the hard work of defining a superset.
6
SPEAKER_01
Right, so you re, right, so there s several things to say there, but also the conversion tooling.
6
SPEAKER_00
should probably give you hints as to how you can improve the code.
6
SPEAKER_00
Exactly.
6
SPEAKER_00
Once you re in the new world, then you can build all kinds of cool tools to say like, hey, should you adopt this feature?
6
SPEAKER_00
We haven t built those tools yet, but I fully expect those tools will exist and then you can, quote unquote, modernize your code or however you want to look at it.
6
SPEAKER_00
One of the things that I think is really interesting about Mojo is that,
6
SPEAKER_00
There have been a lot of projects to improve Python over the years.
6
SPEAKER_00
Everything from, you know, getting Python to run on the Java virtual machine, PyPy, which is a JIT compiler.
6
SPEAKER_00
There s tons of these projects out there that have been working on improving Python in various ways.
6
SPEAKER_00
they fall into one or two camps.
6
SPEAKER_00
So PyPy is a great example of a camp that is trying to be compatible with Python.
6
SPEAKER_00
Even there, not really.
6
SPEAKER_00
It doesn t work with all the C packages and stuff like that, but they re trying to be compatible with Python.
6
SPEAKER_00
There s also another category of these things where they re saying, well, Python is too complicated, and I m gonna cheat on the edges, and integers in Python can be an arbitrary size integer.
6
SPEAKER_00
if you care about it fitting in a, going fast in a register in a computer, that s really annoying, right?
6
SPEAKER_00
And so you can, you can choose to pass on that, right?
6
SPEAKER_00
You can say, well, people don t really use big integers that often, therefore I m gonna just not do it and it ll be fine.
6
SPEAKER_00
Not a Python superset.
6
SPEAKER_00
Or you can do the hard thing and say, okay, this is Python.
6
SPEAKER_00
And you can t be a superset of Python without
6
SPEAKER_00
being a superset of Python, and that s a really hard technical problem, but it s, in my opinion, worth it, right?
6
SPEAKER_00
And it s worth it because it s not about any one package, it s about this ecosystem, it s about what Python means for the world, and it also means we don t wanna repeat the Python 2 to Python 3 transition, like we want people to be able to adopt this stuff quickly, and so by doing that work, we can help lift people.
6
SPEAKER_01
Yeah, the challenge, it s a really interesting technical philosophical challenge
6
SPEAKER_01
really making a language a superset of another language.
6
SPEAKER_01
It s breaking my brain a little bit.
6
SPEAKER_00
Well, it paints you in the corners.
6
SPEAKER_00
So, again, I m very happy with Python.
6
SPEAKER_00
All joking aside, I think that the annotation thing is not the actual important part of the problem.
6
SPEAKER_00
Yes.
6
SPEAKER_00
Right?
6
SPEAKER_00
But the fact that Python has amazing dynamic metaprogramming features, and they translate to beautiful static metaprogramming features, I think is profound.
6
SPEAKER_00
I think that s huge, right?
6
SPEAKER_00
And so Python, I ve talked with Guido about this, it s like, it was not designed to do what we re doing.
6
SPEAKER_00
That was not the reason they built it this way, but because they really cared and they were very thoughtful about how they designed the language, it scales very elegantly in the space.
6
SPEAKER_00
But if you look at other languages, for example, C and C++, right?
6
SPEAKER_00
If you re building a superset, you get stuck with the design decisions of the subset.
6
SPEAKER_00
right?
6
SPEAKER_00
And so, you know, C++ is way more complicated because of C in the legacy than it would have been if they would have theoretically designed a from scratch thing.
6
SPEAKER_00
And there s lots of people right now that are trying to make C++ better and re-syntax C++.
6
SPEAKER_00
It s gonna be great, we ll just change all the syntax.
6
SPEAKER_00
But if you do that, now suddenly you have zero packages.
6
SPEAKER_00
You don t have compatibility.
6
SPEAKER_01
So what are the, if you could just linger on that, what are the biggest challenges of keeping that superset status?
6
SPEAKER_01
What are the things you re struggling with?
6
SPEAKER_01
Is it all boiled down to having a big integer?
6
SPEAKER_00
No, I mean, what are the other things like?
6
SPEAKER_00
Usually, it s the long tail weird things.
6
SPEAKER_00
So, let me give you a war story.
6
SPEAKER_00
Okay.
6
SPEAKER_00
So, war story in the space is, you go way back in time, project I worked on is called Clang.
6
SPEAKER_00
Clang, what it is, is a C++ parser, right?
6
SPEAKER_00
And when I started working on Clang,
6
SPEAKER_00
It must ve been like 2006 or something, or 2007, 2006 when I first started working on it, right?
6
SPEAKER_00
It s funny how time flies.
6
SPEAKER_00
I started that project, and I m like, okay, well, I wanna build a C parser, C++ parser for LLVM.
6
SPEAKER_00
It s gonna be, GCC is yucky.
6
SPEAKER_00
This is me in earlier times.
6
SPEAKER_00
It s yucky, it s unprincipled, it has all these weird features, all these bugs.
6
SPEAKER_00
It s yucky, so I m going to build a standard compliant C and C++ parser.
6
SPEAKER_00
It s going to be beautiful.
6
SPEAKER_00
It ll be amazing, well-engineered, all the cool things an engineer wants to do.
6
SPEAKER_00
So I start implementing and building it out, and building it out, and building it out, and then I got to include standard io.h.
6
SPEAKER_00
All of the headers in the world use all the GCC stuff.
6
SPEAKER_00
And so again, come back away from theory back to reality, right?
6
SPEAKER_00
I was at a fork in the road.
6
SPEAKER_00
I could have built an amazingly beautiful academic thing that nobody would ever use.
6
SPEAKER_00
Or I could say, well, it s yucky in various ways.
6
SPEAKER_00
All these design mistakes, accidents of history, the legacy.
6
SPEAKER_00
At that point, GCC was like over 20 years old.
6
SPEAKER_00
Which, by the way, now LLVM s over 20 years old.
6
SPEAKER_00
So it s funny how time catches up to you, right?
6
SPEAKER_00
And so you say, OK, well,
6
SPEAKER_00
What is easier, right?
6
SPEAKER_00
I mean, as an engineer, it s actually much easier for me to go implement long tail compatibility, weird features, even if they re distasteful, and just do the hard work and figure it out, reverse engineer, understand what it is, write a bunch of test cases, try to understand behavior.
6
SPEAKER_00
It s way easier to do all that work as an engineer than it is to go talk to all C programmers and argue with them and try to get them to rewrite their code.
6
SPEAKER_00
Because that breaks a lot more things.
6
SPEAKER_00
Yeah, and you have realities like nobody actually even understands how the code works because it was written by the person who quit 10 years ago, right?
6
SPEAKER_00
And so this software is kind of frustrating that way, but that s how the world works.
6
SPEAKER_01
Yeah, unfortunately it can never be this perfect, beautiful thing.
6
SPEAKER_00
Well, there are occasions in which you get a build, like you invent a new data structure or something like that.
6
SPEAKER_00
Or there s this beautiful algorithm that just makes you super happy.
6
SPEAKER_00
And I love that moment.
6
SPEAKER_00
But when you re working with people, and you re working with code and dusty deck code bases and things like this, it s not about what s theoretically beautiful.
6
SPEAKER_00
It s about what s practical, what s real, what people actually use.
6
SPEAKER_00
And I don t meet a lot of people that say, I want to rewrite all my code.
6
SPEAKER_00
just for the sake of it.
6
SPEAKER_01
By the way, there could be interesting possibilities, and we ll probably talk about it, where AI can help rewrite some code.
6
SPEAKER_01
That might be a farther out future, but it s a really interesting one, how that could create more.
6
SPEAKER_01
Be a tool in the battle against this monster of complexity that you mentioned.
6
SPEAKER_01
You mentioned Guido, the benevolent dictator for life of Python.
6
SPEAKER_01
What does he think about Mojo?
6
SPEAKER_01
Have you talked to him much about it?
6
SPEAKER_00
I have talked with him about it.
6
SPEAKER_00
He found it very interesting.
6
SPEAKER_00
We actually talked with Guido before it launched, and so he was aware of it before it went public.
6
SPEAKER_00
I have a ton of respect for Guido for a bunch of different reasons.
6
SPEAKER_00
You talk about Walrus Operator, and Guido s pretty amazing in terms of
6
SPEAKER_00
steering such a huge and diverse community and, and, and like driving it forward.
6
SPEAKER_00
And I think Python is what it is thanks to him.
6
SPEAKER_00
Right.
6
SPEAKER_00
And so to me, it was really important starting to work on Mojo to get his feedback and get his input and get his eyes on this.
6
SPEAKER_00
Right.
6
SPEAKER_00
Now, a lot of what Guido was and is, I think, concerned about is, how do we not fragment the community?
6
SPEAKER_00
We don t want a Python 2 to Python 3 thing.
6
SPEAKER_00
That was really painful for everybody involved.
6
SPEAKER_00
And so we spent quite a bit of time talking about that and some of the tricks I learned from Swift, for example.
6
SPEAKER_00
So in the migration from Swift, we managed to not just convert
6
SPEAKER_00
Objective-C into a slightly prettier Objective-C, which we did.
6
SPEAKER_00
We then converted, not entirely, but almost an entire community to a completely different language.
6
SPEAKER_00
And so there s a bunch of tricks that you learn along the way that are directly relevant to what we do.
6
SPEAKER_00
And so this is where, for example, you leverage CPython while bringing up the new thing.
6
SPEAKER_00
That approach is, I think, proven and comes from experience.
6
SPEAKER_00
And so Guido was very interested in like, okay, cool.
6
SPEAKER_00
I think that Python is really his legacy.
6
SPEAKER_00
It s his baby.
6
SPEAKER_00
I have tons of respect for that.
6
SPEAKER_00
Incidentally, I see Mojo as a member of the Python family.
6
SPEAKER_00
I m not trying to take Python away from Guido and from the Python community.
6
SPEAKER_00
And so to me, it s really important that we re a good member of that community.
6
SPEAKER_00
And so I think that, again, you would have to ask Guido this, but I think that he was very interested in this notion of like, cool, Python gets beaten up for being slow.
6
SPEAKER_00
maybe there s a path out of that, right?
6
SPEAKER_00
And that, you know, if the future is Python, right?
6
SPEAKER_00
I mean, look at the far outside case on this, right?
6
SPEAKER_00
And I m not saying this is Guido s perspective, but you know, there s this path of saying like, okay, well, suddenly Python can suddenly go all the places it s never been able to go before.
6
SPEAKER_00
Right, and that means that Python can go even further and can have even more impact on the world.
6
SPEAKER_01
So in some sense, Mojo could be seen as Python 4.0.
6
SPEAKER_00
I would not say that.
6
SPEAKER_00
I think that would drive a lot of people really crazy.
6
SPEAKER_01
Because of the PTSD of the 3.0 2.0.
6
SPEAKER_00
I m willing to annoy people about Emacs versus Vim, or about Tabs versus Spaces.
6
SPEAKER_01
Not that one.
6
SPEAKER_00
I don t know, that might be a little bit far even for me.
6
SPEAKER_00
My skin may not be that thick.
6
SPEAKER_01
But the point is, the step to being a superset and allowing all of these capabilities, I think, is the evolution of a language.
6
SPEAKER_01
It feels like an evolution of a language.
6
SPEAKER_01
So he s interested by the ideas that you re playing with, but also concerned about the fragmentation.
6
SPEAKER_01
So how, what are the ideas you ve learned?
6
SPEAKER_01
What are you thinking about?
6
SPEAKER_01
How do we avoid fragmenting the community?
6
SPEAKER_01
Where the Pythonistas and the,
6
SPEAKER_01
I don t know what to call the mojo people.
6
SPEAKER_00
Magicians.
6
SPEAKER_01
Magicians, I like it.
6
SPEAKER_01
Can coexist happily and share code and basically just have these big code bases that are using CPython and more and more moving towards mojo.
6
SPEAKER_00
Well, so again, these are lessons I learned from Swift and here we face very similar problems, right?
6
SPEAKER_00
In Swift you have Objective-C, super dynamic,
6
SPEAKER_00
They re very different syntax, right?
6
SPEAKER_00
But you re talking to people who have large-scale code bases.
6
SPEAKER_00
I mean, Apple s got the biggest, largest-scale code base of Objective-C code, right?
6
SPEAKER_00
And so, you know, none of the companies, none of the iOS developers, none of the other developers want to rewrite everything all at once, and so you want to be able to adopt things piece at a time.
6
SPEAKER_00
And so a thing that I found that worked very well in the Swift community was saying, okay, cool.
6
SPEAKER_00
And this was when Swift was very young.
6
SPEAKER_00
And she d say, okay, you have a million line of code Objective-C app, don t rewrite it all.
6
SPEAKER_00
But when you implement a new feature, go implement that new class using Swift.
6
SPEAKER_00
And so now this turns out is a very wonderful thing for an app developer.
6
SPEAKER_00
But it s a huge challenge for the compiler team and the systems people that are implementing this, right?
6
SPEAKER_00
And this comes back to what is this trade-off between doing the hard thing that enables scale versus doing the theoretically pure and ideal thing, right?
6
SPEAKER_00
And so Swift adopted and built a lot of different machinery to deeply integrate with the Objective-C runtime.
6
SPEAKER_00
And we re doing the same thing with Python.
6
SPEAKER_00
Now, what happened in the case of Swift is that Swift as a language got more and more and more mature over time.
6
SPEAKER_00
Incidentally, Mojo is a much simpler language than Swift in many ways, and so I think that Mojo will develop way faster than Swift for a variety of reasons.
6
SPEAKER_00
But as the language gets more mature, in parallel with that, you have new people starting new projects.
6
SPEAKER_00
Right, and so when the language is mature and somebody s starting a new project, that s when they say, okay, cool, I m not dealing with a million lines of code, I ll just start and use the new thing for my whole stack.
6
SPEAKER_00
Now the problem is, again, you come back to where communities and where people that work together, you build a new subsystem or a new feature or a new thing in Swift, or you build a new thing in Mojo, then you want it to end up being used on the other side.
6
SPEAKER_00
Right, and so then you need to work on integration back the other way.
6
SPEAKER_00
And so it s not just Mojo talking to Python, it s also Python talking to Mojo.
6
SPEAKER_00
And so what I would love to see, and I don t wanna see this next month, but what I wanna see over the course of time is I would love to see people that are building these packages, like NumPy or TensorFlow, these packages that are half Python, half C++.
6
SPEAKER_00
And if you say, okay, cool, I want to get out of this Python C++ world into a unified world, and so I can move to Mojo, but I can t give up all my Python clients.
6
SPEAKER_00
These libraries get used by everybody, and they re not all gonna switch all at once, and maybe never, right?
6
SPEAKER_00
Well, so the way we should do that is we should vend Python interfaces to the Mojo types.
6
SPEAKER_00
And that s what we did in Swift, and it worked great.
6
SPEAKER_00
I mean, it was a huge implementation challenge for the compiler people, right?
6
SPEAKER_00
But there s only a dozen of those compiler people, and there are millions of users.
6
SPEAKER_00
And so it s a very expensive, capital-intensive, skill-set-intensive problem.
6
SPEAKER_00
once you solve that problem, it really helps adoption, it really helps the community progressively adopt technologies.
6
SPEAKER_00
And so I think that this approach will work quite well with the Python and the Mojo world.
6
SPEAKER_01
So for a package ported to Mojo and then create a Python interface.
6
SPEAKER_00
Yep.
6
SPEAKER_01
So how do just to linger on these
7
SPEAKER_00
packages, NumPy, PyTorch, and TensorFlow.
7
SPEAKER_00
How do they play nicely together?
7
SPEAKER_00
So is Mojo supposed to be, let s talk about the machine learning ones.
7
SPEAKER_00
Is Mojo kind of vision to replace PyTorch and TensorFlow, to incorporate it?
7
SPEAKER_00
What s the relationship in this?
7
SPEAKER_01
All right, so take a step back.
7
SPEAKER_01
So I wear many hats.
7
SPEAKER_01
So you re angling in on the Mojo side.
7
SPEAKER_01
Mojo is a programming language, and so it can help solve the C, C++, Python feud that s happening.
7
SPEAKER_01
The fire emoji got me.
7
SPEAKER_01
I m sorry.
7
SPEAKER_01
We should be talking modular.
7
SPEAKER_01
Yes.
7
SPEAKER_01
Yes.
7
SPEAKER_01
OK.
7
SPEAKER_01
So the fire emoji is amazing.
7
SPEAKER_01
I love it.
7
SPEAKER_01
It s a big deal.
7
SPEAKER_01
The other side of this is the fire emoji is in service of solving some big AI problems, right?
7
SPEAKER_01
And so the big AI problems are, again, this fragmentation, this hardware nightmare, this explosion of new potential, but it s not getting felt by the industry, right?
7
SPEAKER_01
And so when you look at how does the modular engine help TensorFlow and PyTorch, right?
7
SPEAKER_01
It s not replacing them, right?
7
SPEAKER_01
In fact, when I talk to people, again, they don t like to rewrite all their code,
7
SPEAKER_01
You have people that are using a bunch of PyTorch, a bunch of TensorFlow.
7
SPEAKER_01
They have models that they ve been building over the course of many years.
7
SPEAKER_01
And when I talk to them, there s a few exceptions, but generally, they don t want to rewrite all their code.
7
SPEAKER_01
And so what we re doing is we re saying, OK, well, you don t have to rewrite all your code.
7
SPEAKER_01
What happens is the modular engine goes in there and goes underneath TensorFlow and PyTorch.
7
SPEAKER_01
It s fully compatible.
7
SPEAKER_01
And it just provides better performance, better predictability, better tooling,
7
SPEAKER_01
it s a better experience that helps lift TensorFlow and PyTorch and make them even better.
7
SPEAKER_01
I love Python, I love TensorFlow, I love PyTorch, right?
7
SPEAKER_01
This is about making the world better because we need AI to go further.
7
SPEAKER_00
But if I have a process that trains a model and I have a process that performs inference on that model and I have the model itself, what should I do with that in the long arc of history?
7
SPEAKER_00
in terms of if I use PyTorch to train it, should I rewrite stuff in Mojo?
7
SPEAKER_00
Would that, if I care about performance?
7
SPEAKER_01
Oh, so I mean, again, it depends.
7
SPEAKER_01
So if you care about performance, then writing it in Mojo is gonna be way better than writing it in Python.
7
SPEAKER_01
But if you look at LLM companies, for example, so you look at OpenAI, Rumored, and you look at many of the other folks that are working on many of these LLMs and other innovative machine learning models, on the one hand, they re innovating in the data collection and the model, billions of parameters in the model architecture and the,
7
SPEAKER_01
RLE, HF, and all the cool things that people are talking about.
7
SPEAKER_01
But on the other hand, they re spending a lot of time writing cuda kernels.
7
SPEAKER_01
And so you say, wait a second, how much faster could all this progress go if they were not having to handwrite all these?
7
SPEAKER_01
cuda kernels right and so there are a few technologies that are out there and people have been working on this problem for a while and um and they re trying to solve subsets of the problem again kind of fragmenting the space and so what mojo provides for these kinds of companies is the ability to say cool i can have a unifying theory
7
SPEAKER_01
Right, and again, the better together, the unifying theory, the two-world problem, or the three-world problem, or the n-world problem, this is the thing that is slowing people down.
7
SPEAKER_01
And so as we help solve this problem, I think it ll be very helpful for making this whole cycle go faster.
7
SPEAKER_00
So obviously we ve talked about the transition from Objective-C to Swift.
7
SPEAKER_00
You ve designed this programming language, and you ve also talked quite a bit about the use of Swift for machine learning context.
7
SPEAKER_00
Why have you decided to move away from maybe an intense focus on Swift for the machine learning context versus sort of deciding a new programming language that happens to be a superset?
7
SPEAKER_01
You re saying this is an irrational set of life choices I make?
7
SPEAKER_00
Did you go to the desert and did you meditate on it?
7
SPEAKER_00
Okay, all right.
7
SPEAKER_00
No, it was bold and needed, and I think, I mean, it s just bold, and sometimes to take those leaps is a difficult leap to take.
7
SPEAKER_01
Yeah, well, so, okay, I mean, I think there s a couple of different things.
7
SPEAKER_01
So, actually, I left Apple back in 2017, like January 2017, so it s been a number of years that I left Apple, and the reason I left Apple was to do AI.
7
SPEAKER_01
OK, and again, I won t comment on Apple and AI, but at the time, I wanted to get into and understand and understand the technology, understand the applications, the workloads.
7
SPEAKER_01
And so I was like, OK, I m going to go dive deep into Applied and AI and then the technology underneath it.
7
SPEAKER_01
I found myself at Google.
7
SPEAKER_01
And that was like when TPUs were waking up.
7
SPEAKER_01
Exactly.
7
SPEAKER_01
And so I found myself at Google.
7
SPEAKER_01
And Jeff Dean, who s a rock star, as you know, right?
7
SPEAKER_01
And in 2017, TensorFlow was really taking off and doing incredible things.
7
SPEAKER_01
And I was attracted to Google to help them with the TPUs.
7
SPEAKER_01
And TPUs are an innovative hardware accelerator platform, have now, I mean, I think, proven massive scale and done incredible things.
7
SPEAKER_01
And so one of the things that this led into is a bunch of different projects, which I ll skip over, right?
7
SPEAKER_01
One of which was this Swift for TensorFlow project, right?
7
SPEAKER_01
And so that project was a research project.
7
SPEAKER_01
And so the idea of that is say, okay, well, let s look at innovative new programming models where we can get a fast programming language, we can get automatic differentiation into language, let s push the boundaries of these things in a research setting.
7
SPEAKER_01
Now, that project I think lasted two, three years.
7
SPEAKER_01
There s some really cool outcomes of that.
7
SPEAKER_01
So one of the things that s really interesting is I published a talk at an LLVM conference in 2018, again, that seems like so long ago, about graph program abstraction, which is basically the thing that s in PyTorch 2.
7
SPEAKER_01
And so PyTorch 2, with all this Dynamo Rio thing, it s all about this graph program abstraction thing from Python bytecodes.
7
SPEAKER_01
And so a lot of the research that was done ended up pursuing and going out through the industry and influencing things.
7
SPEAKER_01
And I think it s super exciting and awesome to see that.
7
SPEAKER_01
But the Swift for TensorFlow project itself did not work out super well.
7
SPEAKER_01
And so there s a couple of different problems with that, one of which is that you may have noticed Swift is not Python.
7
SPEAKER_01
There s a few people that write Python code.
7
SPEAKER_01
And so it turns out that all of ML is pretty happy with Python.
7
SPEAKER_00
It s actually a problem that other programming languages have as well, that they re not Python.
7
SPEAKER_00
We ll probably maybe briefly talk about Julia, who s a very interesting, beautiful programming language, but it s not Python.
7
SPEAKER_01
Well, and so if and so like if you re saying I m gonna solve a machine learning problem where all the programmers are Python programmers Yeah, and you say the first thing you have to do is switch to a different language
7
SPEAKER_01
Well, your new thing may be good or bad or whatever, but if it s a new thing, the adoption barrier is massive.
7
SPEAKER_01
It s still possible.
7
SPEAKER_01
Still possible, yeah, absolutely.
7
SPEAKER_01
The world changes and evolves, and there s definitely room for new and good ideas, but it just makes it so much harder, right?
7
SPEAKER_01
And so, lesson learned, Swift is not Python, and people are not always in search of learning a new thing for the sake of learning a new thing, and if you wanna be compatible with all the world s code, turns out,
7
SPEAKER_01
meet the world where it is, right?
7
SPEAKER_01
Second thing is that, you know, a lesson learned is that Swift as a very fast and efficient language, kind of like Mojo, but a different take on it still, really worked well with eager mode.
7
SPEAKER_01
And so eager mode is something that PyTorch does, and it proved out really well.
7
SPEAKER_01
And it enables really expressive and dynamic and easy to debug programming.
7
SPEAKER_01
TensorFlow at the time was not set up for that, let s say.
7
SPEAKER_01
That was not.
7
SPEAKER_01
The timing is also important in this world.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
Yeah.
7
SPEAKER_01
And TensorFlow is a good thing, and it has many, many strengths.
7
SPEAKER_01
But you could say Swift for TensorFlow is a good idea except for the Swift and except for the TensorFlow part.
7
SPEAKER_01
So.
7
SPEAKER_01
Because it s not Python and TensorFlow because it s not.
7
SPEAKER_01
It wasn t set up for eager mode at the time.
7
SPEAKER_00
Yeah.
7
SPEAKER_00
It was 1.0.
7
SPEAKER_01
Exactly.
7
SPEAKER_01
And so one of the things about that is in the context of it being a research project, I m very happy with the fact that we built a lot of really cool technology.
7
SPEAKER_01
We learned a lot of things.
7
SPEAKER_01
I think the ideas went on to have influence in other systems like PyTorch.
7
SPEAKER_01
A few people use that I hear.
7
SPEAKER_01
Right.
7
SPEAKER_01
And so I think that s super cool.
7
SPEAKER_01
And for me personally, I learned so much from it.
7
SPEAKER_01
And I think a lot of the engineers that worked on it also learned a tremendous amount.
7
SPEAKER_01
And so I think that that s just really exciting to see.
7
SPEAKER_01
And I m sorry that the project didn t work out.
7
SPEAKER_01
I wish it did, of course.
7
SPEAKER_01
But it s a research project, and so you re there to learn from it.
7
SPEAKER_01
Well, it s interesting to think about
7
SPEAKER_00
the evolution of programming as we come up with these whole new set of algorithms in machine learning, in artificial intelligence, and what s going to win out.
7
SPEAKER_00
Because it could be a new programming language.
7
SPEAKER_00
It could be...
7
SPEAKER_00
I mean, I just mentioned Julia.
7
SPEAKER_00
I think there s a lot of ideas behind Julia that Mojo shares.
7
SPEAKER_00
What are your thoughts about Julia in general?
7
SPEAKER_01
So I will have to say that when we launched Mojo,
7
SPEAKER_01
One of the biggest things I didn t predict was the response from the Julia community.
7
SPEAKER_01
And so I was not, I mean, I ve, okay, let me take a step back.
7
SPEAKER_01
I ve known the Julia folks for a really long time.
7
SPEAKER_01
They re an adopter of LLVM a long time ago.
7
SPEAKER_01
They ve been pushing state-of-the-art in a bunch of different ways.
7
SPEAKER_01
Julia s a really cool system.
7
SPEAKER_01
I had always thought of Julia as being mostly a scientific computing focused environment, right?
7
SPEAKER_01
And I thought that was its focus.
7
SPEAKER_01
I neglected
7
SPEAKER_01
to understand that one of their missions is to help make Python work end-to-end.
7
SPEAKER_01
And so I think that was my error for not understanding that.
7
SPEAKER_01
And so I could have been maybe more sensitive to that.
7
SPEAKER_01
But there s major differences between what Mojo s doing and what Julia s doing.
7
SPEAKER_01
So as you say, Julia is not Python.
7
SPEAKER_01
And so one of the things that a lot of the Julia people came out and said is, OK, well, if we put a ton of more energy and a ton more money or engineering or whatever into Julia,
7
SPEAKER_01
maybe that would be better than starting mojo right well i mean maybe that s true but it still wouldn t make julia into python so if you ve worked backwards from the goal of let s build something for python programmers without requiring them to relearn syntax then julia just isn t
7
SPEAKER_01
I mean, that s a different thing, right?
7
SPEAKER_01
And so if you anchor on, I love Julia, and I want Julia to go further, then you can look at it from a different lens.
7
SPEAKER_01
But the lens we were coming at was, hey, everybody is using Python.
7
SPEAKER_01
Python isn t, syntax isn t broken.
7
SPEAKER_01
Let s take what s great about Python and make it even better.
7
SPEAKER_01
And so it was just a different starting point.
7
SPEAKER_01
So I think Julia s a great language.
7
SPEAKER_01
The community s a lovely community.
7
SPEAKER_01
They re doing really cool stuff, but it s just a slightly different angle.
7
SPEAKER_00
But it does seem that Python is quite sticky.
7
SPEAKER_00
Is there some philosophical almost thing you could say about why Python, by many measures, seems to be the most popular programming language in the world?
7
SPEAKER_01
Well, I can tell you things I love about it.
7
SPEAKER_01
Maybe that s one way to answer the question, right?
7
SPEAKER_01
So, huge package ecosystem.
7
SPEAKER_01
super lightweight and easy to integrate.
7
SPEAKER_01
It has very low startup time.
7
SPEAKER_00
So what s startup time?
7
SPEAKER_01
You mean like learning curve or what?
7
SPEAKER_01
Yeah, so if you look at certain other languages, you say like Go, and it just takes like Java, for example.
7
SPEAKER_01
It takes a long time to JIT compile all the things, and then the VM starts up and the garbage collector kicks in, and then it revs its engines, and then it can plow through a lot of internet stuff or whatever, right?
7
SPEAKER_01
Python is like scripting.
7
SPEAKER_01
It just goes, right?
7
SPEAKER_01
Python has very little compile time.
7
SPEAKER_01
So you re not sitting there waiting.
7
SPEAKER_01
Python integrates into notebooks in a very elegant way that makes exploration super interactive and it s awesome, right?
7
SPEAKER_01
Python is also, it s like almost the glue of computing.
7
SPEAKER_01
Because it has such a simple object representation, a lot of things plug into it.
7
SPEAKER_01
That dynamic metaprogramming thing we were talking about also enables really expressive and beautiful APIs.
7
SPEAKER_01
So there s lots of reasons that you can look at technical things that Python has done and say like, okay, wow, this is actually a pretty amazing thing.
7
SPEAKER_01
And any one of those you can neglect.
7
SPEAKER_01
People will all just talk about indentation and ignore like the fundamental things.
7
SPEAKER_01
But then you also look at the community side, right?
7
SPEAKER_01
So Python owns machine learning.
7
SPEAKER_00
Machine learning s pretty big.
7
SPEAKER_00
Yeah, and it s growing.
7
SPEAKER_01
And it s growing, right?
7
SPEAKER_01
And it s growing in importance, right?
7
SPEAKER_00
And there s a reputation and prestige to machine learning to where, like, if you re a new programmer, you re thinking about, like, which programming language do I use?
7
SPEAKER_00
Well, I should probably care about machine learning.
7
SPEAKER_00
Therefore, let me try Python.
7
SPEAKER_00
And it kind of builds and builds and builds.
7
SPEAKER_01
And even go back before that.
7
SPEAKER_01
Like, my kids learn Python.
7
SPEAKER_01
Not because I m telling them to learn Python, but because... Were they rebelling against you or what?
7
SPEAKER_01
Well, they also learn Scratch and things like this, too.
7
SPEAKER_01
But it s because Python is taught everywhere.
7
SPEAKER_01
Because it s easy to learn.
7
SPEAKER_01
And because it s pervasive.
7
SPEAKER_00
Back in my day, we learned Java and C++.
7
SPEAKER_00
Yeah, well, uphill both directions.
7
SPEAKER_00
But yes, I guess Python is the main language of teaching software engineering in schools now.
7
SPEAKER_01
Yeah, well, if you look at if you look at this, there s these growth cycles, right?
7
SPEAKER_01
If you look at what causes things to become popular, and then gain in popularity, there s reinforcing feedback loops and things like this.
7
SPEAKER_01
And I think Python has done, again, the whole community has done a really good job of building those growth loops and help propel the ecosystem.
7
SPEAKER_01
And I think that again, you look at what you can get done with just a few lines of code.
7
SPEAKER_01
It s amazing.
7
SPEAKER_00
So this kind of self-building loop is interesting to understand because when you look at Mojo, what it stands for, some of the features, it seems sort of clear that this is a good direction for programming languages to evolve in the machine learning community.
7
SPEAKER_00
But it s still not obvious that it will because of this,
7
SPEAKER_00
whatever the engine of popularity, of virality.
7
SPEAKER_00
Is there something you could speak to, like how do you get people to switch?
7
SPEAKER_01
Yeah, well, I mean, I think that the viral growth loop is to switch people to Unicode.
7
SPEAKER_01
I think the Unicode file extensions are what I m betting on.
7
SPEAKER_01
I think that s gonna be the thing.
7
SPEAKER_00
Tell the kids that you can use the fire emoji and they d be like, what?
7
SPEAKER_01
Exactly.
7
SPEAKER_01
Well, in all seriousness, I think there s really, I ll give you two opposite answers.
7
SPEAKER_01
One is, I hope if it s useful, if it solves problems, and if people care about those problems being solved, they ll adopt the tech.
7
SPEAKER_01
Right, that s kind of the simple answer.
7
SPEAKER_01
And when you re looking to get tech adopted, the question is, is it solving an important problem people need solved?
7
SPEAKER_01
And is the adoption cost low enough that they re willing to make the switch and cut over and do the pain up front so that they can actually do it, right?
7
SPEAKER_01
And so hopefully Mojo will be that for a bunch of people.
7
SPEAKER_01
you know, people building these hybrid packages are suffering.
7
SPEAKER_01
It s really painful, and so I think that we have a good shot of helping people.
7
SPEAKER_01
But the other side is like, it s okay if people don t use Mojo.
7
SPEAKER_01
Like, it s not my job to say like, everybody should do this.
7
SPEAKER_01
Like, I m not saying Python is bad.
7
SPEAKER_01
Like, I hope Python, CPython, like all these implementations, because Python ecosystem s not just CPython, it s also a bunch of different implementations with different trade-offs.
7
SPEAKER_01
And this ecosystem is really powerful and exciting, as are other programming languages.
7
SPEAKER_01
It s not like TypeScript or something is gonna go away, right?
7
SPEAKER_01
And so there s not a winner-take-all thing.
7
SPEAKER_01
And so I hope that Mojo s exciting and useful to people, but if it s not, that s also fine.
7
SPEAKER_00
But I also wonder what the use case for why you should try Mojo would be, so practically speaking.
7
SPEAKER_00
Yeah.
7
SPEAKER_00
It seems like, so there s entertainment, there s a dopamine hit of saying, holy shit, this is 10 times faster.
7
SPEAKER_00
This little piece of code is 10 times faster in Mojo.
7
SPEAKER_00
Out of the box before you get to 35,000.
7
SPEAKER_01
Exactly.
7
SPEAKER_00
I mean, just even that, I mean, that s the dopamine hit that every programmer sort of dreams of is the optimization.
7
SPEAKER_00
It s also the drug,
7
SPEAKER_00
they can pull you in and have you waste way too much of your life optimizing and over-optimizing, right?
7
SPEAKER_00
But so what do you see that would be like comedy?
7
SPEAKER_00
This is very hard to predict, of course, but if you look 10 years from now, Mojo s super successful, what do you think would be the thing
7
SPEAKER_00
where people try it and then use it regularly and it kind of grows and grows and grows and grows.
7
SPEAKER_01
Well, so you talk about dopamine hit.
7
SPEAKER_01
And so, again, humans are not one thing.
7
SPEAKER_01
And some people love rewriting their code and learning new things and throwing themselves in the deep end and trying out new things.
7
SPEAKER_01
In my experience, most people don t.
7
SPEAKER_01
Like, they re too busy, they have other things going on.
7
SPEAKER_01
By number, most people
8
SPEAKER_01
don t like this, I want to rewrite all my code.
8
SPEAKER_01
But even those people, the too busy people, the people that don t actually care about the language, that just care about getting stuff done, those people do like learning new things.
8
SPEAKER_01
Right?
8
SPEAKER_01
And so you talk about the dopamine rush of 10x faster.
8
SPEAKER_01
Wow, that s cool.
8
SPEAKER_01
I want to do that again.
8
SPEAKER_01
Well, it s also like, here s here s the thing I ve heard about in a different domain.
8
SPEAKER_01
And I don t have to write all my code, I can learn a new trick.
8
SPEAKER_01
Right?
8
SPEAKER_01
Well, that s called growth.
8
SPEAKER_01
And so and so one thing that I think is cool about Mojo, and again, this will take a little bit of time for
8
SPEAKER_01
for example the blog posts and the books and like all that kind of stuff develop and the language needs to get further along but what we re doing you talk about types like you can say look you can start with the world you already know and you can progressively learn new things and adopt them where it makes sense if you never do that
8
SPEAKER_01
that s cool, you re not a bad person.
8
SPEAKER_01
If you get really excited about it and wanna go all the way in the deep end and wanna rewrite everything and like whatever, that s cool, right?
8
SPEAKER_01
But I think the middle path is actually the more likely one where it s, you know, you come out with a new idea and you discover, wow, that makes my code way simpler, way more beautiful, way faster, way whatever.
8
SPEAKER_01
And I think that s what people like.
8
SPEAKER_01
Now, if you fast forward and you said like 10 years out, right?
8
SPEAKER_01
I can give you a very different answer on that, which is, I mean, if you go back and look at what computers looked like 20 years ago.
8
SPEAKER_01
every 18 months they got faster for free.
8
SPEAKER_01
Right, 2x faster every 18 months, it was like clockwork, it was free, right?
8
SPEAKER_01
You go back 10 years ago and we entered in this world where suddenly we had multi-core CPUs and we had GPUs.
8
SPEAKER_01
And if you squint and turn your head, what a GPU is, it s just a many core, very simple CPU thing, kind of, right?
8
SPEAKER_01
And 10 years ago it was CPUs and GPUs and graphics.
8
SPEAKER_01
Today, we have CPUs, GPUs, graphics, and AI.
8
SPEAKER_01
Because it s so important, because the compute is so demanding, because of the smart cameras and the watches and all the different places that AI needs to work in our lives, it s caused this explosion of hardware.
8
SPEAKER_01
And so part of my thesis, part of my belief of where computing goes, if you look out 10 years from now, is it s not gonna get simpler.
8
SPEAKER_01
Physics isn t going back to where we came from, it s only gonna get weirder from here on out, right?
8
SPEAKER_01
And so to me, the exciting part about what we re building is it s about building that universal platform, which the world can continue to get weird, because again, I don t think it s avoidable, it s physics.
8
SPEAKER_01
But we can help lift people, scale, do things with it, and they don t have to rewrite their code every time a new device comes out.
8
SPEAKER_01
And I think that s pretty cool.
8
SPEAKER_01
And so if Mojo can help with that problem, then I think that it will be hopefully quite interesting and quite useful to a wide range of people.
8
SPEAKER_01
Because there s so much potential, and maybe analog computers will become a thing or something, right?
8
SPEAKER_01
and we need to be able to get into a mode where we can move this programming model forward, but do so in a way where we re lifting people and growing them instead of forcing them to rewrite all their code and exploding them.
8
SPEAKER_00
Do you think there ll be a few major libraries that go mojo first?
8
SPEAKER_01
Well, so, I mean, the modular engine is all Mojo.
8
SPEAKER_01
So, again, come back to, like, we re not building Mojo because it s fun, we re building Mojo because we had to to solve these accelerator problems.
8
SPEAKER_00
That s the origin story.
8
SPEAKER_00
But I mean ones that are currently in Python.
8
SPEAKER_01
Yeah, so I think that a number of these projects will.
8
SPEAKER_01
And so one of the things, again, this is just my best guess, like, each of the package maintainers also has
8
SPEAKER_01
I m sure plenty of other things going on.
8
SPEAKER_01
People really don t like rewriting code just for the sake of rewriting code.
8
SPEAKER_01
But sometimes people are excited about adopting a new idea.
8
SPEAKER_01
It turns out that while rewriting code is generally not people s first
8
SPEAKER_01
thing, turns out that redesigning something while you rewrite it and using a rewrite as an excuse to redesign can lead to the 2.0 of your thing that s way better than the 1.0, right?
8
SPEAKER_01
And so I have no idea, I can t predict that.
8
SPEAKER_01
But there s a lot of these places where, again, if you have a package that is half C and half Python, right, just solve the pain, make it easier to move things faster, make it easier to debug and evolve your
8
SPEAKER_01
Adopting Mojo kind of makes sense to start with, and then it gives you this opportunity to rethink these things.
8
SPEAKER_00
So the two big gains are that there s a performance gain, and then there s the portability to all kinds of different devices.
8
SPEAKER_01
And there s safety, right?
8
SPEAKER_01
So you talk about real types.
8
SPEAKER_01
I mean, not saying this is for everybody, but that s actually a pretty big thing, right?
8
SPEAKER_01
And so there s a bunch of different aspects of what Value Mojo provides.
8
SPEAKER_01
And so, I mean, it s funny for me.
8
SPEAKER_01
I ve been working on these kinds of technologies and tools for too many years now.
8
SPEAKER_01
But you look at Swift, right?
8
SPEAKER_01
And we talked about Swift for TensorFlow, but Swift as a programming language, right?
8
SPEAKER_01
Swift s now,
8
SPEAKER_01
13 years old from when I started it.
8
SPEAKER_01
So, because I started in 2010, if I remember.
8
SPEAKER_01
And so, that project, and I was involved with it for 12 years or something, right?
8
SPEAKER_01
That project has gone through its own really interesting story arc, right?
8
SPEAKER_01
And it s a mature, successful, used by millions of people system, right?
8
SPEAKER_01
certainly not dead yet, right?
8
SPEAKER_01
But also going through that story arc, I learned a tremendous amount about building languages, about building compilers, about working with community and things like this.
8
SPEAKER_01
And so that experience, like I m helping channel and bring directly into Mojo and other systems, same thing.
8
SPEAKER_01
Like apparently I like building and iterating and evolving things.
8
SPEAKER_01
And so you look at this LLVM thing I worked on 20 years ago and you look at MLIR, right?
8
SPEAKER_01
And so a lot of the lessons learned in LLVM
8
SPEAKER_01
got fed into MLIR, and I think that MLIR is a way better system than LLVM was, and you know, Swift is a really good system, and it s amazing, but I hope that Mojo will take the next step forward in terms of design.
8
SPEAKER_00
In terms of running Mojo, people can play with it, what s Mojo Playground?
8
SPEAKER_00
Yeah.
8
SPEAKER_00
And from the interface perspective, and from the hardware perspective, what s this incredible thing running on?
8
SPEAKER_01
Yeah, so right now, so here we are two weeks after launch.
8
SPEAKER_01
We decided that, okay, we have this incredible set of technology that we think might be good, but we have not given it to lots of people yet.
8
SPEAKER_01
And so we were very conservative and said, let s put it in a workbook so that if it crashes, we can do something about it.
8
SPEAKER_01
We can monitor and track that, right?
8
SPEAKER_01
And so, again, things are still super early, but we re having like one person a minute sign up.
8
SPEAKER_01
with over 70,000 people two weeks in is kind of crazy.
8
SPEAKER_00
So you can sign up to Moja Playground, and you can use it in the cloud in your browser.
8
SPEAKER_01
And so what that s running on is that s running on cloud VMs.
8
SPEAKER_01
And so you share a machine with a bunch of other people.
8
SPEAKER_01
But it turns out there s a bunch of them now because there s a lot of people.
8
SPEAKER_01
And so what you re doing is you re getting free compute, and you re getting to play with this thing in kind of a limited, controlled way so that we can make sure that it doesn t
8
SPEAKER_01
totally crash and be embarrassing right yeah so um now a lot of the feedback we ve gotten is people want to download it locally so we re working on that right now and so that s that s the goal to be able to download locally yeah that s what everybody expects and so we re working on that right now and so we just want to make sure that we do it right and
8
SPEAKER_01
I think this is one of the lessons I learned from Swift also, by the way, is that when we launched Swift, gosh, it feels like forever ago.
8
SPEAKER_01
It was 2014.
8
SPEAKER_01
I mean, it was super exciting.
8
SPEAKER_01
I and we, the team, had worked on Swift for a number of years in secrecy.
8
SPEAKER_01
Four years into this development, roughly, of working on this thing, at that point, about 250 people at Apple knew about it.
8
SPEAKER_01
Okay, so it was secret.
8
SPEAKER_01
Apple s good at secrecy, and it was a secret project.
8
SPEAKER_01
And so we launched this at WWDC, a bunch of hoopla and excitement, and said, developers, you re gonna be able to develop and submit apps to the App Store in three months.
8
SPEAKER_01
Well, several interesting things happened, right?
8
SPEAKER_01
So first of all, we learned that A, it had a lot of bugs.
8
SPEAKER_01
It was not actually production quality and it was extremely stressful in terms of like trying to get it working for a bunch of people.
8
SPEAKER_01
And so what happened was we went from zero to, you know, I don t know how many developers Apple had at the time, but a lot of developers overnight and they ran into a lot of bugs and it was really embarrassing and it was very stressful for everybody involved, right?
8
SPEAKER_01
It was also very exciting because everybody was excited about that.
8
SPEAKER_01
The other thing I learned is that when that happened, roughly every software engineer who did not know about the project at Apple, their head exploded when it was launched, because they didn t know it was coming.
8
SPEAKER_01
And so they re like, wait, what is this?
8
SPEAKER_01
I signed up to work for Apple because I love Objective-C.
8
SPEAKER_01
Why is there a new thing, right?
8
SPEAKER_01
And so
8
SPEAKER_01
Now what that meant, practically, is that the push from launch to, first of all, the fall, but then to 2.0 and 3.0 and, like, all the way forward was super painful for the engineering team and myself.
8
SPEAKER_01
It was very stressful.
8
SPEAKER_01
The developer community was very grumpy about it because they re like, okay, well, wait a second, you re changing and breaking my code and, like,
8
SPEAKER_01
We have to fix the bugs and it was just like a lot of tension and friction on all sides.
8
SPEAKER_01
There s a lot of technical debt in the compiler because we have to run really fast.
8
SPEAKER_01
You have to go implement the thing and unblock the use case and do the thing and you know it s not right, but you never have time to go back and do it right.
8
SPEAKER_01
And I m very proud of the Swift team because they ve come.
8
SPEAKER_01
We, but they came so far and made so much progress over this time since launch.
8
SPEAKER_01
It s pretty incredible, and Swift is a very, very good thing.
8
SPEAKER_01
But I just don t want to do that again, right?
8
SPEAKER_00
And so... So iterate more through the development process.
8
SPEAKER_01
Yeah, and so what we re doing is we re not launching it when it s hopefully 0.9 with no testers.
8
SPEAKER_01
We re launching it and saying it s 0.1, right?
8
SPEAKER_01
And so we re setting expectations of saying like, okay, well, don t use this for production.
8
SPEAKER_01
If you re interested in what we re doing, we ll do it in an open way, and we can do it together, but don t use it in production yet.
8
SPEAKER_01
We ll get there, but let s do it the right way.
8
SPEAKER_01
And I m also saying, we re not in a race.
8
SPEAKER_01
The thing that I want to do is build the world s best thing.
8
SPEAKER_01
Because if you do it right and it lifts the industry, it doesn t matter if it takes an extra two months.
8
SPEAKER_01
like two months is worth waiting.
8
SPEAKER_01
And so doing it right and not being overwhelmed with technical debt and things like this is like, again, war wounds, lessons learned, whatever you wanna say, I think is absolutely the right thing to do, even though right now people are very frustrated that you can t download it or that it doesn t have feature X or something like this.
8
SPEAKER_01
And so.
8
SPEAKER_00
What have you learned in the, in a little bit of time since it s been,
8
SPEAKER_00
Released into the wild that people have been complaining about future X or Y or Z What have they been complaining about what they have been?
8
SPEAKER_00
excited about like, almost like detailed things versus a big vision.
8
SPEAKER_00
I think everyone would be very excited about the big vision.
8
SPEAKER_01
Yeah.
8
SPEAKER_01
Yeah.
8
SPEAKER_01
Well, so, I mean, I ve been very pleased.
8
SPEAKER_01
I mean, in fact, I mean, we ve been massively overwhelmed with response, which is a good problem to have.
8
SPEAKER_01
It s kind of like a success disaster in a sense, right?
8
SPEAKER_01
So, I mean, if you go back in time, when we started Modular, which was just not yet a year and a half ago, so it s still a pretty new company, new team, small but very good team of people, like we started with extreme conviction that there s a set of problems that we need to solve, and if we solve it, then people will be interested in what we re doing, right?
8
SPEAKER_01
But again, you re building in basically secret.
8
SPEAKER_01
You re trying to figure it out.
8
SPEAKER_01
Creation s a messy process.
8
SPEAKER_01
You re having to go through different paths and understand what you want to do and how to explain it.
8
SPEAKER_01
Often when you re doing disruptive and new kinds of things, just knowing how to explain it is super difficult, right?
8
SPEAKER_01
And so when we launched, we hoped people would be excited.
8
SPEAKER_01
But, you know, I m an optimist, but I m also like, don t want to get ahead of myself.
8
SPEAKER_01
And so when people found out about Mojo, I think their heads exploded a little bit, right?
8
SPEAKER_01
And, you know, here s, I think, a pretty credible team that has built some languages and some tools before, and so they have some lessons learned.
8
SPEAKER_01
and are tackling some of the deep problems in the Python ecosystem and giving it the love and attention that it should be getting.
8
SPEAKER_01
And I think people got very excited about that.
8
SPEAKER_01
And so if you look at that, I mean, I think people are excited about ownership and taking a step beyond rust, right?
8
SPEAKER_01
And there s people that are very excited about that.
8
SPEAKER_01
And there s people that are excited about, you know, just like I made Game of Life go 400 times faster.
8
SPEAKER_01
Right, and things like that, and that s really cool.
8
SPEAKER_01
There are people that are really excited about the, okay, I really hate writing stuff in C++, save me.
8
SPEAKER_00
Like systems engineers, they re like stepping up like, oh, yes.
8
SPEAKER_01
So that s me, by the way.
8
SPEAKER_01
Also, I really wanna stop writing C++.
8
SPEAKER_00
I get third-person excitement when people tweet, yeah, I made this code, game of life, or whatever, faster, and you re like,
8
SPEAKER_01
Yeah, and also like, I would also say that, let me cast blame out to people who deserve it.
8
SPEAKER_01
Sure.
8
SPEAKER_01
These terrible people who convinced me to do some of this.
8
SPEAKER_01
Yes.
8
SPEAKER_01
Jeremy Howard.
8
SPEAKER_01
Yes.
8
SPEAKER_01
That guy.
8
SPEAKER_00
Well, he s been pushing for this kind of thing.
8
SPEAKER_00
He s been pushing for Mojo.
8
SPEAKER_00
He s wanted this for years.
8
SPEAKER_00
Yeah, he s wanted this for a long, long time.
8
SPEAKER_01
He s wanted this for years.
8
SPEAKER_00
For people who don t know Jeremy Howard, he s like one of the most legit people in the machine learning community.
8
SPEAKER_00
He s a grassroots, he really teaches, he s an incredible educator, he s an incredible teacher, but also legit in terms of a machine learning engineer himself.
8
SPEAKER_00
He s been running the fast.ai and looking, I think, for exactly what you ve done with Mojo.
8
SPEAKER_01
Exactly, and so, I mean,
8
SPEAKER_01
So I met Jeremy pretty early on, but the first time I sat up and I m like, this guy is ridiculous, is when I was at Google and we were bringing up TPUs and we had a whole team of people and there was this competition called DawnBench of who can train ImageNet fastest, right?
8
SPEAKER_01
And Jeremy and one of his researchers crushed Google.
8
SPEAKER_01
not through sheer force of the amazing amount of compute and the number of TPUs and stuff like that, that he just decided that progressive imagery sizing was the right way to train the model and fewer epochs faster and make the whole thing go vroom, right?
8
SPEAKER_01
And I m like, this guy is incredible.
8
SPEAKER_01
And so you can say, anyways, come back to where s Mojo coming from.
8
SPEAKER_01
Chris finally listened to Jeremy.
8
SPEAKER_01
It s all his fault.
8
SPEAKER_00
There s a kind of very refreshing, pragmatic view that he has about machine learning that
8
SPEAKER_00
I don t know if it s like this mix of a desire for efficiency, but ultimately grounded in a desire to make machine learning more accessible to a lot of people.
8
SPEAKER_00
I don t know what that is.
8
SPEAKER_00
I guess that s coupled with efficiency and performance, but it s not just obsessed about performance.
8
SPEAKER_01
So a lot of AI and AI research ends up being that it has to go fast enough to get scale.
8
SPEAKER_01
So a lot of people don t actually care about performance, particularly on the research side, until it allows them to have a bigger data set.
8
SPEAKER_01
And so suddenly now you care about distributed compute and all these exotic HPC.
8
SPEAKER_01
You don t actually want to know about that.
8
SPEAKER_01
You just want to be able to do more experiments faster and do so with bigger data sets.
8
SPEAKER_01
And so Jeremy has been really pushing the limits.
8
SPEAKER_01
And one of the things I ll say about Jeremy, and there s many things I could say about Jeremy because I m a fanboy of his, but it fits in his head.
8
SPEAKER_01
And Jeremy actually takes the time, where many people don t, to really dive deep into why is the beta parameter of the atom optimizer equal to this?
8
SPEAKER_01
And he ll go survey and understand what are all the activation functions and the trade-offs and why is it that everybody that does
8
SPEAKER_01
you know, this model, pick that thing.
8
SPEAKER_01
So the why, not just trying different values, like really, what is going on here?
8
SPEAKER_01
Right.
8
SPEAKER_01
And so as a consequence of that, like he s always, again, he makes time, but he spends time to understand things at a depth that a lot of people don t.
8
SPEAKER_01
And as you say, he then brings it and teaches people.
8
SPEAKER_01
And his mission is to help lift
8
SPEAKER_01
you know, his website says, making AI uncool again.
8
SPEAKER_01
Like, it s about, like, forget about the hype, let s, it s actually practical and useful, let s teach people how to do this, right?
8
SPEAKER_01
Now, the problem Jeremy struggled with is that he s pushing the envelope.
8
SPEAKER_01
Research isn t about doing the thing that is staying on the happy path or the well-paved road, right?
8
SPEAKER_01
And so a lot of the systems today have been these really fragile fragmented things or special case in this happy path.
8
SPEAKER_01
And if you fall off the happy path, you get eaten by an alligator.
8
SPEAKER_00
What about, so Python has this giant ecosystem of packages and there s a package repository.
8
SPEAKER_00
Do you have ideas of how to do that well for Mojo?
8
SPEAKER_00
How to do a repository of packages?
9
SPEAKER_01
Well, so that s another really interesting problem that I knew about, but I didn t understand how big of a problem it was.
9
SPEAKER_01
Python packaging, a lot of people have very big pain points and a lot of scars with Python packaging.
9
SPEAKER_00
Oh, you mean, so there s several things at stake.
9
SPEAKER_01
Building and distributing and managing dependencies and versioning and all this stuff.
9
SPEAKER_00
So from the perspective of if you want to create your own package.
9
SPEAKER_01
Yes.
9
SPEAKER_01
Or you want to build on top of a bunch of other people s packages and then they get updated and it s like this.
9
SPEAKER_01
Now, I m not an expert in this, so I don t know the answer.
9
SPEAKER_01
I think this is one of the reasons why it s great that we work as a team and there s other really good and smart people involved.
9
SPEAKER_01
But one of the things I ve heard from smart people who ve done a lot of this is that the packaging becomes a huge disaster when you get the Python and C together.
9
SPEAKER_01
And so if you have this problem where you have code split between Python and C, now not only do you have to package the C code, you have to build the C code.
9
SPEAKER_01
C doesn t have a package manager, right?
9
SPEAKER_01
C doesn t have a dependency versioning management system, right?
9
SPEAKER_01
And so I m not experienced in the state of the art and all the different Python package managers, but my understanding is that s a massive part of the problem.
9
SPEAKER_01
And I think Mojo solves that part of the problem directly heads on.
9
SPEAKER_01
Now, one of the things I think we ll do with the community, and this isn t, again, we re not solving all the world s problems at once, we have to be kind of focused to start with, is that I think that we will have an opportunity to reevaluate packaging, right?
9
SPEAKER_01
And so I think that we can come back and say, okay, well, given the new tools and technologies and the cool things we have that we ve built up, because we have not just syntax, we have an entirely new compiler stack that works in a new way, maybe there s other innovations we can bring together and maybe we can help solve that problem.
9
SPEAKER_00
It s almost a tangent to that question from the user perspective of packages.
9
SPEAKER_00
It was always surprising to me that it was not easier to sort of explore and find packages.
9
SPEAKER_00
with PIP install and it just, it feels, it s an incredible ecosystem.
9
SPEAKER_00
It s just interesting that it wasn t made, it s still, I think, not made easier to discover packages to do like a search and discovery, as YouTube calls it.
9
SPEAKER_01
Well, I mean, it s kind of funny because this is one of the challenges of these intentionally decentralized communities.
9
SPEAKER_01
And so I don t know what the right answer is for Python.
9
SPEAKER_01
I mean, there are many people that would
9
SPEAKER_01
I don t even know the right answer for Mojo.
9
SPEAKER_01
So there are many people that would have much more informed opinions than I do.
9
SPEAKER_01
But it s interesting if you look at this, right?
9
SPEAKER_01
Open source communities.
9
SPEAKER_01
You know, there s Git.
9
SPEAKER_01
Git is a fully decentralized, anybody can do it any way they want.
9
SPEAKER_01
But then there s GitHub, right?
9
SPEAKER_01
And GitHub, centralized, commercial in that case, right, thing, really helped pull together and help solve some of the discovery problems and help build a more
9
SPEAKER_00
consistent community, and so maybe there s opportunities for... There s something like a GitHub for... Yeah.
9
SPEAKER_00
Although even GitHub, I might be wrong on this, but the search and discovery for GitHub is not that great.
9
SPEAKER_00
Like, I still use Google Search.
9
SPEAKER_01
Yeah, well, I mean, maybe that s because GitHub doesn t want to replace Google Search, right?
9
SPEAKER_01
I think there is room for specialized solutions to specific problems, but... Sure.
9
SPEAKER_01
I don t know.
9
SPEAKER_01
I don t know the right answer for GitHub either.
9
SPEAKER_01
They can go figure that out.
9
SPEAKER_00
But the point is to have an interface that s usable, that s accessible to people of all different skill levels.
9
SPEAKER_01
Well, and again, what are the benefit of standards, right?
9
SPEAKER_01
Standards allow you to build these next level up ecosystem, next level up infrastructure, next level up things.
9
SPEAKER_01
And so, again, come back to, I hate complexity.
9
SPEAKER_01
C plus Python is complicated.
9
SPEAKER_01
It makes everything more difficult to deal with.
9
SPEAKER_01
It makes it difficult to port, move code around, work with.
9
SPEAKER_01
All these things get more complicated.
9
SPEAKER_01
And so, I mean, I m not an expert, but maybe Mojo can help a little bit by helping reduce the amount of C in this ecosystem and make it therefore scale better.
9
SPEAKER_00
So any kind of packages that are hybrid in nature would be a natural fit to move to Mojo.
9
SPEAKER_00
Which is a lot of them, by the way.
9
SPEAKER_00
A lot of them, especially, they re doing some interesting stuff computation-wise.
9
SPEAKER_00
Let me ask you about some features.
9
SPEAKER_00
So we talked about, obviously, indentation, that it s a typed language, or optionally typed.
9
SPEAKER_00
Is that the right way to say it?
9
SPEAKER_01
It s either optionally or progressively.
9
SPEAKER_01
Progressively.
9
SPEAKER_01
So people have very strong opinions on the right word to use.
9
SPEAKER_01
I don t know.
9
SPEAKER_00
I look forward to your letters.
9
SPEAKER_00
So there s the var versus let, but let is for constants.
9
SPEAKER_00
Var is an optional.
9
SPEAKER_01
Yeah, var makes it mutable, so you can reassign.
9
SPEAKER_00
Okay.
9
SPEAKER_00
Then there s function overloading.
9
SPEAKER_01
Oh, okay, yeah.
9
SPEAKER_00
I mean, there s a lot of source of happiness for me, but function overloading, that s, I guess, is that for performance?
9
SPEAKER_00
Why does Python not have function overloading?
9
SPEAKER_00
So I can speculate.
9
SPEAKER_01
So Python is a dynamic language.
9
SPEAKER_01
The way it works is that Python and Objective-C are actually very similar worlds if you ignore syntax.
9
SPEAKER_01
And so Objective-C is straight line derived from Smalltalk.
9
SPEAKER_01
a really venerable, interesting language that much of the world has forgotten about, but the people that remember it love it, generally.
9
SPEAKER_01
And the way that Smalltalk works is that every object has a dictionary in it, and the dictionary maps from the name of a function, or the name of a value within an object, to its implementation.
9
SPEAKER_01
And so the way you call a method in Objective-C is you say, go look up, the way I call foo is I go look up foo, I get a pointer to the function back, and then I call it.
9
SPEAKER_01
That s how Python works.
9
SPEAKER_01
Right, and so now the problem with that is that the dictionary within a Python object, all the keys are strings.
9
SPEAKER_01
and it s a dictionary, so you can only have one entry per name.
9
SPEAKER_01
You think it s as simple as that?
9
SPEAKER_01
I think it s as simple as that.
9
SPEAKER_01
Now, why do they never fix this?
9
SPEAKER_01
Why do they not change it to not be a dictionary?
9
SPEAKER_01
I ll do other things.
9
SPEAKER_01
Well, you don t really have to in Python because it s dynamic.
9
SPEAKER_01
You can say, I get into the function.
9
SPEAKER_01
Now, if I got past an integer, do some dynamic test for it.
9
SPEAKER_01
If it s a string, go do another thing.
9
SPEAKER_01
There s another additional challenge, which is even if you did support overloading, you re saying, okay, well, here s a version of a function for integers and a function for strings.
9
SPEAKER_01
Well, even if you could put it in that dictionary, you d have to have the caller do the dispatch.
9
SPEAKER_01
So every time you call the function, you d have to say, is it an integer or is it a string?
9
SPEAKER_01
So you d have to figure out where to do that test.
9
SPEAKER_01
So in a dynamic language, overloading is something you don t have to have.
9
SPEAKER_01
But now you get into a typed language, and in Python, if you subscript with an integer, then you get typically one element out of a collection.
9
SPEAKER_01
If you subscript with a range, you get a different thing out.
9
SPEAKER_01
So often, in typed languages, you ll want to be able to express the fact that, cool, I have different behavior depending on what I actually pass into this thing.
9
SPEAKER_01
If you can model that, it can make it safer, and more predictable, and faster, and all these things.
9
SPEAKER_00
It somehow feels safer, yes, but it also feels empowering in terms of clarity, like you don t have to design whole different functions.
9
SPEAKER_01
Yeah, well, this is also one of the challenges with the existing Python typing systems, is that in practice, like you take subscript, like in practice, a lot of these functions, they don t have one signature.
9
SPEAKER_01
They actually have different behavior in different cases, and so this is why it s difficult to retrofit this into existing Python code and make it play well with typing.
9
SPEAKER_01
You kind of have to design for that.
9
SPEAKER_00
Okay, so there s an interesting distinction
9
SPEAKER_00
that people that program Python might be interested in is def versus fn.
9
SPEAKER_00
So it s two different ways to define a function.
9
SPEAKER_00
And fn is a stricter version of def.
9
SPEAKER_00
What s the coolness that comes from the strictness?
9
SPEAKER_01
So here you get into, what is the trade-off with a superset?
9
SPEAKER_01
So a superset, you have to, or you really want to be compatible.
9
SPEAKER_01
If you re doing a superset, you ve decided compatibility with existing code is the important thing, even if some of the decisions they made were maybe not what you d choose.
9
SPEAKER_01
So that means you put a lot of time into compatibility, and it means that you get locked into decisions of the past.
9
SPEAKER_01
even if they may not have been a good thing, right?
9
SPEAKER_01
Now, systems programmers typically like to control things, right?
9
SPEAKER_01
And they wanna make sure that, you know, not in all cases, of course, and even systems programmers are not one thing, right?
9
SPEAKER_01
But often you want predictability.
9
SPEAKER_01
And so one of the things that Python has, for example, as you know, is that if you define a variable, you just say x equals four.
9
SPEAKER_01
I have a variable named x.
9
SPEAKER_01
Now I say some long name equals 17.
9
SPEAKER_01
Print out some long name.
9
SPEAKER_01
Oops, but I typoed it.
9
SPEAKER_01
Right, well, the compiler, the Python compiler doesn t know, in all cases, what you re defining and what you re using, and did you typo the use of it or the definition?
9
SPEAKER_01
And so for people coming from typed languages, again, I m not saying they re right or wrong, but that drives them crazy because they want the compiler to tell them, you typoed the name of this thing.
9
SPEAKER_01
And so what Fn does is it turns on, as you say, it s a strict mode.
9
SPEAKER_01
And so it says, OK, well, you have to actually intentionally declare your variables before you use them.
9
SPEAKER_01
That gives you more predictability, more error checking, and things like this.
9
SPEAKER_01
But you don t have to use it.
9
SPEAKER_01
And this is a way that Mojo is both compatible, because defs work the same way that defs have always worked, but it provides a new alternative that gives you more control and it allows certain kinds of people that have a different philosophy to be able to express that and get that.
9
SPEAKER_00
But usually, if you re writing Mojo code from scratch, you ll be using FN.
9
SPEAKER_01
It depends, again, it depends on your mentality, right?
9
SPEAKER_01
It s not that def is Python and fn is Mojo.
9
SPEAKER_01
Mojo has both, and it loves both, right?
9
SPEAKER_01
It really depends on.
9
SPEAKER_01
That is just strict.
9
SPEAKER_01
Yeah, exactly.
9
SPEAKER_01
Are you playing around and scripting something out, and is it a one-off throwaway script?
9
SPEAKER_01
Cool, like Python is great at that.
9
SPEAKER_00
I ll still be using fn, but yeah.
9
SPEAKER_00
Well, so.
9
SPEAKER_00
I love strictness.
9
SPEAKER_01
Okay, well, so.
9
SPEAKER_00
Control.
9
SPEAKER_01
suffering, right?
9
SPEAKER_00
Yes.
9
SPEAKER_01
Go hand in hand.
9
SPEAKER_01
How many pull-ups?
9
SPEAKER_00
I ve lost count at this point.
9
SPEAKER_01
So that s cool.
9
SPEAKER_01
I love you for that.
9
SPEAKER_01
Yeah.
9
SPEAKER_01
And I love other people who like strict things, right?
9
SPEAKER_01
But I don t want to say that that s the right thing because Python is also very beautiful for hacking around and doing stuff and research and these other cases where you may not want that.
9
SPEAKER_00
See, I just feel like
9
SPEAKER_00
maybe I m wrong on that, but it feels like strictness leads to faster debugging.
9
SPEAKER_00
So in terms of going from, even on a small project, from zero to completion, it just, I guess it depends how many bugs you generate, usually.
9
SPEAKER_01
Well, so I mean, again, lessons learned in looking at the ecosystem, it s really,
9
SPEAKER_01
I mean, I think it s, if you study some of these languages over time, like the Ruby community, for example.
9
SPEAKER_01
Now, Ruby is a pretty well-developed, pretty established community, but along their path, they really invested in unit testing.
9
SPEAKER_01
So I think that the Ruby community has really pushed forward the state of the art of testing because they didn t have a type system that caught a lot of bugs in compile time.
9
SPEAKER_01
And so you can have the best of both worlds.
9
SPEAKER_01
You can have good testing and good types and things like this.
9
SPEAKER_01
But I thought that it was really interesting to see how certain challenges get solved.
9
SPEAKER_01
And in Python, for example,
9
SPEAKER_01
the interactive notebook kind of experiences and stuff like this are really amazing.
9
SPEAKER_01
And if you typo something, it doesn t matter, it just tells you, it s fine, right?
9
SPEAKER_01
And so I think that the trade-offs are very different if you re building a large-scale production system versus you re building and exploring a notebook.
9
SPEAKER_00
And speaking of control, the hilarious thing, if you look at code I write just for myself for fun, it s like littered with asserts everywhere.
9
SPEAKER_00
It s basically saying in a dictatorial way, this should be true now, otherwise everything stops.
9
SPEAKER_01
And that is the sign.
9
SPEAKER_01
I love you, man.
9
SPEAKER_01
That is a sign of somebody who likes control.
9
SPEAKER_01
And so yes, I think that you ll like FN.
9
SPEAKER_01
I think you ll like Mojo.
9
SPEAKER_00
Therapy session, yes, I definitely will.
9
SPEAKER_00
Speaking of asserts, exceptions are called errors.
9
SPEAKER_00
Why is it called errors?
9
SPEAKER_01
So we, I mean, we use the same, we re the same as Python, right?
9
SPEAKER_01
But we implement in a very different way, right?
9
SPEAKER_01
And so if you look at other languages, like we ll pick on C++, our favorite, right?
9
SPEAKER_01
C++ has a thing called zero cost exception handling.
9
SPEAKER_01
Okay, and this is, in my opinion, something to learn lessons from.
9
SPEAKER_00
It s a nice polite way of saying it.
9
SPEAKER_01
And so zero-cost exception handling, the way it works is that it s called zero-cost because
9
SPEAKER_01
If you don t throw an exception, there s supposed to be no overhead for the non-error code.
9
SPEAKER_01
So it takes the error path out of the common path.
9
SPEAKER_01
It does this by making throwing an error extremely expensive.
9
SPEAKER_01
So if you actually throw an error with a C++ compiler using exceptions, has to go look up in tables on the side and do all this stuff.
9
SPEAKER_01
So throwing an error could be like 10,000 times more expensive than referring from a function.
9
SPEAKER_01
Also, it s called zero-cost exceptions, but it s not zero-cost by any stretch of the imagination, because it massively bloats out your code, your binary.
9
SPEAKER_01
It also adds a whole bunch of different paths because of destructors and other things like that that exist in C++.
9
SPEAKER_01
And it reduces the number of optimizations, it adds all these effects.
9
SPEAKER_01
And so this thing that was called zero-cost exceptions,
9
SPEAKER_01
It really ain t.
9
SPEAKER_01
Now, if you fast forward to newer languages, and this includes Swift and Rust and Go and now Mojo.
9
SPEAKER_01
Well, in Python, it s a little bit different because it s interpreted, and so it s got a little bit of a different thing going on.
9
SPEAKER_01
But if you look at compiled languages,
9
SPEAKER_01
Many newer languages say, OK, well, let s not do that zero cost exception handling thing.
9
SPEAKER_01
Let s actually treat throwing an error the same as returning a variant, returning either the normal result or an error.
9
SPEAKER_01
Now, programmers generally don t want to deal with all the typing machinery and pushing around a variant, and so you use all the syntax that Python gives us.
9
SPEAKER_01
For example, try and catch functions that raise and things like this.
9
SPEAKER_01
You can put a raises decorator on your functions, stuff like this, if you want to control that, and then the language can provide syntax for it.
9
SPEAKER_01
But under the hood, the way the computer executes it, throwing an error is basically as fast as returning something.
9
SPEAKER_00
So it s exactly the same way from a compiler perspective.
9
SPEAKER_01
And so this is actually, I mean, it s a fairly nerdy thing, right?
9
SPEAKER_01
Which is why I love it.
9
SPEAKER_01
But this has a huge impact on the way you design your APIs, right?
9
SPEAKER_01
So in C++, huge communities turn off exceptions because the cost is just so high, right?
9
SPEAKER_01
And so the zero cost cost is so high, right?
9
SPEAKER_01
And so that means you can t actually use exceptions in many libraries.
9
SPEAKER_01
And even for the people that do use it, well, okay, how and when do you want to pay the cost?
9
SPEAKER_01
If I try to open a file, should I throw an error?
9
SPEAKER_01
Well, what if I m probing around looking for something, right?
9
SPEAKER_01
I m looking it up in many different paths.
9
SPEAKER_01
Well, if it s really slow to do that, maybe I ll add another function.
9
SPEAKER_01
that doesn t throw an error, it returns an error code instead, and now I have two different versions of the same thing, and so it causes you to fork your APIs.
9
SPEAKER_01
And so, you know, one of the things I learned from Apple and that I still love is the art of API design is actually really profound.
9
SPEAKER_01
I think this is something that Python s also done a pretty good job at in terms of building out this large-scale package ecosystem.
9
SPEAKER_01
It s about having standards and things like this.
9
SPEAKER_01
And so, you know, we wouldn t want to enter a mode where,
9
SPEAKER_01
you know, there s this theoretical feature that exists in language, but people don t use it in practice.
9
SPEAKER_01
Now, I ll also say one of the other really cool things about this implementation approach is that it can run on GPUs and it can run on accelerators and things like this.
9
SPEAKER_01
And that standard zero cost exception thing would never work on an accelerator.
9
SPEAKER_01
And so this is also part of how Mojo can scale all the way down to like little embedded systems and to running on GPUs and things like that.
9
SPEAKER_00
Can you actually say about the
9
SPEAKER_00
Maybe, is there some high-level way to describe the challenge of exceptions and how they work in code during compilation?
9
SPEAKER_00
So just this idea of percolating up a thing, an error.
9
SPEAKER_01
Yeah, so the way to think about it is, think about a function that doesn t return anything, just as a simple case, right?
9
SPEAKER_01
And so you have function one calls function two,
10
SPEAKER_01
function2, calls function3, calls function4, along that call stack that are tri-blocks, right?
10
SPEAKER_01
And so if you have function1, calls function2, function2 has a tri-block, and then within it, it calls function3, right?
10
SPEAKER_01
Well, what happens if function3 throws?
10
SPEAKER_01
Well, actually, start simpler.
10
SPEAKER_01
What happens if it returns?
10
SPEAKER_01
Well, if it returns, it s supposed to go back out and continue executing, and then fall off the bottom of the try block, and keep going, and all s good.
10
SPEAKER_01
If the function throws, you re supposed to exit the current function, and then get into the accept clause, right?
10
SPEAKER_01
And then do whatever code s there, and then keep following on and going on.
10
SPEAKER_01
So the way that a compiler like Mojo works is that the call to that function, which happens in the accept block, calls the function, and then instead of returning nothing, it actually returns a variant between nothing and an error.
10
SPEAKER_01
So if you return normally, fall off the bottom or do a return,
10
SPEAKER_01
You return nothing, and if you throw an error, you return the variant that is, I m an error, right?
10
SPEAKER_01
So when you get to the call, you say, okay, cool, I called a function.
10
SPEAKER_01
Hey, I know locally I m in a try block, right?
10
SPEAKER_01
And so I call the function, and then I check to see what it returns.
10
SPEAKER_01
Aha, if it s that error thing, jump to the accept block.
10
SPEAKER_00
And that s all done for you behind the scenes.
10
SPEAKER_01
Exactly, and so the compiler does all this for you.
10
SPEAKER_01
And I mean, one of the things, if you dig into how this stuff works in Python, it gets a little bit more complicated because you have finally blocks, which now you need to go into, do some stuff, and then.
10
SPEAKER_01
Those can also throw and return.
10
SPEAKER_01
Wait, what?
10
SPEAKER_01
Nested?
10
SPEAKER_01
Like the stuff matters for compatibility.
10
SPEAKER_01
Really?
10
SPEAKER_01
You nest them?
10
SPEAKER_01
There s with clauses and so with clauses are kind of like finally blocks with some special stuff going on and so there s.
10
SPEAKER_00
Nesting in general, nesting of anything, nesting of functions should be illegal.
10
SPEAKER_00
It just feels like it adds a level of complexity.
10
SPEAKER_01
I m merely an implementer, and so this is, again, one of the trade-offs you get when you decide to build a superset is you get to implement a full fidelity implementation of the thing that you decided is good.
10
SPEAKER_00
Yeah, I mean, we can complain about the reality of the world and shake our fists, but... It always feels like you shouldn t be allowed to do that, like to declare functions inside functions.
10
SPEAKER_01
Wait, what happened to Lex, the Lisp guy?
10
SPEAKER_00
No, I understand that, but Lisp is what I used to do in college.
10
SPEAKER_01
So now you ve grown up?
10
SPEAKER_00
You know, we ve all done things in college, we re not proud.
10
SPEAKER_00
I love LISP.
10
SPEAKER_01
I love LISP.
10
SPEAKER_01
I was going to say, you re afraid of me irritating the whole internet?
10
SPEAKER_00
I love LISP.
10
SPEAKER_00
It worked as a joke in my head.
10
SPEAKER_01
So nested functions are, joking aside, actually really great for certain things, right?
10
SPEAKER_01
And so these are also called closures.
10
SPEAKER_01
Closures are pretty cool, and you can pass callbacks.
10
SPEAKER_01
There s a lot of good patterns.
10
SPEAKER_00
So speaking of which, I don t think you have nested functions implemented yet in Mojo.
10
SPEAKER_01
We don t have lambda syntax, but we do have nested functions, yeah.
10
SPEAKER_00
There s a few things on the roadmap that you have that would be cool to sort of just fly through, because it s interesting to see how many features there are in a language, small and big.
10
SPEAKER_01
Yep.
10
SPEAKER_00
do you have to implement?
10
SPEAKER_00
So first of all, there s tuple support, and that has to do with some very specific aspect of it, like the parentheses are not parentheses, that?
10
SPEAKER_01
Yeah, this is just totally a syntactic thing.
10
SPEAKER_00
A syntactic thing.
10
SPEAKER_00
Okay, but it s cool still.
10
SPEAKER_00
So keyword arguments and functions?
10
SPEAKER_01
Yeah, so this is where in Python you can say, call a function x equals four.
10
SPEAKER_00
Yeah.
10
SPEAKER_00
And x is the name of the argument.
10
SPEAKER_00
That s a nice sort of documenting, self-documenting feature.
10
SPEAKER_01
Yeah, I mean, and again, this isn t rocket science to implement, this is just the laundry list.
10
SPEAKER_01
It s just on the list.
10
SPEAKER_01
The bigger features are things like traits.
10
SPEAKER_01
So traits are when you want to define abstract.
10
SPEAKER_01
So when you get into typed languages, you need the ability to write generics.
10
SPEAKER_01
And so you want to say, I want to write this function, and now I want to work on all things that are arithmetic-like.
10
SPEAKER_01
What does arithmetic-like mean?
10
SPEAKER_01
Well, arithmetic-like is a categorization of a bunch of types.
10
SPEAKER_01
Again, you can define it in many different ways, and I m not going to go into ring theory or something, but you can say it s arithmetic-like if you can add, subtract, multiply, divide it, for example.
10
SPEAKER_01
So what you re saying is you re saying there s a set of traits that apply to a broad variety of types,
10
SPEAKER_01
And so all these types are arithmetic-like, all these tensors and floating point integer, and there s this category of types.
10
SPEAKER_01
And then I can define, on an orthogonal axis, algorithms that then work against types that have those properties.
10
SPEAKER_01
And so this is a, again, it s a widely known thing.
10
SPEAKER_01
It s been implemented in Swift and Rust and many languages.
10
SPEAKER_01
So it s not Haskell, which is where everybody learns their tricks from.
10
SPEAKER_01
But we need to implement that and that ll enable a new level of expressivity.
10
SPEAKER_01
So classes.
10
SPEAKER_01
Yeah, classes are a big deal.
10
SPEAKER_00
It s a big deal still to be implemented.
10
SPEAKER_00
Like you said, Lambda syntax.
10
SPEAKER_00
And there s like detailed stuff like whole module import, support for top level code and file scope.
10
SPEAKER_00
And then global variables also.
10
SPEAKER_00
So being able to have variables outside of a top level.
10
SPEAKER_01
Well, and so this comes back to the where Mojo came from and the fact that this is 0.1, right?
10
SPEAKER_01
And so we re building, so Modular s building an AI stack, right?
10
SPEAKER_01
And an AI stack has a bunch of problems working with hardware and writing high-performance kernels and doing this kernel fusion thing I was talking about and getting the most out of the hardware.
10
SPEAKER_01
And so we ve really prioritized and built Mojo to solve Mojo s problem.
10
SPEAKER_01
Right now, our North Star is build out and support all the things.
10
SPEAKER_01
And so we re making incredible progress.
10
SPEAKER_01
By the way, Mojo is only like seven months old.
10
SPEAKER_01
So that s another interesting thing.
10
SPEAKER_00
I mean, part of the reason I wanted to mention some of these things is like, there s a lot to do.
10
SPEAKER_00
And it s pretty cool how you just kind of
10
SPEAKER_00
Sometimes you take for granted how much there is in a programming language, how many cool features you can rely on.
10
SPEAKER_00
This is a nice reminder when you lay it as a to-do list.
10
SPEAKER_01
Yeah.
10
SPEAKER_01
But also you look into, it s amazing how much is also there.
10
SPEAKER_01
You take it for granted that a value, if you define it, it will get destroyed automatically.
10
SPEAKER_01
Like that little feature itself is actually really complicated, given the way the ownership system has to work, and the way that works within Mojo is a huge step forward from what Rust and Swift have done.
10
SPEAKER_01
But can you say that again?
10
SPEAKER_01
When value, when you define it, it gets destroyed automatically?
10
SPEAKER_01
Yeah, so like say you have a string, right?
10
SPEAKER_01
So you just find a string on the stack, okay?
10
SPEAKER_01
Or whatever that means, like in your local function, right?
10
SPEAKER_01
And so you say, like whether it be in a def, and so you just say x equals hello world, right?
10
SPEAKER_01
Well, if your string type requires you to allocate memory, then when it s destroyed, you have to deallocate it.
10
SPEAKER_01
So in Python and Mojo, you define that with the del method, right?
10
SPEAKER_01
Where does that get run?
10
SPEAKER_01
Well, it gets run sometime between the last use of the value and the end of the program.
10
SPEAKER_01
Like in this, you now get into garbage collection, you get into like all these long debated, you talk about religions and trade-offs and things like this, this is a hugely hotly contested world.
10
SPEAKER_01
If you look at C++ the way this works is that if you define a variable or a set of variables within a function They get destroyed in a last in first out order.
10
SPEAKER_01
So it s like nesting.
10
SPEAKER_01
Okay?
10
SPEAKER_01
This has a huge problem because if you define you have a big scope and you define a whole bunch of values at the top
10
SPEAKER_01
and then you use them, and then you do a whole bunch of code that doesn t use them, they don t get destroyed until the very end of that scope, right?
10
SPEAKER_01
And so this also destroys tail calls, so good functional programming, right?
10
SPEAKER_01
This has a bunch of different impacts on, you know, you talk about reference counting optimizations and things like this, a bunch of very low-level things.
10
SPEAKER_01
And so what Mojo does is it has a different approach on that from any language I m familiar with, where it destroys them as soon as possible.
10
SPEAKER_01
And by doing that, you get better memory use, you get better predictability, you get tail calls that work, you get a bunch of other things, you get better ownership tracking.
10
SPEAKER_01
There s a bunch of these very simple things that are very fundamental, that are already built in there in Mojo today, that are the things that nobody talks about generally, but when they don t work right, you find out and you have to complain about.
10
SPEAKER_00
Is it trivial to know what s the soonest possible to delete a thing that s not gonna be used again?
10
SPEAKER_01
Yeah, well, I mean, it s generally trivial.
10
SPEAKER_01
It s after the last use of it.
10
SPEAKER_01
So if you find X as a string, and then you have some use of X somewhere in your code.
10
SPEAKER_01
Within that scope, you mean?
10
SPEAKER_01
Within the scope that is accessible?
10
SPEAKER_01
Yeah, exactly.
10
SPEAKER_01
So you can only use something within its scope.
10
SPEAKER_01
And so then it doesn t wait until the end of the scope to delete it.
10
SPEAKER_01
It destroys it after the last use.
10
SPEAKER_00
So there s kind of some very eager machine that s just sitting there and deleting.
10
SPEAKER_01
Yeah, and it s all in the compiler, so it s not at runtime, which is also cool.
10
SPEAKER_01
And so, yeah, and so what, and this is actually non-trivial because you have control flow.
10
SPEAKER_01
Right.
10
SPEAKER_01
And so it gets complicated pretty quickly.
10
SPEAKER_01
And so, like, getting this right was not... Oh, so you have to insert delete, like, in a lot of places.
10
SPEAKER_01
Potentially, yeah, exactly.
10
SPEAKER_01
So the compiler has to reason about this, and this is where...
10
SPEAKER_01
Again, it s experience building languages and not getting this right.
10
SPEAKER_01
So again, you get another chance to do it and you get basic things like this, right?
10
SPEAKER_01
But it s extremely powerful when you do that, right?
10
SPEAKER_01
And so there s a bunch of things like that that kind of combine together.
10
SPEAKER_01
And this comes back to the, you get a chance to do it the right way, do it the right way and make sure that every brick you put down is really good.
10
SPEAKER_01
So that when you put more bricks on top of it, they stack up to something that s beautiful.
10
SPEAKER_00
Well, there s also like how many,
10
SPEAKER_00
design discussions, do there have to be about particular details, like implementation of particular small features?
10
SPEAKER_00
Because the features that seem small, I bet some of them might be like really, require really big design decisions.
10
SPEAKER_01
Well, so, I mean, let me give you another example of this.
10
SPEAKER_01
Python has a feature called Async Await.
10
SPEAKER_01
So it s a new feature, I mean, in the long arc of Python history, it s a relatively new feature, right, that allows way more expressive asynchronous programming.
10
SPEAKER_01
Again, this is a, Python s a beautiful thing, and they did things that are great for Mojo for completely different reasons.
10
SPEAKER_01
The reason that Async Await got added to Python, as far as I know, is because Python doesn t support threads.
10
SPEAKER_01
Okay, and so Python doesn t support threads, but you want to work with networking and other things like that that can block.
10
SPEAKER_01
I mean, Python does support threads, it s just not its strength.
10
SPEAKER_01
And so they added this feature called async await.
10
SPEAKER_01
It s also seen in other languages like Swift and JavaScript and many other places as well.
10
SPEAKER_01
Async await in Mojo is amazing, because we have a high-performance heterogeneous compute runtime underneath the covers that then allows non-blocking IOs so you get full use of your accelerator.
10
SPEAKER_01
That s huge, it turns out.
10
SPEAKER_01
It s actually really an important part of fully utilizing the machine.
10
SPEAKER_01
You talk about design discussions.
10
SPEAKER_01
That took a lot of discussions, right?
10
SPEAKER_01
And it probably will require more iteration.
10
SPEAKER_01
And so my philosophy with Mojo is that we have a small team of really good people that are pushing forward, and they re very good at
10
SPEAKER_01
the extremely deep, knowing how the compiler and runtime and all the low-level stuff works together.
10
SPEAKER_01
But they re not perfect.
10
SPEAKER_01
Same thing as the Swift team.
10
SPEAKER_01
And this is where one of the reasons we released Mojo much earlier is so we can get feedback.
10
SPEAKER_01
And we ve already renamed a keyword due to community feedback.
10
SPEAKER_01
Which one?
10
SPEAKER_01
We use an ampersand and now it s named in-out.
10
SPEAKER_01
We re not renaming existing Python keywords because that breaks compatibility.
10
SPEAKER_01
We re renaming things we re adding and making sure that they are designed well, we get usage experience, we iterate and work with the community.
10
SPEAKER_01
Because again, if you scale something really fast and everybody writes all their code and they start using it in production, then it s impossible to change.
10
SPEAKER_01
So you want to learn from people, you want to iterate and work on that early on.
10
SPEAKER_01
This is where design discussions, it s actually quite important.
10
SPEAKER_00
Could you incorporate an emoji into the language, into the main language?
10
SPEAKER_00
I could.
10
SPEAKER_00
Do you have a favorite one?
10
SPEAKER_00
Well, I really like, in terms of humor, I like rawful, whatever, rolling on the floor laughing.
10
SPEAKER_00
So that could be like a, what would that be the use case for that?
10
SPEAKER_00
Like throw an exception of some sort?
10
SPEAKER_01
You should totally file a feature request.
10
SPEAKER_00
Or maybe a heart one.
10
SPEAKER_00
It has to be a heart one.
10
SPEAKER_01
People have told me that I m insane.
10
SPEAKER_01
So this is, this is, this is, I m liking this.
10
SPEAKER_00
I m gonna use the viral nature of the internet to actually get this passed.
10
SPEAKER_01
I mean, it s funny, you come back to the flame emoji file extension, right?
10
SPEAKER_01
We have the option to use the flame emoji, which just even that concept caused, for example, the people at GitHub to say, now I ve seen everything.
10
SPEAKER_00
Yeah, there s something, it s reinvigorating.
10
SPEAKER_00
It s like, oh, that s possible.
10
SPEAKER_00
That s really cool.
10
SPEAKER_00
For some reason, that makes everything else look really exciting.
10
SPEAKER_01
I think the world is ready for this stuff, right?
10
SPEAKER_01
And so when we have a package manager, we ll clearly have to innovate by having the compiled package thing be the little box with the bow on it, right?
10
SPEAKER_01
I mean, it has to be done.
10
SPEAKER_00
It has to be done.
10
SPEAKER_00
Is there some stuff on the roadmap that you particularly
10
SPEAKER_00
stressed about or excited about that you re thinking about a lot?
10
SPEAKER_01
I mean, as a today snapshot, which will be obsolete tomorrow, the lifetime stuff is really exciting.
10
SPEAKER_01
And so lifetimes give you safe references to memory without dangling pointers.
10
SPEAKER_01
And so this has been done in languages like Rust before.
10
SPEAKER_01
And so we have a new approach, which is really cool.
10
SPEAKER_01
I m very excited about that.
10
SPEAKER_01
That ll be
10
SPEAKER_01
out to the community very soon.
10
SPEAKER_01
The traits feature is really a big deal, and so that s blocking a lot of API design, and so there s that.
10
SPEAKER_01
I think that s really exciting.
10
SPEAKER_01
A lot of it is these kind of table stakes features.
10
SPEAKER_01
One of the things that is, again, also lessons learned with Swift
10
SPEAKER_01
is that programmers in general like to add syntactic sugar.
10
SPEAKER_01
And so it s like, oh, well this annoying thing, like in Python, you have to spell underbar, underbar, add.
10
SPEAKER_01
Why can t I just use plus?
10
SPEAKER_01
Def plus, come on, why can t I just do that, right?
10
SPEAKER_01
And so trivial bit of syntactic sugar, it makes sense, it s beautiful, it s obvious.
10
SPEAKER_01
We re trying not to do that.
10
SPEAKER_01
And so for two different reasons, one of which is that, again, lesson learned with Swift, Swift has a lot of syntactic sugar, which may be a good thing, maybe not, I don t know.
10
SPEAKER_01
But because it s such an easy and addictive thing to do, sugar, like make sure blood get crazy, right?
10
SPEAKER_01
Like the community will really dig into that and want to do a lot of that.
10
SPEAKER_01
And I think it s very distracting from building the core abstractions.
10
SPEAKER_01
The second is we want to be a good member of the Python community.
10
SPEAKER_01
Right, and so we wanna work with the broader Python community and yeah, we re pushing forward a bunch of systems programming features and we need to build them out to understand them.
10
SPEAKER_01
But once we get a long ways forward, I wanna make sure that we go back to the Python community and say, okay, let s do some design reviews.
10
SPEAKER_01
Let s actually talk about this stuff.
10
SPEAKER_01
Let s figure out how we want this stuff all to work together.
10
SPEAKER_01
And syntactic sugar just makes all that more complicated, so.
10
SPEAKER_00
And yeah, list comprehensions are yet to be implemented.
10
SPEAKER_00
And my favorite, I mean, dictionaries.
10
SPEAKER_01
Yeah, there s some basic stuff.
10
0.1.
10
SPEAKER_01
0.1.
10
SPEAKER_01
Yeah, but nonetheless, it s actually still quite interesting and useful.
10
SPEAKER_00
As you mentioned, modular is very new.
10
SPEAKER_00
Mojo is very new.
10
SPEAKER_00
It s a relatively small team that s building up this gigantic stack, this incredible stack that s going to perhaps define the future of development of our AI overlords.
10
SPEAKER_01
We just hope it will be useful.
10
SPEAKER_00
As do all of us.
10
SPEAKER_00
So what have you learned from this process of building up a team?
10
SPEAKER_00
Maybe one question is how do you hire
10
SPEAKER_00
Great programmers, great people that operate in this compiler, hardware, machine learning, software, interface design space, and maybe are a little bit fluid in what they can do.
10
SPEAKER_01
Language design too.
10
SPEAKER_01
So building a company is just as interesting in different ways as building
10
SPEAKER_01
a language, like different skill sets, different things, but super interesting.
10
SPEAKER_01
And I ve built a lot of teams in a lot of different places.
10
SPEAKER_01
If you zoom in from the big problem into recruiting, well, so here s our problem.
10
SPEAKER_01
Okay.
10
SPEAKER_01
I ll just, I ll be very straightforward about this.
10
SPEAKER_01
We started modular with a lot of conviction about, we understand the problems, we understand the customer pain points.
10
SPEAKER_01
We need to work backwards from the
11
SPEAKER_00
suffering in the industry and if we solve those problems we think it ll be useful for people.
11
SPEAKER_00
But the problem is, is that the people we need to hire, as you say, are all these super specialized people that have jobs at big tech worlds, right?
11
SPEAKER_00
And I don t think we have product market fit in the way that a normal startup does, or we don t have product market fit challenges, because right now, everybody s using AI, and so many of them are suffering, and they want help.
11
SPEAKER_00
And so, again, we start with strong conviction.
11
SPEAKER_00
Again, you have to hire and recruit the best, and the best all have jobs.
11
SPEAKER_00
And so what we ve done is we ve said, okay, well, let s build an amazing culture.
11
SPEAKER_00
start with that.
11
SPEAKER_00
That s usually not something a company starts with.
11
SPEAKER_00
Usually you hire a bunch of people, and then people start fighting, and it turns into a gigantic mess, and then you try to figure out how to improve your culture later.
11
SPEAKER_00
My co-founder, Tim, in particular, is super passionate about making sure that that s right, and we ve spent a lot of time early on to make sure that we can scale.
11
SPEAKER_01
Can you comment, sorry, before we get to the second, what makes for a good culture?
11
SPEAKER_00
So, I mean, there s many different cultures, and I have learned many things from many different people.
11
SPEAKER_01
Yeah, you worked with several very unique, almost famously unique cultures.
11
SPEAKER_00
And some of them I learned what to do, and some of them I learned what not to do.
11
SPEAKER_00
And so, we want an inclusive culture.
11
SPEAKER_00
I believe in amazing people working together.
11
SPEAKER_00
And so, I ve seen cultures where people, you have amazing people and they re fighting each other.
11
SPEAKER_00
I see amazing people and they re told what to do.
11
SPEAKER_00
Like, doubt shalt line up and do what I say.
11
SPEAKER_00
It doesn t matter if it s the right thing, do it, right?
11
SPEAKER_00
And neither of these is the, and I ve seen people that have no direction, they re just kind of floating in different places.
11
SPEAKER_00
And they want to be amazing, they just don t know how.
11
SPEAKER_00
And so a lot of it starts with have a clear vision.
11
SPEAKER_00
And so we have a clear vision of what we re doing.
11
SPEAKER_00
And so I kind of grew up at Apple in my engineering life, right?
11
SPEAKER_00
And so a lot of the Apple DNA rubbed off on me.
11
SPEAKER_00
My co-founder, Tim, also is like a strong product guy.
11
SPEAKER_00
And so what we learned is, you know, I was taught at Apple that you don t work from building cool technology.
11
SPEAKER_00
You don t work from like, come up with a cool product and think about the features you ll have in the big check boxes and stuff like this.
11
SPEAKER_00
Because if you go talk to customers, they don t actually care about your product.
11
SPEAKER_00
They don t care about your technology.
11
SPEAKER_00
What they care about is their problems, right?
11
SPEAKER_00
And if your product can help solve their problems, well, hey, they might be interested in that, right?
11
SPEAKER_00
And so if you speak to them about their problems, if you understand and you have compassion, you understand what people are working with, then you can work backwards to building an amazing product.
11
SPEAKER_00
So the vision starts by defining the problem.
11
SPEAKER_00
And then you can work backwards in solving technology.
11
SPEAKER_00
And at Apple, like it s, I think, pretty famously said that, you know, for every, you know, there s 100 no s for every yes.
11
SPEAKER_00
I would refine that to say that there s 100 not yet s for every yes.
11
SPEAKER_00
But famously, if you go back to the iPhone, for example, right, the iPhone 1, I mean, many people laughed at it because it didn t have 3G, it didn t have copy and paste.
11
SPEAKER_00
Right, and then a year later, okay, finally, it has 3G, but it still doesn t have copy and paste, it s a joke.
11
SPEAKER_00
Nobody will ever use this product, blah, blah, blah, blah, blah, blah, blah, right?
11
SPEAKER_00
Well, year three, it had copy and paste, and people stopped talking about it, right?
11
SPEAKER_00
And so being laser-focused and having conviction and understanding what the core problems are and giving the team the space to be able to build the right tech is really important.
11
SPEAKER_00
Also, I mean, you come back to recruiting,
11
SPEAKER_00
You have to pay well, right?
11
SPEAKER_00
So we have to pay industry-leading salaries and have good benefits and things like this.
11
SPEAKER_00
That s a big piece.
11
SPEAKER_00
We re a remote-first company, and so we have to... So remote-first has a very strong set of pros and cons.
11
SPEAKER_00
On the one hand, you can hire people from wherever they are, and you can attract amazing talent, even if they live in strange places or unusual places.
11
SPEAKER_00
On the other hand, you have time zones.
11
SPEAKER_00
On the other hand, you have, like,
11
SPEAKER_00
Everybody on the internet will fight if they don t understand each other, and so we ve had to learn how to have a system where we actually fly people in, and we get the whole company together periodically, and then we get work groups together, and we plan and execute together.
11
SPEAKER_01
And there s like an intimacy to the in-person brainstorming that I guess you lose, but maybe you don t.
11
SPEAKER_01
Maybe if you get to know each other well and you trust each other, maybe you can do that.
11
SPEAKER_00
Well, so when the pandemic first hit, I mean, I m curious about your experience too.
11
SPEAKER_00
The first thing I missed was having whiteboards.
11
SPEAKER_00
Yeah.
11
SPEAKER_00
Right?
11
SPEAKER_00
Those design discussions were like, I can high intensity work through things, get things done, work through the problem of the day, understand where you re on, figure out and solve the problem and move forward.
11
SPEAKER_00
Yeah.
11
SPEAKER_00
But we figured out ways to work around that.
11
SPEAKER_00
now with you know all these uh screen sharing and other things like that that we do the thing i miss now is sitting down a lunch table with the team yeah the spontaneous things like those the the coffee the coffee bar things and the and the bumping into each other and getting to know people outside of the transactional solve a problem over zoom
11
SPEAKER_01
And I think there s just a lot of stuff that, I m not an expert at this, I don t know who is, hopefully there s some people, but there s stuff that somehow is missing on Zoom.
11
SPEAKER_01
Even with the whiteboard, if you look at that, if you have a room with one person at the whiteboard and there s like three other people at a table,
11
SPEAKER_01
There s a, first of all, there s a social aspect to that where you re just shooting the shit a little bit, almost like.
11
SPEAKER_00
Yeah, it s people just kind of coming in and.
11
SPEAKER_01
Yeah, that, but also while, like it s a breakout discussion that happens for like seconds at a time, maybe an inside joke, or it s like this interesting dynamic that happens that s zoom.
11
SPEAKER_00
And you re bonding, yeah.
11
SPEAKER_01
You re bonding, you re bonding, but through that bonding,
11
SPEAKER_01
you get the excitement, there s certain ideas that are like complete bullshit and you ll see that in the faces of others that you won t see necessarily on Zoom.
11
SPEAKER_01
It feels like that should be possible to do without being in person.
11
SPEAKER_00
Well, I mean, being in person is a very different thing.
11
SPEAKER_00
It s worth it, but you can t always do it.
11
SPEAKER_00
And so, again, we re still learning.
11
SPEAKER_00
We re also learning, like, humanity with this new reality, right?
11
SPEAKER_00
But what we found is that getting people together, whether it be a team or the whole company or whatever,
11
SPEAKER_00
is worth the expense because people work together and are happier after that.
11
SPEAKER_00
There s a massive period of time where you go out and things start getting frayed, pull people together, and then you realize that we re all working together, we see things the same way, we work through the disagreement or the misunderstanding, we re talking across each other, and then you work much better together.
11
SPEAKER_00
And so things like that I think are really quite important.
11
SPEAKER_01
What about people that are specialized in very different aspects of the stack working together?
11
SPEAKER_01
What are some interesting challenges there?
11
SPEAKER_00
Yeah, well, so I mean, there s lots of interesting people, as you can tell.
11
SPEAKER_00
I m hard to deal with, too.
11
SPEAKER_00
You re one of the most lovable people.
11
SPEAKER_00
So there s different philosophies in building teams.
11
SPEAKER_00
For me, and so some people say, hire 10x programmers, and that s the only thing, whatever that means, right?
11
SPEAKER_00
What I believe in is building well-balanced teams.
11
SPEAKER_00
Teams that have people that are different in them, like if you have all generals and no troops, or all troops and no generals, or you have all people that think in one way and not the other way, what you get is you get a very biased and skewed and weird situation where people end up being unhappy.
11
SPEAKER_00
And so what I like to do is I like to build teams of people where they re not all the same.
11
SPEAKER_00
You know, we do have teams that are focused on like runtime or compiler, GPU or whatever the speciality is, but people bring a different take and have a different perspective.
11
SPEAKER_00
And I look for people that complement each other.
11
SPEAKER_00
And particularly if you look at leadership teams and things like this, you don t want everybody thinking the same way.
11
SPEAKER_00
You want people bringing different perspectives and experiences.
11
SPEAKER_00
And so I think that s really important.
11
SPEAKER_01
That s team, but what about building a company as ambitious as Modular?
11
SPEAKER_01
What are some interesting lessons there?
11
SPEAKER_00
Oh, I mean, so many.
11
SPEAKER_00
One of the things I love about... Okay, so Modular is the first company I built from scratch.
11
SPEAKER_00
One of the first things that was profound was I m not cleaning up somebody else s mess.
11
SPEAKER_00
And so if you look at- That s liberating to some degree.
11
SPEAKER_00
It s super liberating.
11
SPEAKER_00
And also, many of the projects I ve built in the past have not been core to the product of the company.
11
SPEAKER_00
Swift is not Apple s product, right?
11
SPEAKER_00
MLIR is not Google s revenue machine or whatever, right?
11
SPEAKER_00
It s important, but it s like working on the accounting software for, you know, the retail giant or something, right?
11
SPEAKER_00
It s like enabling infrastructure and technology.
11
SPEAKER_00
And so at Modular, the tech we re building is,
11
SPEAKER_00
here to solve people s problems, like it is directly the thing that we re giving to people.
11
SPEAKER_00
And so this is a really big difference.
11
SPEAKER_00
And what it means for me as a leader, but also for many of our engineers is they re working on the thing that matters.
11
SPEAKER_00
And that s actually pretty, I mean, again, for compiler people and things like that, that s usually not the case, right?
11
SPEAKER_00
And so that s also pretty exciting and quite nice.
11
SPEAKER_00
But one of the ways that this manifests is it makes it easier to make decisions.
11
SPEAKER_00
And so one of the challenges I ve had in other
11
SPEAKER_00
is it s like, okay, well, community matters somehow for the goodness of the world, or open source matters, theoretically, but I don t wanna pay for a T-shirt.
11
SPEAKER_00
Or some swag, like, well, T-shirts cost 10 bucks each, you can have 100 T-shirts for $1,000, to a megacorp, $1,000 is uncountably, can t count that low.
11
SPEAKER_00
But justifying it and getting a T-shirt, by the way, if you d like a T-shirt,
11
SPEAKER_01
I would 100% like a T-shirt, are you joking?
11
SPEAKER_00
You can have a Fire Emoji T-shirt.
11
SPEAKER_00
I will treasure this.
11
SPEAKER_00
Is that a good thing?
11
SPEAKER_01
I will pass it down to my grandchildren.
11
SPEAKER_00
And so, you know, it s very liberating to be able to decide, I think that Lex should have a T-shirt.
11
SPEAKER_00
Right?
11
SPEAKER_00
And it becomes very simple, because I like Lex.
11
SPEAKER_01
This is awesome.
11
SPEAKER_01
So...
11
SPEAKER_01
I have to ask you about the, one of the interesting developments with large language models is that they re able to generate code recently really well.
11
SPEAKER_01
To a degree that maybe a,
11
SPEAKER_01
I don t know if you understand, but I struggle to understand because it forces me to ask questions about the nature of programming, of the nature of thought, because the language models are able to predict the kind of code I was about to write so well that it makes me wonder how unique my brain is and where the valuable ideas actually come from.
11
SPEAKER_01
How much do I contribute?
11
SPEAKER_01
in terms of ingenuity, innovation, to code, I write, or design, and that kind of stuff.
11
SPEAKER_01
When you stand on the shoulders of giants, are you really doing anything?
11
SPEAKER_01
And what LLMs are helping you do is they help you stand on the shoulders of giants in your program.
11
SPEAKER_01
There s mistakes, they re interesting that you learn from, but I just, it would love to get your opinion first high level of what you think about this impact of large language models when they do program synthesis, when they generate code.
11
SPEAKER_00
Yeah, well, so I don t know where it all goes.
11
SPEAKER_00
I m an optimist, and I m a human optimist.
11
SPEAKER_00
I think that things I ve seen are that a lot of the LLMs are really good at crushing leak code projects, and they can reverse the link list like crazy.
11
SPEAKER_00
Well, it turns out there s a lot of
11
SPEAKER_00
instances of that on the internet, and it s a pretty stock thing, and so if you want to see standard questions answered, LLMs can memorize all the answers, and that can be amazing.
11
SPEAKER_00
And also, they do generalize out from that, and so there is good work on that, but I think that, in my experience, building things, building something, like you talk about Mojo, or you talk about these things, or you talk about building an applied solution to a problem, it s also about working with people.
11
SPEAKER_00
It s about understanding the problem.
11
SPEAKER_00
What is the product that you want to build?
11
SPEAKER_00
What are the use case?
11
SPEAKER_00
What are the customers?
11
SPEAKER_00
You can t just go survey all the customers because they ll tell you that they want a faster horse.
11
SPEAKER_00
Maybe they need a car, right?
11
SPEAKER_00
And so a lot of it comes into, you know, I don t feel like we have to compete with LLMs.
11
SPEAKER_00
I think they ll help automate a ton of the mechanical stuff out of the way.
11
SPEAKER_00
And just like, you know, I think we all try to scale through delegation and things like this.
11
SPEAKER_00
Delegating wrote things to an LLM I think is extremely valuable.
11
SPEAKER_00
approach that will help us all scale and be more productive.
11
SPEAKER_00
But I think it s a fascinating companion.
11
SPEAKER_00
But I d say I don t think that means that we re gonna be done with coding.
11
SPEAKER_00
Sure, sure.
11
SPEAKER_00
But there s power in it as a companion.
11
SPEAKER_01
Yeah, absolutely.
11
SPEAKER_01
And from there, I would love to zoom in onto Mojo a little bit.
11
SPEAKER_01
Do you think about that?
11
SPEAKER_01
Do you think about LLMs generating Mojo code and helping sort of,
11
SPEAKER_01
Like, when you design a new programming language, it almost seems like, man, it would be nice to sort of, almost as a way to learn how I m supposed to use this thing for them to be trained on some of the mojo code.
11
SPEAKER_00
So I do lead an AI company, so maybe there will be a mojo LLM at some point.
11
SPEAKER_00
But if your question is like, how do we make a language to be suitable for LLMs, I think that the,
11
SPEAKER_00
I think the cool thing about LLMs is you don t have to.
11
SPEAKER_00
And so if you look at what is English, or any of these other terrible languages that we as humans deal with on a continuous basis, they re never designed for machines, and yet they re the intermediate representation, they re the exchange format that we humans use to get stuff done.
11
SPEAKER_00
And so these programming languages, they re an intermediate representation between the human and the computer, or the human and the compiler, roughly, right?
11
SPEAKER_00
And so I think the LLMs will have no problem learning whatever keyword we pick.
11
SPEAKER_01
Maybe the Phi Emoji is gonna... Oh, maybe that s gonna break it, it doesn t tokenize?
11
SPEAKER_01
No, the reverse of that, it will actually enable it, because one of the issues I could see with being a superset of Python is there would be confusion about the gray area.
11
SPEAKER_01
So it would be mixing stuff.
11
SPEAKER_00
Well, I m a human optimist.
11
SPEAKER_00
I m also an LLM optimist.
11
SPEAKER_00
I think that we ll solve that problem.
11
SPEAKER_00
But you look at that and you say, okay, well...
11
SPEAKER_00
reducing the rote thing, right?
11
SPEAKER_00
Turns out compilers are very particular and they really want things, they really want the indentation to be right, they really want the colon to be there on your else or else it ll complain, right?
11
SPEAKER_00
I mean, compilers can do better at this, but LLMs can totally help solve that problem.
11
SPEAKER_00
And so I m very happy about the new predictive coding and copilot type features and things like this because I think it ll all just make us more productive.
11
SPEAKER_01
It s still messy and fuzzy and uncertain, unpredictable.
11
SPEAKER_01
But is there a future you see, given how big of a leap GPT-4 was, where you start to see something like LLMs inside a compiler?
11
SPEAKER_00
Or no?
11
SPEAKER_00
I mean, you could do that.
11
SPEAKER_00
Yeah, absolutely.
11
SPEAKER_00
I mean, I think that d be interesting.
11
SPEAKER_00
Is that wise?
11
SPEAKER_00
Well, I mean it would be very expensive.
11
SPEAKER_00
So compilers run fast and they re very efficient and LLMs are currently very expensive.
11
SPEAKER_00
There s on-device LLMs and there s other things going on and so maybe there s an answer there.
11
SPEAKER_00
I think that one of the things that I haven t seen enough of is that
11
SPEAKER_00
So LLMs to me are amazing when you tap into the creative potential of the hallucinations, right?
11
SPEAKER_00
And so if you re doing creative brainstorming or creative writing or things like that, the hallucinations work in your favor.
11
SPEAKER_00
If you re writing code that has to be correct because you re going to ship it in production, then maybe that s not actually a feature.
11
SPEAKER_00
And so I think that there has been research and there has been work on building algebraic reasoning systems and kind of like figuring out more things that feel like proofs.
11
SPEAKER_00
And so I think that there could be interesting work in terms of building more reliable at scale systems, and that could be interesting.
11
SPEAKER_00
But if you chase that rabbit hole down, the question then becomes, how do you express your intent to the machine?
11
SPEAKER_00
And so maybe you want LLM to provide the spec, but you have a different kind of net that then actually implements the code.
11
SPEAKER_01
Right, so it s to use the documentation and inspiration versus the actual implementation.
11
SPEAKER_00
Yeah, potentially.
11
SPEAKER_01
Since a successful modular will be the thing that runs, I say so jokingly, our AI overlords, but AI systems that are used across, I know it s a cliche term, but in and out of things, so across.
11
SPEAKER_00
So I ll joke and say like, AGI should be written in Mojo.
11
SPEAKER_01
Yeah, AGI should be written in Mojo.
11
SPEAKER_01
You re joking, but it s also possible that it s not a joke.
11
SPEAKER_01
that a lot of the ideas behind Mojars seems like the natural set of ideas that would enable at scale
12
SPEAKER_01
training and inference of AI systems.
12
SPEAKER_01
So I just have to ask you about the big philosophical question about human civilization.
12
SPEAKER_01
So folks like Eliezer Yudkowsky are really concerned about the threat of AI.
12
SPEAKER_01
Do you think about the good,
12
SPEAKER_01
and the bad that can happen at scale deployment of AI systems.
12
SPEAKER_00
Well, so I ve thought a lot about it.
12
SPEAKER_00
And there s a lot of different parts to this problem, everything from job displacement to Skynet, things like this.
12
SPEAKER_00
And so you can zoom into subparts of this problem.
12
SPEAKER_00
I m not super optimistic about AGI being solved next year.
12
SPEAKER_00
I don t think that s gonna happen personally.
12
SPEAKER_01
So you have a kind of zen-like calm about, is there s a nervousness because the leap of GPT-4 seemed so big.
12
SPEAKER_00
Sure, it s huge.
12
SPEAKER_01
It s like we re almost, there s some kind of transitionary period.
12
SPEAKER_01
You re thinking.
12
SPEAKER_00
Well, so, I mean, there s a couple of things going on there.
12
SPEAKER_00
One is, I m sure GPT-5 and 7 and 19 will be also huge leaps.
12
SPEAKER_00
They re also getting much more expensive to run.
12
SPEAKER_00
And so there may be a limiting function in terms of just expense on the one hand, and train, like that could be a limiter that slows things down.
12
SPEAKER_00
But I think the bigger limiter is outside of like,
12
SPEAKER_00
Skynet takes over, and I don t spend any time thinking about that, because if Skynet takes over and kills us all, then I ll be dead, so I don t worry about that.
12
SPEAKER_00
So, you know, I mean, that s just, okay, if other things worry about, I ll just focus on, I ll focus and not worry about that one.
12
SPEAKER_00
But I think that the other thing I d say is that AI moves quickly, but humans move slowly, and we adapt slowly.
12
SPEAKER_00
And so what I expect to happen is, just like any technology diffusion,
12
SPEAKER_00
like the promise and then the application takes time to roll out.
12
SPEAKER_00
And so I think that I m not even too worried about autonomous cars defining away all the taxi drivers.
12
SPEAKER_00
Remember, autonomy was supposed to be solved by 2020?
12
SPEAKER_00
Yeah, boy, do I remember.
12
SPEAKER_00
And so I think that on the one hand, we can see amazing progress.
12
SPEAKER_00
But on the other hand, we can see that the reality is a little bit more complicated, and it may take longer to roll out than you might expect.
12
SPEAKER_01
Well, that s in the physical space.
12
SPEAKER_01
I do think in the digital space is the stuff that s built on top of LLMs that runs the millions of apps that could be built on top of them and that could be run on millions of devices, millions of types of devices.
12
SPEAKER_01
I just think that the rapid effect it has on human civilization could be truly transformative to it.
12
SPEAKER_00
Yeah, we don t even know.
12
SPEAKER_00
And there I think it depends on are you an optimist or a pessimist, or a masochist.
12
SPEAKER_00
Just to clarify, optimist about human civilization.
12
SPEAKER_00
Me too.
12
SPEAKER_00
And so I look at that as saying, okay, cool, what will AI do?
12
SPEAKER_00
And so some people say, oh my God, is it gonna destroy us all, how do we prevent that?
12
SPEAKER_00
I kinda look at it from a, is it gonna unlock us all?
12
SPEAKER_00
You talk about coding, is it gonna make so I don t have to do all the repetitive stuff?
12
SPEAKER_00
Well, suddenly that s a very optimistic way to look at it, and you look at what a lot of these technologies have done to improve our lives, and I want that to go faster.
12
SPEAKER_00
So.
12
SPEAKER_01
What do you think the future of programming looks like in the next 10, 20, 30, 50 years?
12
SPEAKER_01
With LLMs and with Mojo, with Modular, like your vision for devices, the hardware to the compilers, to the different stacks of software.
12
SPEAKER_00
Well, so what I want, I mean, coming back to my arch nemesis, right?
12
SPEAKER_00
It s complexity, right?
12
SPEAKER_00
So again, me being the optimist, if we drive down complexity, we can make these tools, these technologies, these cool hardware widgets accessible to way more people.
12
SPEAKER_00
And so what I d love to see is more personalized experiences, more things, the research getting into production instead of being lost at NeurIPS, right?
12
SPEAKER_00
And like these things that impact people s lives by entering products.
12
SPEAKER_00
And so one of the things that I m a little bit concerned about is right now, the big companies are investing huge amounts of money and are driving the top line of AI capability forward really quickly.
12
SPEAKER_00
But if it means that you have to have $100 million to train a model or more, $100 billion, well, that s going to make it very concentrated with very few people in the world that can actually do this stuff.
12
SPEAKER_00
I d much rather see lots of people across the industry
12
SPEAKER_00
be able to participate and use this, right?
12
SPEAKER_00
And you look at this, you know, I mean, a lot of great research has been done in the health world and looking at, like, detecting pathologies and doing radiology with AI and, like, doing all these things.
12
SPEAKER_00
Well, the problem today is that to deploy and build these systems, you have to be an expert in radiology and an expert in AI.
12
SPEAKER_00
And if we can break down the barriers so that more people can use AI techniques, and it s more like programming Python, which roughly everybody can do if they want to, right, then I think that we ll get a lot more practical application of these techniques and a lot more nichier, cool, but narrower demands.
12
SPEAKER_00
And I think that s gonna be really cool.
12
SPEAKER_01
Do you think we ll have more or less programmers in the world than now?
12
SPEAKER_00
Well, so I think we ll have more programmers, but they may not consider themselves to be programmers.
12
SPEAKER_00
That d be a different name for it.
12
SPEAKER_00
Right.
12
SPEAKER_00
I mean, do you consider somebody that uses, you know, I think that arguably the most popular programming language is Excel.
12
SPEAKER_00
Yeah.
12
SPEAKER_00
right?
12
SPEAKER_00
And so do they consider themselves to be programmers?
12
SPEAKER_00
Maybe not.
12
SPEAKER_00
I mean, some of them make crazy macros and stuff like that.
12
SPEAKER_00
But what you mentioned, Steve Jobs, it s the bicycle for the mind that allows you to go faster, right?
12
SPEAKER_00
And so I think that as we look forward, right, what is AI?
12
SPEAKER_00
I look at it as
12
SPEAKER_00
hopefully, a new programming paradigm.
12
SPEAKER_00
It s like object-oriented programming, right?
12
SPEAKER_00
If you want to write a cat detector, you don t use for loops.
12
SPEAKER_00
It turns out that s not the right tool for the job, right?
12
SPEAKER_00
And so right now, unfortunately, because, I mean, it s not unfortunate, but it s just kind of where things are, AI is this weird, different thing that s not integrated into programming languages and normal tool chains, and all the technology is really weird and doesn t work right, and you have to babysit it, and every time you switch hardware, it s different.
12
SPEAKER_00
It shouldn t be that way.
12
SPEAKER_00
When you change that, when you fix that, suddenly, again, the tools and technologies can be way easier to use.
12
SPEAKER_00
You can start using them for many more things.
12
SPEAKER_00
And so that s what I would be excited about.
12
SPEAKER_01
What kind of advice could you give to somebody in high school right now or maybe early college who s curious about programming?
12
SPEAKER_01
and feeling like the world is changing really quickly here.
12
SPEAKER_01
What kind of stuff to learn?
12
SPEAKER_01
What kind of stuff to work on?
12
SPEAKER_01
Should they finish college?
12
SPEAKER_01
Should they go work at a company?
12
SPEAKER_01
Should they build a thing?
12
SPEAKER_01
What do you think?
12
SPEAKER_00
Well, so, I mean, one of the things I d say is that you ll be most successful if you work on something you re excited by.
12
SPEAKER_00
And so don t get the book and read the book
12
SPEAKER_00
cover to cover and study and memorize and recite and flashcard and go build something.
12
SPEAKER_00
Like go solve a problem, go build the thing that you want to exist, go build an app, go build, train a model, like go build something and actually use it and set a goal for yourself.
12
SPEAKER_00
And if you do that, then you ll...
12
SPEAKER_00
there s a success, there s the adrenaline rush, there s the achievement, there s the unlock that I think is where if you keep setting goals and you keep doing things and building things, learning by building is really powerful.
12
SPEAKER_00
In terms of career advice, I mean, everybody s different, it s very hard to give generalized advice.
12
SPEAKER_00
I ll speak as a compiler nerd.
12
SPEAKER_00
If everybody s going left, sometimes it s pretty cool to go right.
12
SPEAKER_00
And so just because everybody s doing a thing, it doesn t mean,
12
SPEAKER_00
you have to do the same thing and follow the herd.
12
SPEAKER_00
In fact, I think that sometimes the most exciting paths through life lead to being curious about things that nobody else actually focuses on, right?
12
SPEAKER_00
And it turns out that understanding deeply parts of the problem that people want to take for granted makes you extremely valuable and specialized in ways that
12
SPEAKER_00
the herd is not, and so, again, I mean, there s lots of rooms for specialization, lots of rooms for generalists, there s lots of room for different kinds and parts of the problem, but I think that it s, you know, just because everybody s doing one thing doesn t mean you should necessarily do it.
12
SPEAKER_01
And now the herd is using Python, so if you wanna be a rebel, go check out Mojo.
12
SPEAKER_01
and help Chris and the rest of the world fight the archnemesis of complexity, because simple is beautiful.
12
SPEAKER_00
There you go.
12
SPEAKER_01
Chris, you re an incredible person.
12
SPEAKER_01
You ve been so kind to me ever since we met.
12
SPEAKER_01
You ve been extremely supportive.
12
SPEAKER_01
I m forever grateful for that.
12
SPEAKER_01
Thank you for being who you are, for being legit, for being kind, for fighting this really interesting problem of how to make AI accessible to a huge number of people, huge number of devices.
12
SPEAKER_00
Yeah, well, so Lex, you re a pretty special person too, right?
12
SPEAKER_00
And so I think that one of the funny things about you is that besides being curious and pretty damn smart, you re actually willing to push on things.
12
SPEAKER_00
And I think that you ve got an agenda to make the world think, which I think is a pretty good agenda.
12
SPEAKER_01
It s a pretty good one.
12
SPEAKER_01
Thank you so much for talking to me, Chris.
12
SPEAKER_00
Yeah, thanks, Lex.
12
SPEAKER_01
Thanks for listening to this conversation with Chris Ladner.
12
SPEAKER_01
To support this podcast, please check out our sponsors in the description.
12
SPEAKER_01
And now let me leave you with some words from Isaac Asimov.
12
SPEAKER_01
I do not fear computers.
12
SPEAKER_01
I fear the lack of them.
12
SPEAKER_01
Thank you for listening and hope to see you next time.