This article records the original content of the speech and summarizes it in Chinese.
Original content
It really makes me cringe listening to all that.
Thank you for that kind introduction, but I hate hearing about myself.
Just as a show, maybe if you could just applaud.
How many of you know who NVIDIA is?
And how many of you know what a GPU is?
Okay good, I don’t have to change my speech.
Ladies and gentlemen, President Rosenbaum, esteemed faculty members, distinguished guests,
proud parents, and above all, the 2024 graduating class of Caltech.
This is a really happy day for you guys, you gotta look more excited.
You know you’re graduating from Caltech.
This is the school of the great Richard Feynman, Linus Pauling, and someone who is very influential
to me and our industry, Carver Mead.
Yeah, this is a very big deal.
Today is a day of immense pride and joy.
It is a dream come true for all of you, but not just for you, because your parents and
families have made countless sacrifices to see you reach this milestone.
So let’s take this moment and congratulate them, thank them, and let them know you love them.
You don’t want to forget that because you don’t know how long you’re going to be living at home.
You want to be super grateful today.
As a proud parent, I really loved it when my kids didn’t move out, and it was great
to see them every day, but now they’ve moved out, it makes me sad.
So hopefully you guys get to spend some time with your parents.
Your journey here is a testament of your character, your determination, willingness to make sacrifices
for your dreams, and you should be proud.
The ability to make sacrifices, endure pain and suffering.
You will need these qualities in life.
You and I share some things in common.
First, both chief scientists of NVIDIA were from Caltech.
And one of the reasons why I’m giving the speech today is because I’m recruiting.
And so I want to tell you that NVIDIA is a really great company.
I’m a very nice boss, universally loved, come work at NVIDIA.
You and I share a passion for science and engineering.
And although we’re separated by about 40 years, we are both at the peaks of our career.
For all of you who have been paying attention to NVIDIA and myself, you know what I mean.
It’s just that in your case, you’ll have many, many more peaks to go.
I just hope that today is not my peak, not the peak.
And so I’m working as hard as ever to make sure that I have many more peaks ahead.
Last year, I was honored to give the commencement address at Taiwan University.
I shared several stories about NVIDIA’s journey and the lessons that we learned that might
be valuable for graduates.
I have to admit that I don’t love giving advice, especially to other people’s children.
And so my advice today will largely be disguised in some stories that I’ve enjoyed and some
life experiences that I’ve enjoyed.
I’m the longest running tech CEO in the world today, I believe.
Over the course of 31 years, I’ve managed not to go out of business, not get bored and
not get fired.
And so I have the great privilege of enjoying a lot of life’s experiences, starting from
NVIDIA from nothing to what it is today.
And so I spoke about the long road of creating CUDA, a programming model, the programming
model that we dedicated over 20 years to invent, and that is revolutionizing computing today.
I spoke about a very quite public, canceled Sega game console project we worked on, and
where intellectual honesty, something that I know Richard Feynman cares very deeply about
and spoke quite often about, where intellectual honesty and humility saved our company and
how a retreat, a strategic retreat, was one of our best strategies.
All of these are counterintuitive lessons that I spoke about at the commencement.
But I encouraged the graduates to engage with AI, the most consequential technology
of our time, and I’ll speak a little bit more about that later, but all of you know
about AI.
It’s hard not to be immersed in it and surrounded by it and a great deal of discussion about
it, and of course, I hope that all of you are using it and playing with it with surprising
results and some magical, some disappointing, and some surprising.
But you have to enjoy it, you have to engage it, because it’s advancing so quickly.
It is the only technology that I’ve known that is advancing on multiple exponentials
at the same time, and so the technology is changing very, very quickly.
So I advised the students at the Taiwan University to run, don’t walk, and engage the AI revolution.
And yet, one year later, it’s incredible how much has changed.
And so today, what I wanted to do is share with you my perspective from my vantage point
of some of the important things that are happening that you’re graduating into.
And these are extraordinary things that are happening that you should have an intuitive
understanding for, because it’s going to matter to you, it’s going to matter to the industry,
and hopefully, you take advantage of the opportunity ahead of you.
The computer industry is transforming from its foundations, literally from studs.
Everything is changing from studs on up, and across every layer and soon, every industry
will also be transformed, and the reason for that is quite obvious,
because computers today are the single most important instrument of knowledge,
and it’s foundational to every single industry and every field of science.
If we are transforming the computer so profoundly, it will, of course,
have implications in every industry, and I’ll talk about that in just a little bit.
And as you enter industry, it’s important you know what’s happening.
Modern computing traces back to the IBM System 360.
That was the architecture manual that I learned from.
It’s an architecture manual that you don’t need to learn from.
A lot better documentation and better descriptions of computers
and architectures has been presented since,
but the System 360 was incredibly important at its time, and in fact,
the basic ideas of the System 360, the architecture of it, the principal ideas
and architecture and strategy of the System 360 are still governing the computer industry today,
and it was introduced a year after my birth.
In the 80s, I was among the first generation of VLSI engineers who learned to design chips
from Mead and Conway’s landmark textbook, and I’m not sure if it’s still being taught here.
It should be in the introduction of VLSI systems.
Based on Carver Mead’s pioneering work here at Caltech on chip design methodologies
and textbook that revolutionized IC design, and it enabled our generation
to design supergiant chips and ultimately the CPU.
The CPU led to exponential growth in computing.
The performance, the incredible technology advances that’s called Moore’s Law,
fueled the information technology revolution.
The industrial revolution that we are part of, that my generation was part of,
saw the mass production of something the world had never seen before,
the mass production of something that was invisible, easy to copy,
the mass production of software, and it led to a $3 trillion industry.
When I sat where you sat, the IT industry was minuscule, and the concept
that you could make money selling software was a fantasy, and yet today,
it’s one of the most important commodities, most important technologies
and product creations that our industry produces.
However, the limits of DENARD scaling, of transistor scaling,
and instruction-level parallelism have slowed CPU performance,
and the slowed CPU performance gains is happening at a time
when computing demand continues to grow exponentially.
This exponentially growing gap between demand of computing and the capabilities
of computers, if not addressed, computing energy consumption
and cost inflation would eventually stifle every industry.
We see very clear signs of computing inflation as we speak, and after two decades
of advancing NVIDIA’s CUDA, NVIDIA’s accelerated computing, offers a path forward.
That’s the reason why I’m here, because finally, the industry realized
of the incredible effectiveness of accelerated computing at precisely the time
that we’re witnessing computing inflation after several decades.
By offloading time-consuming algorithms to a GPU that specializes in parallel processing,
we routinely achieve 10, 100, sometimes 1,000-fold speedups saving money, cost, and energy.
We now accelerate application domains from computer graphics, ray tracing, of course,
to gene sequencing, scientific computing, astronomy, quantum circuit simulations,
SQL data processing, and even pandas, data science.
Accelerated computing has reached a tipping point.
That is our first great contribution to the computer industry, our first great contribution
to society, accelerated computing.
It now gives us a path forward for sustainable computing where cost will continue to decline
as computing requirement continues to grow.
A hundred-fold, a hundred-fold of anything in time or cost or energy savings
that accelerated computing opened surely would trigger a new development somewhere else.
We just didn’t know what it was until deep learning came to our consciousness.
A whole new world of computing emerged.
Jeff Hinton, Alice Krzyzewski, and Ilya Suskover used NVIDIA CUDA GPUs to train AlexNet
and shocked the computer vision community by winning the 2012 ImageNet Challenge.
This was the big moment, the big bang of deep learning, a pivotal moment
that marked the beginning of the AI revolution.
Our decisions after AlexNet transformed our company is something that’s worth taking note of.
Our decisions after AlexNet transformed our company and likely everything else.
We saw the potential of deep learning and believed, just believed through principle thinking,
believed through our own analysis of the scalability of deep learning.
We believed the approach could learn other valuable functions.
That maybe deep learning is a universal function learner and how many problems are difficult
or impossible to express using fundamental first principles.
And so when we saw this, when we saw this, we thought this is a technology we really have
to pay attention to because its limits are potentially only limited by model and data scale.
However, there were challenges at the time.
This is 2012, shortly after 2012.
How could we explore the limits of deep learning without having
to build these massive GPU clusters?
At the time we were a rather small company,
building these massive GPU clusters could cost hundreds and hundreds of millions of dollars.
And if we didn’t though, there was no assurance that it would be effective if we scaled.
However, no one knew how far deep learning could scale.
And if we didn’t build it, we’d never know.
This is one of those, if you build it, will they come?
Our logic is if we don’t build it, they can’t come.
And so we dedicated ourselves based on our first principle beliefs and our analysis.
And we got ourselves to the point where we believed this was going to be so effective.
And when the company believes something, we should go act on it.
So we dove deep into deep learning.
And over the next decade, systematically reinvented everything.
We reinvented every computing layer.
Starting with the GPU itself.
The invention of the modern GPU, which is very different than the GPU of the past,
that we invented in the first place.
And we went on to invent just about every other aspect of computing.
The interconnects, the systems, the networking, and of course, software.
We invested billions.
We invested billions into the unknown.
Thousands of engineers for a decade worked on deep learning.
And advancing and scaling deep learning without really knowing how far we could really take
the technology.
We invested billions.
And we designed and built supercomputers to explore the limits of deep learning and AI.
Then in 2016, we announced DGX1, our first AI supercomputer.
And I delivered the first one to a startup in San Francisco.
A startup nobody knew anything about, a group of friends of mine who were working on artificial
intelligence, a company called OpenAI.
In 2022, 10 years after AlexNet, and about a million-fold increase in computing later,
a million-fold.
If you could just imagine, what would it be like if your laptop was a million-fold more
capable?
A million-fold later, OpenAI launched chat GPT, and AI went mainstream.
During this decade, NVIDIA transformed ourselves from a graphics company that many of you probably
first knew us as, that builds GPUs, to now an AI company that builds massive data center
scale supercomputers.
We transformed our company completely.
We also transformed computing completely.
The fundamental way of doing computing today has been radically changed.
The computing stack now uses GPU to process large language models that are trained on
supercomputers, rather than CPUs that are processing instructions written by programmers.
We are now creating software that no humans can write.
We are now creating software that does things that no humans can imagine, even just 10 years
ago.
Computers are now intention-driven, rather than instruction-driven.
Tell a computer what you want, and it will figure it out, it will figure out how.
And like humans, AI applications will understand the mission, reason, plan, and orchestrate
a team of large language models to perform tasks.
Computer applications will do and perform very similar to the way we do things, assemble
teams of experts, use tools, reason and plan, and execute our mission.
Software and what software can do has been completely changed.
Even our industry, as it’s being changed and transformed, created yet another industry,
an industry the world’s never seen before.
An industry is forming right in front of our eyes.
AI’s input and output are tokens.
For all the engineers in the room, you know what I mean.
These are floating-point numbers that embed intelligence.
Companies are now building a new type, a new type of data center that didn’t exist before
that specialized in producing intelligence tokens.
Especially AI factories.
Like AC generators that Nikola Testa invented of the past industrial revolution, we now
have AI token generators, and they will be the factories of a new industrial revolution.
There’s large industries producing energy, electricity.
We now have a large industry producing something invisible called software.
In the future, in the very near future, we’ll have industries that are producing manufacturing
intelligence tokens, AI generators.
A new computing model has emerged, and a new industry has emerged, all because we reasoned
from first principles, formed our belief about the future, and we acted on it.
The next wave of AI is robotics, where AI, in addition to a language model, also has
a physical world model.
We work with hundreds of companies building robots, robotic vehicles, pick-and-place arms,
humanoid robots, and even entire gigantic warehouses that are robotic.
But unlike our AI factory strategy and our experience there, which was really formed
through reasoning and deliberate action, our robotics journey resulted from a series of
setbacks.
As you know, NVIDIA invented the GPU.
This was before we invented AI factories.
Our first great contribution to the computer industry was reinventing computer graphics
through programmable shaders.
We invented the GPU and programmable shading in 2000.
We wanted to integrate GPUs into every computer, and so we started to combine our GPUs with
motherboard chips.
And we launched a fabulous integrated graphics chip at the time for AMD CPUs.
Our chipset business was an instant success.
I think it went from zero to a billion dollars practically overnight.
But then all of a sudden, AMD wanted to control all of the technology in the PC, and we wanted
to stay independent, so they purchased ATI and no longer needed us.
We turned to Intel.
That probably wasn’t a great idea, but we turned to Intel and negotiated a license to
connect to Intel CPUs.
Intel was excited about what we were building and asked us to work on a new computer with
them, which became the first MacBook Air.
Well, Intel saw what happened and decided they didn’t want us to do that anymore, and
so they terminated our agreement.
Well, we pivoted again, and this time we went and licensed ARM, and we built a low-power
SoC, a mobile SoC, the world’s first SoC that was essentially a computer, a full operating
computer, and it was incredible.
Our chip excited Google, and they asked us to work on a new device, which turned out
to have been the Android mobile device.
Well, Qualcomm decided they didn’t want us to do that, and so they didn’t want us to
connect to their modems, and it’s hard to build a mobile device without being connected
to a modem, and there were no other LTE modem companies, so we had to exit the mobile device
market.
Well, this happened practically on a year rhythm, and we would build something, it would
be incredibly successful, generate enormous amounts of excitement, and then one year later
we were kicked out of those markets.
Well, with no more markets to turn to, we decided to build something where we are sure
there are no customers, because one of the things you can definitely guarantee is where
there are no customers, there are also no competitors, and nobody cares about you.
And so we chose a market with no customers, a zero billion dollar market, and it was robotics.
We built the world’s first robotics computer, processing an algorithm nobody understood
at the time called deep learning.
This is over ten years ago now.
Ten years later, I can’t be happier with what we’ve built and the opportunity to create
the next wave of AI.
More importantly, we developed agility and a culture of resilience.
One setback after another, we shook it off and skated to the next opportunity.
Each time, we gained skills and strengthened our character.
We strengthened our corporate character.
Our company is really hard to distract and really hard to discourage, and no setback
that comes our way doesn’t look like an opportunity these days.
Ironically, the robotics computer that we built today doesn’t even need graphics, which
is why our journey started in the first place.
So where we are today tells us something and teaches us something.
The world is uncertain, as Richard Feynman would say, and the world can be unfair and
deal you with tough cards.
Swiftly shake it off.
You’ve apparently been paying too much attention to your books.
Swiftly shake it off.
Come on, that’s pretty clever.
I made myself laugh.
There’s another opportunity out there, or create one.
Let me tell you one more story.
I used to work from one of our international sites for one month each summer.
When our kids were in their teens, we spent a summer in Japan.
Over a weekend, we visited Kyoto and the Silver Temple.
If you haven’t had a chance to go, you must.
It’s renowned for its exquisite moss garden.
The day we visited was quintessential Kyoto summer day, suffocatingly hot and humid, sticky.
Heat is radiating from the ground.
The air was thick, still.
Along with the other tourists, we wandered through the meticulously groomed moss garden.
I noticed the lone gardener.
Now, remember, the moss garden, this is the Silver Temple.
The moss garden is gigantic.
It’s about the size of this courtyard, and it has the collection, the largest collection
of just about, apparently, every species of moss in the world, and just exquisitely maintained.
I noticed the lone gardener squatting, carefully picking at the moss with a bamboo stick.
Bamboo tweezer, and putting it in the bamboo basket.
It’s a bamboo tweezer, and it’s just this one gardener.
The basket looked empty.
For a moment there, I thought he was picking imaginary moss into a pile of imaginary dead
moss.
I walked up to him, and I said, what are you doing?
In his English, he said, I’m picking dead moss.
I’m taking care of my garden.
I said, but your garden is so big.
He responded, I have cared for my garden for 25 years.
I have plenty of time.
That was one of the most profound learnings in my life, and it really taught me something.
This gardener has dedicated himself to his craft and doing his life’s work.
When you do that, you have plenty of time.
I begin each morning, I do every single morning exactly the same way, I begin each morning
by doing my highest priority work first.
I have a very clear priority list, and I start from the highest priority work first.
Before I even get to work, my day is already a success.
I’ve already completed my most important work and can dedicate my day to helping others.
And when people apologize for interrupting me, I always say, I have plenty of time, and
I do.
Graduates of the class of 2024, I can hardly imagine anyone more prepared for the future
than you.
You dedicated yourself, you worked hard, you earned a world-class education from one of
the most prestigious schools in the world.
And as you commence into the next stage, take my learnings and hopefully they’ll help you
along the way.
I hope you believe in something, something unconventional, something unexplored.
But let it be informed and let it be reasoned, then dedicate yourself to making it happen.
You may find your GPU, you may find your CUDA, you may find your generative AI, you may find
your NVIDIA.
I hope you will see setbacks as new opportunities.
Your pain and suffering will strengthen your character, your resilience and agility, and
they are the ultimate superpowers.
Of all of the things that I value most about my abilities, intelligence is not top of that
list.
My ability to endure pain and suffering, my ability to work on something for a very, very
long period of time, my ability to handle setbacks and see the opportunities just around
the corner I consider to be my superpowers.
And I hope they’re yours.
And I hope you find a craft.
I hope you find a craft.
It’s not important to decide on day one.
It’s not even important to decide anytime soon.
But I hope you do find a craft that you want to dedicate your lifetime to perfecting, to
hone the skills of and let it be your life’s work.
And then lastly, prioritize your life.
There are so many things going on, there are so many things to do, but prioritize your
life and you will have plenty of time to do the important things.
Congratulations, Class of 2024!
Go get them!
0 Comments