Dear Reader,
One of the surest signs of a technology boom in a specific sector is when there is a shortage in equipment necessary to fuel the growth.
And not surprisingly, that is happening right now in the world of artificial intelligence (AI). For the last few weeks, rumors have been popping up around the complete lack of ability to buy or rent computing systems that are based on graphics processing units (GPUs).
We can think of servers running GPUs as the “workhorses” of the artificial intelligence and machine learning industry. They can be used for any application related to AI for training and running this powerful software.
GPUs are in higher demand than ever before with the explosion in development of large language models (LLMs) like ChatGPT and GPT-4. They are also used to train popular text to image generators as well, such as DALL-E, Stable Diffusion, and Midjourney.
With the companies behind this amazing technology raising massive venture capital (VC) rounds now valued in the billions, there is an absolute gold rush in the tech industry to build and monetize the next great generative AI products and services.
And in order to do that, we need more servers running on GPUs.
To date, most of the industry has relied on cloud-based server infrastructure that can be leased for as long as necessary to train an AI. Microsoft, Google, Amazon, and Oracle are the four largest cloud service providers that offer this kind of computing infrastructure.
But not only is the physical hardware hard to come by now, even finding some to lease has become largely problematic.
This is where large, well-financed companies are now at a competitive advantage. Those with the capital can rent this kind of computing infrastructure full-time. Whether they use it or not, its theirs and no one else can use it. At a time like this, during a gold rush, controlling the infrastructure can actually slow down the smaller competition. After all, if a startup can’t find computing infrastructure to rent, it won’t be able to train its AI. And without that, it can’t build a product.
This has fast become a chokepoint for the industry. And it has been a boon for specific players.
The two largest GPU players in the world, NVIDIA and AMD, are both running. NVIDIA is up 147% and AMD is up 60% from the bear market low in October of last year. As a reminder, the boom in generative AI kicked off in December.
And the company that manufactures both NVIDIA and AMD semiconductors, Taiwan Semiconductor Manufacturing (TSM), is up 44% in the same time frame. TSM wins no matter what.
But there’s another breed of beneficiaries that are going to get supercharged as a result of the current shortage in GPUs and computing systems. It’s the new generation of AI application-specific semiconductors that are designed to be far more efficient. They’re also more cost effective in running artificial intelligence compared to the “general purpose” nature of GPUs.
These are private companies like Cerebras, Graphcore, Mythic, SambaNova Systems, Habana Labs (now owned by Intel), Groq, and Lightmatter. All of these companies will be familiar to regular Bleeding Edge readers. And their products are all suddenly in very high demand not only because of what these new products are capable of doing, but because of the sheer scarcity of NVIDIA and AMD GPU systems.
It’s natural to wonder why a company needs to raise a $10 billion round like OpenAI. Or why Chartacter.ai needs $150 million, or why Inflection needs $675 million. Historically, early-stage tech companies raised a small fraction of that amount to keep development moving in the right direction.
But now it’s easy to understand why. These companies need to build infrastructure that allows them to build and train their AIs unimpeded by any computing constraints. Many of them will be building their own AI training data centers, much in the same way that Tesla built its AI “Dojo.”
And whether these companies buy and build or lease the computing resources, they have to maintain control so that they don’t risk being slowed down. This is a gold rush, and one that’s moving at the speed of light.
It’s not a fad. This technology will proliferate through most parts of society in a remarkably short period of time. And this existential race isn’t just between well-funded companies. It’s between powerful countries that have a glimpse of the future and what it might become.
Some very cool research out of Cornell University got me thinking about a possible new form of computer interface. A team there developed a pair of glasses loaded with artificial intelligence and acoustic-sensing technology – a form of sonar.
The glasses connect to a computing system such as a smartphone or a tablet. And then, the sonar technology enables the wearer to give voice commands just by moving their lips as though they were speaking.
This is wild. Here’s the prototype below:
Source: Cornell University
Here we can see that the glasses are a bit thick, but otherwise they are a normal pair of glasses.
And we can see the acoustic sensors are mounted to the bottom of the frames. They are labeled S1, S2, M1, and M2. One thing to keep in mind, this is just a prototype, a proof of concept, so it doesn’t have to look perfect. The sensors can be made small enough to embed in the frames so that they would be barely visible at all.
So when the wearer mouths words, these sensors capture the inaudible sound waves. Then, the AI translates those waves into voice commands.
And get this – the AI is able to ingest the sound waves and use deep learning technology to interpret natural language with 95% accuracy. Incredible, right?
Obviously, this device would be a game-changer for people who have lost their ability to speak. But I think that there is a far more practical, mass-market application for a technology like this.
It’s easy to envision glasses with this technology being useful when we’re in places where it’s not appropriate to speak out loud. Maybe we’re on an airplane or in the library where it’s not polite to make much noise – this tech would enable us to still issue silent voice commands without causing a disturbance.
This got me thinking – the technology already interprets the sound waves and then uses natural language processing to understand the commands. This same technology could be used for real-time communications.
The AI could synthesize our voice based on previous recordings. Then, it could translate our inaudible voice commands into audible speech that’s in our voice. That would enable us to take a phone call in a quiet place without making a sound, only by moving our lips.
At that point, we would be able to have phone conversations without making a noise. We just mouth our words, and the AI delivers the speech in our voice to the person on the other end of the line. Amazing.
And there’s an even bigger play here…
Longtime readers know that I’ve long said that augmented reality (AR) glasses will one day replace the smartphone as our primary interface for computing. That’s because AR glasses can do everything a smartphone does and a lot more. And they can enable us to do everything hands-free using voice commands.
The only downside is that our world would be pretty chaotic if everyone were running around barking out orders to their AR headset all the time.
Think about how many people we see engaged with their smartphone or tablet when we’re out in public… With AR headsets, many of those people would be issuing voice commands instead of using the device’s keypad. It would be like a cacophony of chatter everywhere we’d go.
The folks at Cornell just solved that problem with this technology.
If we were to incorporate this technology onto an AR headset, suddenly everyone can issue silent voice commands. It makes perfect sense.
So this is fantastic research. And I think we’ll see a company spin up to commercialize this tech very quickly. I suspect it will be a major player in consumer electronics. This would be a strong point of competitive differentiation for augmented or mixed reality. Given Apple’s stated focus on privacy and security, it feels like the consumer electronics giant would be an ideal fit.
We’ve been talking about generative AI seemingly every day this year. There is good reason to as well. The landscape is changing daily, and we have to stay on top of these incredible developments as this technology will dramatically impact our lives. Of course, it all started when OpenAI released ChatGPT upon the world about five months ago.
We’ve been exploring a wide range of applications for this technology. But one of my favorite applications of this technology is the complete democratization of first-class education. It has the potential to provide a world class, private education to every child on Earth regardless of their language or their wealth (or lack thereof).
I’ve been thinking a lot about AI-based tutors. Simply put, they could be transformational for students all over the world.
Imagine a tutor that can learn the absolute best way to teach any subject to an individual student. Imagine a tutor with a completely optimized, one-on-one tutoring style based on the student’s ideal way to learn.
This just isn’t possible in our educational system today. But with generative AI technology, we could give every student in the world their own personalized AI tutor. That would transform education overnight.
And guess what?
That’s exactly what the Khan Academy is working on right now.
The Khan Academy is a non-profit organization that provides free, online resources for learning. It was founded in 2008 when one of the founders began producing short, simple educational videos for his cousin. And the operation has since become an incredible global success.
Today, the Khan Academy offers over 10,000 videos and exercises on every subject out there. There are videos on math, science, history, economics, and a lot more – for a wide range of ages and skill levels.
What’s more, this content is now available in over 40 languages. And the Khan Academy reports having over 120 million registered users across 190 countries.
And now the Khan Academy wants to empower students with its own generative AI. They are calling it Khanmigo.
Khanmigo is designed to mimic one-on-one tutoring experiences. It will provide support customized to each user. And it will suggest relevant exercises and resources as well.
They are testing Khanmigo in a private school in Palo Alto right now – the Khan Lab School. They’ll work out the kinks in that closed environment before making the AI-based tutor available to all students online.
That’s going to be an absolute game-changer.
With Khanmigo, anyone with a computer, smartphone, or tablet will have access to a first-class education in their own native language. And we should keep in mind that roughly 70% of the world’s population owns a smartphone.
So when it’s ready, Khanmigo will be like turning on a light switch. Anyone with a smartphone can simply go to the Khan Academy site and get access to this powerful AI tutor.
I can’t tell you how excited I am about this. We could be looking at the biggest educational advancement in history… I can’t wait to see how it plays out.
We’ll wrap up today with yet another big development on the generative AI front. Bloomberg’s getting in on the act…
Bloomberg is a critical resource for financial professionals. It provides real-time data on just about every market, security, and economic indicator out there. My team and I use Bloomberg’s terminal every day, throughout the day, to do our work.
The user interface takes some getting used to, and the sheer volume of information can be overwhelming. It can be hard to find things sometimes, and the search capabilities are definitely lacking.
But the most valuable asset of Bloomberg is the data that it houses and maintains for its subscribers. So it makes perfect sense for Bloomberg to develop its own generative AI. They are calling it BloombergGPT. And it will function very similar to ChatGPT.
The difference is, BloombergGPT doesn’t need to be trained on the entire open internet. We can think of it as an application-specific large language model that is focused on the world of finance and economics. There’s no need to train the model on anything else as it could muddy the outputs.
Instead, Bloomberg will train its AI exclusively on its own data within the Bloomberg terminal. That way all of its responses will be clear, concise, and entirely related to the subject matter at hand.
In industry terms, training AIs on smaller, more focused data sets like this will lead to “cleaner” outputs.
I’m very excited to experiment with BloombergGPT. It could help make financial professionals far more efficient. And it will quickly become a fantastic productivity tool for both experienced and beginner users of the platform.
What’s more, I expect Bloomberg may leverage its AI into a completely new business line.
As it stands, Bloomberg’s entire business is based on a traditional platform licensing model. Enterprises pay tens of thousands of dollars every year for each “seat” in order to access Bloomberg’s system.
So I’m interested to see if Bloomberg makes BloombergGPT available only on a subscription basis. That could become a very material revenue stream for the company.
And this speaks to what I think will become a much larger trend…
When it comes to training AIs, the data set is by far the most important element. Nobody can produce a functional AI without good data to train it on. And so many large language models have already been open sourced, which means that having access to a LLM isn’t the competitive advantage. The competitive advantage is having access to the necessary computing resources and the quality of the output of the LLM, which is largely driven by the quality of the curated data set that it is trained on.
And if we extrapolate one step further, it means those companies out there that are sitting on a mountain of valuable data like Bloomberg are quickly going to become hot commodities. Many of these companies may not even be good businesses… but suddenly their value will grow if their data is useful for training an AI.
So I think we’re going to see a rash of acquisitions and licensing deals take place in the coming months. Data is set to become the new gold rush… and that means big profits are coming for those companies sitting on highly valuable and curated data sets.
Regards,
Jeff Brown
Editor, The Bleeding Edge
The Bleeding Edge is the only free newsletter that delivers daily insights and information from the high-tech world as well as topics and trends relevant to investments.
The Bleeding Edge is the only free newsletter that delivers daily insights and information from the high-tech world as well as topics and trends relevant to investments.
