Questions about the next wave of compute

On Stratechery last week, Ben Thompson wrote about Microsoft's earnings ($), and closed with the observation that “Wintel is dead, long live Microvidia”.   He is bullish on both Microsoft and NVIDIA as the AI wave continues.  And this got me thinking about the decline of the Intel architecture and the growth of GPGPU-based processing.

And then I read this thread, debating whether or not quantum computing will be a great fit for AI computation.   I am not qualified to make that judgment.  But I started to wonder about the growth prospects for GPGPUs.

And so I went down a rabbit hole — how has compute power grown? What architectures have been dominant?  And most importantly, what does this suggest for the future?

An aside on analysis

Vibe coding is all the rage, and analyzing the history of computing is a lot easier than coding, so I put my trust in Google Gemini and OpenAI ChatGPT to be my research analysts.   I asked for reports on both the history of computing power and architectures, as well as some forecasts.   I haven’t bothered to check every last citation and detail; I am going with the vibes.   The reports appear to be generally accurate; they align with my own personal experience.   They don’t completely agree, and I am sure there are errors in them, but directionally, they seem fine.

You can find the reports here.   Apologize for formatting on both — the reports look great within the respective apps, but all attempts to export result in horrific-looking documents.  

Massive growth in MFLOPs

My working career began in 1984, a convenient point from which to examine the growth of computing.  It was the early days of PCs; the first cell phone started selling the previous year; the internet was an academic curiosity.

The total compute power shipped that year — the sum of the computing power of every computer shipped in the entire global market — was about 75-100 MFLOPS.  By the way, your phone has WAY more compute power than the entire world had in 1984, orders of magnitude more power.   We have seen an 11 orders of magnitude increase in world compute power during my working career — absolutely stunning.

The chart suggests that growth has slowed slightly since 2004.   Without really tearing apart the numbers, I would take that with a grain of salt — it is challenging to compare MFLOPs across x86, ARM, and GPU architectures.   I suspect any variation in the slope of this curve is based on this mix shift.

And that mix shift is the biggest story of these 40 years — the dramatic changes in architectures and suppliers, which I'll walk through.

1984 — The Start of the PC Era

It was still the early days of the PC in 1984.  The Mac had just launched; Apple was selling much more of the Apple ][ family.  The IBM PC was a big seller.  The market was dominated by mainframes and minis using proprietary OEM processors, and PCs using primarily x86 processors.  There were also a lot of PCs using 68K, Z80, and 6502 processors, but these contributed only a modest amount of computing power.  

CPUs dominated computing, and Intel was ascendant as the PC began to penetrate the market.  GPUs were nonexistent.

1994 and 2004 — The PC Era

By 1994, Windows 3.x was in the market, Windows 95 was nearing release, and the PC was absolutely the king of the market.  Mainframes, minis, and workstations were holding onto their legacy positions, but the writing was on the wall.   The minicomputer business was nearly dead; in 4 years, DEC would be out of business, acquired by Compaq.  GPUs were still nonexistent in the market.  Smartphones didn’t exist.  It was an Intel instruction set world.

Twenty percent of the x86 MFLOPs were provided by non-Intel chips, including AMD.   The massive success of Intel attracted brains and money.  Work was definitely underway on ARM processors, although they had not yet made a significant dent in the market.

Shifting ahead to 2004, Intel machines continued to dominate the market, both on the client and server sides.   AMD continued to provide competition in the Intel-compatible space.

But more interestingly, GPUs had started to ship in high volumes for desktop machines and were delivering a large amount of compute power.  Beyond games, applications had not yet fully harnessed this power.  But it was clear to many people that GPUs could be used in more general-purpose ways — NVIDIA would release CUDA in 2006, and the idea of a GPGPU with compute shaders was definitely in the air.

2014 — GPUs and Mobile dominate

It was an eventful decade, with NVIDIA driving GPUs into a broader range of compute scenarios, and with iPhones and Android phones shipping in volume.   Intel CPUs were still a significant business, but growth had shifted primarily to mobile and GPUs.  And underlying this, of course, was the growth of the foundry business.

And by 2024, AI has pushed GPUs into the data center, and the industry supply of MFLOPs is dominated by GPUs on clients and on the backend.   NVIDIA represents 70-80% of the GPU MFlops volume, and this market is attracting entrants from all sides.

Questions raised about trends and mix

In aggregate, the curves suggest a slowdown in overall MFLOPs growth, but this doesn’t feel like my lived experience.  I suspect that this is just the challenge in comparing GPU MFLOPs to CPU MFLOPs.   But the interesting question is — will we continue at this growth pace, or will it accelerate, or will it moderate?  AI has just scratched the surface of applications; as an investor, I would bet on this growth or even faster.  We are likely limited by chip supply and energy supply constraints, which raises further investment and policy issues.

We have seen significant shifts in architecture and suppliers — from captive mainframe providers, to Intel CPUs, to GPUs and mobile; from proprietary designs and captive manufacture, to freely licensable designs and foundries.  New architectures and new business models have proven to be more effective at servicing evolving compute loads and dramatic scale increases.  So, over the next 20-40 years, as scale continues to drive upward and as compute loads evolve, what new models and new players will arise?   Intel seemed unassailable and yet.   Is Nvidia any more unassailable?   Their team will age, and turnover will happen.  As an investor, I would be betting on new players and new compute architectures.

The US has been fortunate in that most of the leading architectures have been invented by US companies, and much of the economic benefit has been captured by US companies.   ARM is the outlier, tho it has a strong US heritage.   Are we making the policy decisions to encourage the next wave of architectures and suppliers to be US-based?   The Chips Act is great, but it is focused on the production problems of today — and there is nothing wrong with that, but it is not sufficient to insure economic success in the future.  We must encourage basic research on the next generation of architectures.   Is quantum computing the answer?   Which quantum computing approach?  Some other approach?   I certainly don’t know, but I do know we should aggressively encourage research through direct investment, tax policy, and immigration policy.