August 15th 2024

The EU's AI Act; Nvidia's chip woes; and AI and LLM arms race inertia in enterprise deployment. Oh, and the real story behind Zuck's new necklace.

The doozy AI summer continues

The last time I wrote the AI arms race was having a crazy summer week. Llama 3.1 launched alongside OpenAI’s search and Cohere raised a punchy $500M. The pace hasn’t slowed: large model press releases continue to be rampant.

The EU released its 144-page ‘AI Act’ which is mainly a) posturing at the extremities of risk management around development for hostile uses and b) virtue signalling to broader centre-left Europeans and Sam Altman skeptics (see the Guardian’s “OpenAI’s Sam Altman is becoming one of the most powerful people on Earth. We should be very afraid”).

Intel laid-off 15,000 people citing decreasing revenues due to various failures in AI initiatives.

Hours into August, Google Gemini 1.5 Pro overtook OpenAI GPT-4o in a significant community AI benchmark. GPT-4o achieved a score of 1,286, while Claude-3 secured 1,271. Previous iteration. Gemini 1.5 Pro, scored 1,261. For the past year or so, Anthropic and OpenAI have been the dominating contenders; no longer. Whereas AI firm Galileo released a hallucination index ranking top large language models (LLMs). The index focuses on RAG (Retrieval Augmented Generation) putting Anthropic’s Claude 3.5 Sonnet ahead of Gemini 1.5 Flash.

Is the bubble bursting?

Google have advanced in AI-robots, developing droids that beat humans at ping pong. Amusing anecdotes of the cyborgs dominating humans in sports aside, the core sentiment across markets has been one of volatility, with recent easing. The VIX Index calming after it’s 4 year high is probably just a momentary and much needed mid August sigh of relief in a high octane pace of evolution across the entire market.

News that Apple side stepped Nvidia in favour of chips designed by Google for its AI software infrastructure, didn’t help Google amidst wider market woes earlier in August. Investors cited caution about the rate of return across AI and chip technologies and in some cases were all-out freaked.

Chips on the other hand, taking NVIDIA as a case in point, we see commentators including respected Hedgefund, Elliot Management describing a plausible bubble.

The basic thesis is that unless both exceptional (world-class, sustained) execution is matched with sustained high-growth demand, then NVIDIA will struggle to realise the trillions in returns it needs to recoup value for its enormous market cap. NVIDIA shed 20% from it’s share price between July and August, only to increase it back in the last week.

A finance friend recently sent me this visualisation, showing how diversification consistently matters as much as not trying to time the market: Magnificent 7 Mania: Why Diversification Still Matters

I suspect this is more of a slowing or a frothing than a bubble bursting. The market needs to figure out where and how to apply Generative AI for pragmatic and valuable use cases at scale. As for the foundational hardware side of things, NVIDIA have a mountain to climb in a hugely complex multi-faceted economic model. Many parameters of chip design and supply chain are almost continually on the science-innovation event horizon and that’s before we bring in geopolitical and natural resource constraints and dynamics. Heavy, man.

Scale challenges

This is a case of two deeply intertwined industry economic models needing to become efficient at an unprecedented pace. Two key (and current at time of writing) supply and demand dynamics are (1) the success of chip manufacturers hitting delivery targets and (2) the pace of large companies deploy AI at scale to creating a sustained demand.

Chip flaws and aggregate supply currently underpin scale growth curves. The underlying GPU (Graphical Processing Unit) layer is incredibly complicated and could slow things down properly. Nvidia reported delays its next AI chip due to a design flaw affecting its substrate layer exclusive to the Blackwell GPU family. Nvidia will be another Blackwell GPU (B200A) which isn’t reliant on problematic CoWoS-L substrate. Instead, Nvidia’s chip will use an older technology which is simpler, but less future proofed, called CoWoS-S. Go deeper here with a fantastic analysis from semianalysis.com)

Zuckerberg and (Jensen) Huang said every company will have an AI. Of course they’re bullish, but they’re also sound-biting for us mere mortals.

Every business will have an ever evolving complex web of interacting AI.

AI scale is still hampered by limited clarity of where AI should be deployed beyond traditional deployment of AI for uses such as translation. Where we have those existing applications, AI companies and enterprise alike are rapidly deploying to maximise value as we’re seeing with Deepl in the translation space.

Part of the hesitation or inertia that is evident at strategic corporate levels is really about how AI can be deployed accurately and safely. No doubt, enterprises will deploy AI in ways we can’t yet appreciate as we see a shift in AI’s centre of power towards the need to integrate these models at the training and prompt level with other data sources (RAG is one method of ensuring an LLM responds to a prompt by sending a request to another external data source to retrieve authoritative or credible data). As companies figure out how to deploy accurately and responsibly, they’ll also figure out targeted areas to deploy.

(More on the extensibility of LLMs in a wider context in this nice interview with Yoav Shoham, former principal scientist at Google, an emeritus professor at Stanford University and co-founder of AI21 Labs.)

We’re early in this current contextual evolution but things are pointing at complex webs of LLMs combined with multiple neural networks and proprietary data sources.

So, yes, every company will have an AI, but will that AI be a static single LLM? Likely not at all. These models will become increasingly context aware and integrated as each company has a model that is a living evolving language model.

Fresh money, money

Fear not. There’s still plenty of capital being dished out at eye watering, industry shifting levels. This is still the best time in history for growth, industry, innovation, peace and generally, quality of life. $250+bn is spent on SaaS apps a year.

Content creators Canva announced their purchase of Leonardo.Ai – with its generative AI foundational model for image generation, accelerating Canva to become a genuine AI native leader.

Other chip manufacturers are arming with serious capital for the race. Groq’s value reached $2.8bn after a reported $640M funding round to take on Nvidia. Last year Groq adapted Meta’s LLaMA to work on their chips instead of Nvidia’s which Meta had until that point trained them on. Meta’s chief AI scientist Yann LeCun was named its newest technical adviser.

Necklaces and AI

And finally, leaving on an uncynical note: the thing that’s really behind Mark Zuckerberg’s new necklace (tip: it’s a 48 second video, and a genuinely much more human story than the more unnervingly dystopian Friend necklace).

If you liked this, please give it a share on your favourite network: