Software
• I am excited about the prospect of synthetic data or self-play in order to further scale LLMs (see: https://www.dwarkeshpatel.com/p/will-scaling-work), as we appear to be hitting capacity constraints in terms of the supply of human generated data on the internet.
• The emergence of multi-modal is key for LLMs – we need a form factor that isn’t typing text into an interface, but rather communicating with models in a more realistic and intuitive manner.

AI Safety

• The emergence of models that reject the contemporary AI safety doctrine, like Erik Voorhees’s Venice (https://moneyandstate.com/blog/the-separation-of-mind-and-state), or Meta’s open source Llama 3, which appears to be very performant relative to close source models. Keeping models open source is critical in terms of allowing AI development to exist free from the monoculture in Silicon Valley. Zuckerberg should be praised for this (see: https://www.dwarkeshpatel.com/p/mark-zuckerberg) . Firms that maintain the full inference stack and refuse to censor AI outputs are equally important. Hopefully, the nexus of crypto and decentralized AI can also help here, although these projects seem to be less performant than conventional inference setups at present. I am still evaluating the crypto-AI space, haven’t found anything I find super compelling just yet.

Hardware

• The emergence of NVIDIA H200s which have far better performance than their predecessor, the A100 (https://nvidianews.nvidia.com/news/nvidia-supercharges-hopper-the-worlds-leading-ai-computing-platform). We are seeing inference speed getting far better recently, and I believe this is partly due to H200s
• Massive datacenter scale, such as Microsoft’s planned 100b datacenter (https://www.reuters.com/technology/microsoft-openai-planning-100-billion-data-center-project-information-reports-2024-03-29/). So far, datacenters have been pretty small (the largest are ~100MW). New versions will be in the gigawatt range, but this infra takes time to build.

Products

• I’m excited about AI wearables like Rewind or Tab (I am an angel here). Although recent products like the Rabbit or the Humane pin were panned by reviewers, I think AI wearables that ingest multimodal data from your everyday life will be extremely compelling, as they are a kind of “external storage for your brain”. You can query them about anything that you did or said throughout the day. I think these will be very compelling products.
• Not LLM based, but the two new products that stood out to me recently are Sora (text to video) https://openai.com/index/sora/ and Suno (music) https://suno.com/about. Both truly remarkable products, especially Suno

Note, from the investment perspective, I am much more interested in AI applications rather than infrastructure. I believe models will be largely commoditized (see this post: https://press.airstreet.com/p/alchemy-is-all-you-need) , whereas applications can create enormous value without huge upfront fixed costs.