Jonathan Frankle on MosaicML Cloud

Weaviate Podcast #26. Thank you so much for watching the 26th episode of the Weaviate Podcast! This is another really special episode! Jonathan Frankle is one of the world's experts in Deep Learning and is making incredible advances at MosaicML in efficient Deep Learning training. The headline event is the release of MosaicML Cloud and a set of new cost estimates for GPT language models at different scales (linked below). Jonathan explains that these numbers are a baseline and he predicts they could get to as low as $100K as they seek opportunities for efficiency optimizations. This story has already played out in the realm of ResNet ImageNet training as MosaicML has demolished expectations of how fast we can train these models and it seems highly likely they will do the same for large language model costs. Jonathan and I also discussed the general space of Language Models and their applications, especially discussing their role as Databases in things like the Weaviate Vector Search Engine. We also discussed Self-Ask, Chain-of-thought Prompting, and tool use in Language Models. I had an awesome time picking Jonathan's brain about these topics and I hope you all enjoy the podcast, more than happy to answer any questions or entertain any ideas / discussion! Thanks again for watching!  Blog post: GPT-3 Quality for less than $500K - https://www.mosaicml.com/blog/gpt-3-q...