D2C218: What’s Inside The AI Magic Box?

AI and machine learning are being more widely used in IT and elsewhere. Today’s episode opens the AI magic box to better understand what’s inside, including software and hardware. We discuss essentials such as training models and parameters, software components, GPUs, networking, and storage. We also discuss using cloud-based AI platforms vs. building your own in-house, and what to consider when assembling your own AI infrastructure. Episode Guest: Fredric Van Haren is CTO & Founder of HighFens Inc. He’s an expert in AI and High Performance Computing (HPC). He has over 20 years of hands-on experience in high-tech and has built large HPC & AI environments for several customers from the ground up. He provides technical leadership and strategic direction to organizations aiming to become leaders in today’s AI market. He’s also co-host of the Utilizing AI podcast. LinkedIn | X Sponsor: DoiT An award-winning strategic partner of Google Cloud and AWS, DoiT works alongside more than 3,000 customers to save them time and money. Combining intelligent software with expert consultancy and unlimited support, DoiT delivers the true promise of the cloud at peak efficiency with ease, not cost. Their technology is backed by deep multicloud expertise in the analytics, optimization and governance of cloud architecture, as well as specializations in Kubernetes, AI, and more. Learn more at doit.com. Episode Links: HighFens HighFens Blog MLCommons Transcript: This episode was transcribed by AI and lightly formatted. We make these transcripts available to help with content accessibility and searchability but we can’t guarantee accuracy. There are likely to be errors and inaccuracies in the transcription.  Ethan Banks (00:00:01) – Today’s podcast is sponsored by do reduce your cloud spend by improving your cloud efficiency with do It and award winning strategic partner of Google Cloud and AWS. Find out more at DeWit. That’s it. Ned Bellavance (00:00:17) – Welcome to day two cloud, you awesome human being. Today we are getting into the nuts and bolts of AI. This isn’t going to be architecture. We’re getting into true infrastructure and to help us out with that, we have a special guest, Frederik Van Buren. He’s the CTO and founder of Hyphens Inc, and he knows a thing or two about AI, doesn’t he, Ethan? Ethan Banks (00:00:38) – Yeah, more than a thing or two. I mean, he is a hands on engineer approach with deep knowledge of both hardware and software stacks that are used to do AI computing, and he understands the business of AI deeply. I had a feeling, even though we chatted with him for, I don’t know, about 45 minutes, Ned. We were just scratching the surface. Ned Bellavance (00:00:57) – Oh, absolutely. So enjoy this conversation with Frederik van Herren.