Why not just run it ourselves? Self-hosting AI for fun and for profit

Your AI costs are skyrocketing, the frontier model providers are changing quality out from underneath you, and meanwhile you’re sending your core IP across the open Internet. How did we end up here? Open-weight AI models have closed the quality gap with proprietary ones and are surprisingly easy to run. In this keynote, Linkerd co-creator and Buoyant CEO William Morgan walks you through what it takes to serve modern LLMs on infrastructure that you control, as a first-class part of your platform: with cost, reliability, and performance measured, managed,  and controlled. If you can operate a web service, you can add local inference to your platform. You'll leave this session with the ability to judge whether self-hosting makes sense for your organization, and knowing what to build first if it does.

https://javascript-conference.com/generativeai/self-hosting-open-weight-llms/