tech
Kog is going deeper to squeeze more inference out of GPUs
The idea that GPUs are poorly suited for agentic workflows may be a misconception, according to French startup Kog.
TL;DR
- Kog is a French startup focused on accelerating AI inference speed through software optimization on existing GPUs.
- The company challenges the notion that specialized hardware is necessary for fast AI inference.
- Kog's approach involves deep, low-level optimization of GPU architecture.
- Their CEO, Gaël Delalleau, has a background in solid-state physics and offensive cybersecurity, influencing their methodology.
- The startup aims to achieve "30x faster LLM inference" and plans to demonstrate this on major models by September.
- Kog has secured initial business leads, with software engineering identified as a primary use case.
- The company is backed by French entities like Scaleway, Bpifrance, and the French Tech 2030 program.