Upscale AI & NVIDIA: Scaling Heterogeneous AI Infrastructure

Upscale AI announces open, highly scalable & interoperable AI fabric, built on NVIDIA Spectrum X Ethernet switch silicon & AI-optimised SONiC software

TL;DR

Upscale AI is introducing an open, scalable, and interoperable AI fabric.
The fabric utilizes NVIDIA Spectrum X Ethernet switch silicon and AI-optimized SONiC software.
It is designed for high-performance, low-latency networking for intensive AI workloads.
The system supports heterogeneous architectures, including compute, accelerators, memory, and storage.
Upscale AI is joining the NVIDIA Partner Network to leverage its ecosystem and validated designs.
The AI fabric offers predictable performance, operational simplicity, and reliability.
ASIC-native telemetry provides real-time performance data for lossless Ethernet behavior.
The scale-out systems will be available for AI data centers in late 2026.

Weiter zum Originalartikel

Demis Hassabis reposted

Google DeepMind

This is Decoupled DiLoCo: our new resilient and flexible way to train advanced AI models across multiple data centres. 🧵

Aravind Srinivas reposted

Perplexity

We’ve developed our own inference engine Runtime-Optimized Serving Engine (ROSE) to serve models ranging from embeddings to trillion-parameter LLMs.

With CuTeDSL integrated into our inference engine, Perplexity can build the specialized GPU kernels faster to bring models up to peak performance on NVIDIA Hopper and Blackwell GPUs.

Aravind Srinivas reposted

NVIDIA AI

Perplexity runs on NVIDIA.

Nice breakdown from the team on how they’re using the CUTLASS Python stack to optimize their models for inference 👇

Aravind Srinivas reposted

Perplexity

We published new research on how we serve post-trained Qwen3 235B models on NVIDIA GB200 NVL72 Blackwell racks.

GB200 is a major step up over Hopper for high-throughput inference on large MoE models, not just a training platform. https://t.co/yYZuPRXWzr

Aravind Srinivas

GB 200s change how one does the prefill and decode disaggregation when serving large MoEs like Qwen. We’ve published details of our stack quantifying the throughput benefits compared to serving on Hoppers.

clem 🤗 reposted

Georgi Gerganov

llama.cpp adds MTP for the Qwen3.6 family

This is a significant milestone for the local AI ecosystem. The performance jump with these changes is massive and elevates local inference on commodity hardware further.

Special thanks to Aman Gupta for leading this development!

https://t.co/vjaMwEpIaR

clem 🤗

I believe on-prem and local AI - based on @huggingface open-source models - will be an important answer to the GPU shortages this year (because they are cheaper, faster, safer than cloud APIs)!

Great collaboration between @huggingface & @MichaelDell @Dell to make this a reality for enterprise today. Announced at the main keynote of Dell Technologies World.

clem 🤗 reposted

seanpixel 🫧

i own 0 A100s

but i can run cloud inference with sub 80ms network latency by renting one in Kansas (0% action queue starvation)

this is an early checkpoint of smolvla trained on 50 episodes for ~20k steps. now the goal is to improve on this with 0 expert demonstrations using RL https://t.co/1ajdwAy5I9

clem 🤗 reposted

Patrick

Harvey just published a study showing a hybrid setup, open source GLM 5.1 as primary worker, routing to Opus 4.7 only when needed beats pure Opus 4.7 on quality and costs less.

This is the multi-model routing thesis proved in production on one of the hardest benchmarks in enterprise AI.

The insight isn’t that open source beat frontier. It’s that smart routing beat brute force. Using the most expensive model for every task is not a quality strategy. It’s a laziness tax. The teams building routing layers that send each task to the right model at the right cost are now demonstrably ahead on both dimensions simultaneously.

Inference optimization just became a first-class competitive advantage. Legal proved it first because the stakes forced the discipline.

Satya Nadella

New Azure milestone. The fastest time to train yet at the largest reported scale for this leading AI training benchmark.

A great example of what is possible when we bring together full-stack innovation across silicon, systems, networking, and software, along with our deep partnership with @nvidia, to advance the frontier of AI infra. https://t.co/5ADybErndz

clem 🤗 reposted

0xSero

Best models for your hardware

- 4gb to 12gb vram -

VibeThinker-3B - smokes everything remotely close to its weight class. Challenging 30b models! Last version was also topping math benchmarks

https://t.co/RTchJFFTnV

- 12gb to 24gb vram -

Gemma-12B-coder

Built on top of an already strong model, reduced refusals and 262k context window trained on fable traces https://t.co/DVAhlQ7Y4n

- 24gb to 64gb vram -

Gemma-4-26b-diffusion

This model was already by far one of the most functional and capable models, now it’s hitting 500+ tok/s on consumer hardware! Smart AF made by Google deepmind https://t.co/mSaWPFpgXQ

Cohere North-Mini-Code 30B

A new coding model made by an already impressive lab, its priming worth a shot if you’re looking to test the limits of local coding https://t.co/gDPEj6lPAW

———

For those with 4x 6000s or 3x DGX Spark I think my GLM-5.2-REAP is worth a shot.

Lmk how it goes!

clem 🤗

HF is quietly becoming the best place to store data, public AND private, especially for brutal domains like robotics and video AI where the files are massive, append-only, and never stop growing.

Example? Public robotics datasets exploded from 1,000 in early 2025 to 60,000 today, and there's twice as many private ones.

Why? A single robot records at 140 MB/s, all day, forever. That data has to be stored, streamed to GPUs, and shipped back to hardware on repeat. Get it wrong and your GPUs sit idle at 0 MB/s waiting for a dataset to land. Get it right (stream straight from the Hub, pre-warmed cache) and those same GPUs scream along at ~1,326 MB/s, fully fed. 🚀

Here's how LeRobot + Hugging Face Storage Buckets pull it off: https://t.co/SOUPAUpiZc