Anthropic Tech Phone Screen: Designing Inference Systems with GPU Load Balancing

Explore system design strategies for Anthropic's inference system focusing on GPU load balancing, batch strategies, and cache hit rates during a software engineer tech phone screen.