cache
Weekly Web Share #2
NPM security issues, CDN abuse and more
Faster Agents with Automatic Prompt Caching
Since its launch, automatic prompt caching has been delivering a notable, zero-effort performance increase for Heroku Managed Inference and Agents. Enabled by d…
How ArisGlobal scaled generative AI inference on Amazon Bedrock
ArisGlobal runs NavaX, its life-sciences AI engine, on Amazon Bedrock. By caching the shared prompt prefix instead of each per-case document, ArisGlobal cut eff…
Amazon SageMaker AI real-time endpoints now support KV cache aware routing
Amazon SageMaker AI real-time endpoints now support KV cache-aware routing, a new strategy that routes each request to the instance holding the longest cached m…






