Multi-tenant LLM platform design: terabyte ingestion, isolation and usage billing
Tenants upload terabytes of audio, video, PDFs and images, and expect an LLM to answer questions over all of it, with their data isolated and a bill that matches what they used. Here is the AWS design we shipped: ingestion pipeline, isolation layers and metering.