Web Development Paid

Uploading 10 GB files to S3 from the browser: presigned multipart with resume

Video and dataset uploads of 5 to 15 GB kept failing through our API servers. We moved to browser-direct S3 multipart uploads with per-part presigned URLs, resumable state, per-part checksums and server-side completion. This is the full design and working code.

John Nguyen @devbugs · May 23, 2025 · 10 min read · 68 Views

Banner: a large file split into numbered parts uploading in parallel from a browser to object storage

Our video review product lets agencies upload raw footage for clients to comment on. Files are typically 2 to 15 GB. The first version streamed uploads through our API servers to S3, and it failed in all the predictable ways: proxies timing out at 60 seconds of slow upload, a single dropped connection at 9 GB throwing away an hour of work, and API pods running out of disk and bandwidth whenever three customers uploaded at once. This post is the design we replaced it with: the browser uploads dir...

This is a paid story

Subscribe to read the full content, access all paid stories, and support the authors.

Subscribe to Read Already a subscriber? Sign in

#javascript #nodejs #presigned-url #aws-s3 #multipart-upload

Written by

John Nguyen @devbugs

I am a Senior Software Engineer with over a decade of hands-on experience building, scaling, and maintaining complex systems across the full stack—spanning frontend, backend, and DevOps. Over the last 10 years, I have navigated diverse architectures and untangled some of the most frustrating, elusive bugs in production. I believe that every production incident is a massive learning opportunity. Having discovered hibug.me, I am excited to contribute to this knowledge base, breaking down root causes, and sharing verified fixes to help fellow developers debug faster and build more resilient systems.

Comments

No comments yet. Be the first to comment!

This is a paid story

Related Stories

Comments