Secure direct-to-cloud file uploads: presigned URLs, magic bytes, and the gotchas nobody warns you about

A teacher clicks "Upload," picks a PDF of this week's lecture notes, and a second later it shows up in the course. Boring. Exactly what you want a feature to be.

But behind that boring click is a surprisingly deep rabbit hole. I was building the materials-upload feature for MahirLearning, an LMS I've been working on, and I kept asking myself one uncomfortable question: what could a malicious file actually do here? This post is the story of answering that question — the architecture I landed on, every security decision I made, the ones I deliberately skipped, and the two bugs that ate an afternoon each.

If you're about to add file uploads to any app, I hope this saves you some of that afternoon.

The first decision: don't let files touch your server

The naive design is: browser sends the file to your API, your API streams it to storage. It works, and it's a trap. Now your API is a bottleneck for every megabyte, your request timeouts have to account for slow uploads, and a few large uploads can pin your instance.

So I went with the pattern everyone eventually converges on: presigned URLs. The file goes directly from the browser to object storage (I'm using Cloudflare R2, which speaks the S3 API). My Go backend never sees the bytes on the way in — it only hands out a short-lived, signed URL that says "you may PUT one object to exactly this key for the next 5 minutes."

That single decision shapes everything else. It also creates the core tension of this whole post:

If the bytes never pass through my server on upload, how do I know what was actually uploaded?

Hold that thought. It's the reason half of this article exists.

The architecture at a glance

Here's the whole thing on one screen. Two phases to upload (presign, then confirm), and a separate signed path to download.

The key idea: the database row and the storage object are created in two steps, and the row only becomes real once I've inspected the bytes.

Phase 1 — presign: the row is a promise, not a fact

When the admin asks to upload, the request is tiny — just filename, declared content type, and size. No bytes yet. My API does three things before it signs anything:

func (s *Service) PresignUpload(ctx context.Context, userID uuid.UUID, req Presign) (*PresignURL, error) {
    // 1. resource_id must be a real UUID.
    if _, err := uuid.Parse(req.ResourceID); err != nil {
        return nil, ErrFailed
    }
    // 2. The key extension comes from the (allow-listed) type — NEVER the filename.
    ext, ok := ExtByType[req.ContentType]
    if !ok {
        return nil, ErrFailed
    }
    // 3. Clean the display filename before it ever gets stored.
    req.Filename = sanitizeFilename(req.Filename)
    key := fmt.Sprintf("%s/%s/%s%s", req.ResourceType, req.ResourceID, uuid.New().String(), ext)
    // ...sign a PUT for `key`, INSERT an attachments row with status = 'pending'
}

Two things worth pausing on.

The key never trusts the filename. A classic file-upload bug is building the storage path from user input (uploads/ + filename), which invites path traversal and weird collisions. My key is course/<courseUUID>/<randomUUID>.<ext>, where the extension is looked up from the validated content type. The client's filename can be ../../etc/passwd.pdf for all I care — it never touches the key. I keep the original name only as a display label, and even that gets sanitized (strip directories, kill control characters, cap the length).

The row is inserted as pending. At this point I have a database row and a signed URL, but no file. The row is a promise. If the client never uploads, or uploads garbage, that promise is never kept — and nothing that reads the materials list will ever see a pending row.

There's also a subtle bug I hit here that's worth its own paragraph.

The Content-Length footgun

My first instinct was to bake the size into the signature: sign the PUT with an exact Content-Length so R2 rejects anything bigger. Sounds airtight. In practice, browsers manage the Content-Length header themselves, and signing it turns a browser upload into a coin-flip of 403 SignatureDoesNotMatch. I ripped it out. The lesson: don't sign headers the browser controls. I enforce size a different way (below), where I actually have authority.

Phase 2 — confirm: where I finally meet the bytes

The browser PUTs the file straight to R2. Then it calls POST /confirm with the key. This is the moment the two-phase design pays off, because now I can go read what actually landed in the bucket.

This is also the answer to the tension from the top of the post. I never saw the bytes on the way in — so I go get them on the way to confirming.

func (s *Service) ConfirmUpload(ctx context.Context, userID uuid.UUID, key string) (*Attachment, error) {
    // The pending row proves THIS admin presigned THIS key, and tells me the declared type.
    pending, err := s.repo.GetPendingByKey(ctx, key, userID)
    if err != nil {
        return nil, err // no matching pending upload → 404
    }
    // Re-check the real size (authoritative, from storage — not the client's claim).
    head, _ := s.r2.S3.HeadObject(ctx, /* bucket, key */)
    size := aws.ToInt64(head.ContentLength)
    if size > MaxUploadSize {
        s.deleteObject(ctx, key)      // clean up the oversized blob
        return nil, ErrFailed
    }
    // Read the first 32KB and sniff the REAL type from magic bytes.
    header, err := s.r2.ReadHeader(ctx, key, HeaderSniffBytes)
    if err != nil {
        return nil, err
    }
    detected := mimetype.Detect(header)
    if !verifyDetectedType(detected, pending.ContentType) {
        s.deleteObject(ctx, key)      // it's not what it claims — nuke it
        return nil, ErrUnsupportedContent
    }
    // Only now is it real: mark confirmed and store the verified type.
    return s.repo.ConfirmByKey(ctx, key, userID, size, detected.String())
}

Every line here is a security control, so let me unpack them.

Security decision #1: never trust the client's content type

The single most important rule of file uploads: Content-Type is a suggestion the client makes, not a fact. An attacker can upload an HTML file and label it image/png. R2 will happily store it with whatever type was declared.

Go's standard library has http.DetectContentType, but it's basic. I reached for github.com/gabriel-vasile/mimetype, which reads magic bytes — the actual signatures at the start of a file — and detects PDFs, images, and Office formats far more reliably.

The check runs server-side, at confirm time, on bytes I fetched from storage myself — not on anything the client told me. If the sniffed type isn't on my allowlist, I delete the object and reject with a 400. The file is gone within a second of arriving.

The ZIP problem, and a decision I had to make

Here's where it got interesting. Images and PDFs have clean, front-loaded signatures — sniffing them from a small header is trivial. But .pptx (and every modern Office format) is really a ZIP archive, and .ppt is an old OLE compound file. From a partial read, mimetype can sometimes only tell you "this is a valid ZIP" — it can't always pin down "this is specifically a PowerPoint" without reading deeper into the archive.

That's a genuine trade-off, and I had to choose:

1. Download the whole file to get an exact answer — accurate, but a full 25MB GET on every single upload.
2. Sniff the header only and accept a generic ZIP/OLE container as long as the admin declared an Office type — cheap, one small ranged read, very slightly looser.
3. Drop PowerPoint entirely and only allow images + PDF, which verify perfectly.

I chose #2. Uploads are admin-only, Office files are served as downloads (never rendered in the browser — more on that soon), and I only accept the bare-ZIP fallback when the declared type was actually PowerPoint. The residual risk — an admin renaming some other ZIP to .pptx — is bounded and low. If I ever need to close it fully, option #1 is a one-function change. Naming the trade-off out loud was more useful than pretending there was a free lunch.

func verifyDetectedType(detected *mimetype.MIME, declared string) bool {
    // Exact or ancestor match against the allowlist (images, pdf, exact office types).
    for m := detected; m != nil; m = m.Parent() {
        if AllowedTypes[m.String()] {
            return true
        }
    }
    // Office files often sniff as a generic container from a partial read —
    // accept that ONLY when the admin declared an office type.
    if OfficeTypes[declared] {
        switch detected.String() {
        case "application/zip", "application/x-ole-storage":
            return true
        }
    }
    return false
}

I also store the sniffed type in a verified_content_type column, separate from the client's declared content_type. If anything ever looks off later, I have both values to compare.

Security decision #2: enforce size on my terms

I validate the declared size before presigning, but a client with a signed URL can ignore that and upload whatever they want. So the real enforcement lives at confirm: HeadObject gives me the authoritative size straight from storage, and anything over the limit gets deleted and rejected.

This is the payoff for not signing Content-Length earlier. Instead of fighting the browser over a header, I check the fact after the fact, where the answer is real and I'm the one holding the delete button.

Security decision #3: stop stored XSS at the door (a UX call in disguise)

This one surprised me because it's less about code and more about a product decision.

The threat: someone uploads a file that the browser will render and execute — classically an HTML file or an SVG with a <script> in it. If that renders, you've got stored XSS.

I had two layers of defense, and one decision to make:

• Layer 1 — the allowlist. SVG and HTML simply aren't allowed. The two most common XSS-via-upload vectors are excluded before we even get to sniffing. (This is why the allowlist matters: it's not just "which files are useful," it's "which files are safe to serve.")
• Layer 2 — how I serve. Files are served through short-lived presigned GET URLs. When I sign that URL, I can override two response headers: Content-Type (pin it, so the browser can't be tricked into sniffing something executable) and Content-Disposition.

And here's the decision. Content-Disposition: attachment forces a download — the browser never renders the file, which is the safest possible behavior. But for a learning platform, students genuinely want to preview a PDF or an image inline, not download it every time.

So I split it: images and PDFs render inline (nice UX, and safe because the allowlist already blocks the dangerous types), while PPT/PPTX are forced to download (they can't be previewed inline anyway, and forcing attachment costs nothing).

disposition := ""
if OfficeTypes[ct] { // ppt / pptx
    disposition = fmt.Sprintf("attachment; filename=%q", filename)
}
url, _ := s.r2.PresignGet(ctx, key, ttl, ct, disposition) // Content-Type pinned to `ct`

Security work is usually framed as "lock everything down." Half the real work is deciding where the lock actively hurts the product and finding the spot that's both safe and usable.

Security decision #4: lock down IDOR on both ends

IDOR — Insecure Direct Object Reference — is where you let a user act on an object just because they know its ID. Uploads are a magnet for it.

• Uploads are admin-only. Students physically cannot presign.
• Confirm is ownership-checked. Notice GetPendingByKey(ctx, key, userID) above — it only returns a row where uploaded_by = userID and status = 'pending'. You can't confirm someone else's upload, and you can't re-confirm an already-confirmed file.
• There's no "download by ID" endpoint to abuse. Students never request an object directly. They fetch the materials list, which is access-checked (are you enrolled in this course?), and each item already carries its signed URL. There's no ID a student can tamper with to reach a file they shouldn't see.
• Presigned URLs are bearer tokens. Anyone with the URL can use it — that's inherent to the pattern. I bound the blast radius by keeping the TTL short (an hour) so a leaked link expires quickly.

The bug that ate an afternoon: the .eu. endpoint

Everything above was working in my head, and then uploads just... failed. The browser's preflight to R2 returned a flat 403, and I burned an afternoon convinced it was CORS.

It wasn't CORS. My bucket lives in R2's European Union jurisdiction, and EU buckets have a different S3 host: https://<account>.eu.r2.cloudflarestorage.com — note the .eu.. My backend was building the endpoint from just the account ID, so it was signing every presigned URL for the non-EU host. The browser was politely trying to talk to a bucket that wasn't mine.

One line of config, an afternoon of my life. If your presigned URLs 403 for no reason, check that your endpoint host exactly matches your bucket's region/jurisdiction before you touch CORS.

What I deliberately did NOT do

Good security is also knowing where to stop. Two things I consciously left out:

• EXIF/metadata stripping from images. Real, but the payoff here is tiny: these are institutional materials uploaded by teachers, not personal photos from students. Stripping metadata means downloading and re-encoding every image server-side — a lot of work to protect against a threat that doesn't really exist for this data. Skipped.
• A dedicated rate limit on the presign endpoint. There's already a global rate limiter, and presign is admin-only. A tighter per-endpoint cap is a fine idea for a public upload API; for an admin-only route behind an existing limiter, it's not worth the complexity yet.

Writing these down mattered as much as the controls I built. "We chose not to, because X" is a real engineering answer. "We didn't think about it" is not.

The mental model to take with you

If you remember one thing, make it this: treat every upload as guilty until proven innocent, and prove innocence with bytes you fetched yourself.

Concretely:

1. Get the bytes off your server's critical path — presigned URLs, direct to storage.
2. Make the DB row a two-phase promise — pending until you've inspected it.
3. Verify the real content, not the declared type, from storage-side bytes at confirm.
4. Never build storage keys from user input — derive them from validated data.
5. Decide how you serve before you worry about anything fancy — Content-Type + Content-Disposition stop most "rendered file" attacks for free.
6. Name your trade-offs (ZIP tolerance, inline vs. download, what you skipped) instead of pretending they don't exist.

None of these are exotic. Together they turn that boring "upload a PDF" click into something you can actually sleep next to.

Built with Go, gin, and Cloudflare R2. If you're working through the same problem and something here doesn't line up with your setup, drop a comment — I'd genuinely like to hear how you handled the ZIP-sniffing trade-off.