Remix file uploads using S3, Cloudflare R2 and Hetzner

A guide to uploading files in Remix using S3-compatible storage providers

October 22, 2024

In a traditional SaaS app like you'd build with Remix, you often need to allow users to upload files. Supporting this can be tricky as there are a ton of edge cases to consider around file size, file type, and permissions.

In this guide, we'll walk through how to upload files in a Remix app using S3-compatible services, specifically via presigned URLs.

Presigned URLs

Presigned URLs are URLs that grant temporary access to upload a file to a storage service. They're generated by the storage service and include all the necessary permissions and metadata for the file being uploaded. They're extremely powerful as by allowing users to upload files directly to the storage service, you never have to handle the file itself, saving you bandwidth and processing power as well as improving security.

Quick demo

Before diving into the guide, you can see a working demo of what we'll be building. This app supports switching between S3, Cloudflare and Hetzner seamlessly in a single app to show how flexible it can be.

The implementation is somewhat unrealistic as you probably won't need to support multiple providers like this, plus there's little validation and no real authentication or database persistence, but the principles outlined in this guide are the same.

If you just want the source code, check out the repo.

image info

S3

S3 is one of the oldest services offered by AWS and its popularity is such that other providers have made S3-compatible APIs for their own services. This means that by building off of S3, you can easily switch providers in the future if needed.

Getting set up

To generate presigned URLs you need a few packages:

  • @aws-sdk/client-s3 to create the S3 client that reads and writes to storage
  • @aws-sdk/s3-presigned-post to generate presigned POST URLs
  • @aws-sdk/s3-request-presigner to generate presigned GET URLs (and in some cases POST URLs)
npm install @aws-sdk/client-s3 @aws-sdk/s3-presigned-post @aws-sdk/s3-request-presigner

For this guide we'll be using S3, Hetzner and Cloudflare with the AWS SDK. Initializing the client is slightly different for each:

import { S3Client } from "@aws-sdk/client-s3"; // If using S3 const s3Client = S3Client({ region: process.env.AWS_REGION, credentials: { accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, }, }); // If using Cloudflare const s3Client = S3Client({ region: "auto", endpoint: process.env.CLOUDFLARE_R2_ENDPOINT, credentials: { accessKeyId: process.env.CLOUDFLARE_R2_ACCESS_KEY_ID, secretAccessKey: process.env.CLOUDFLARE_R2_ACCESS_SECRET_KEY, }, forcePathStyle: true, }); // If using Hetzner const s3Client = S3Client({ endpoint: process.env.HETZNER_ENDPOINT, credentials: { accessKeyId: process.env.HETZNER_ACCESS_KEY, secretAccessKey: process.env.HETZNER_SECRET_KEY, }, });

General structure for presigned URL uploads

There are three main places you'll need to interact with S3 for file uploads in a Remix app:

  1. loader - Here you'll generate a presigned POST URL to upload the file to S3. This will be a simple JSON response with the URL and any required fields. This doesn't technically need to be a loader, and you could generate it at a later point from an API route, but for simplicity we'll use a loader here.
  2. Form - You'll need to create a form that submits to the presigned URL. This form will need to include any fields required by the presigned URL, such as the file field and any additional fields.
  3. action - After the file is uploaded to S3, you'll need to save the file details to your database. This will likely be done in an action that is called after the form is submitted.

Uploading the file (common parts)

Before we dive into the specific implementations for different services, let's look at the common structure we'll use for file uploads. We'll use a traditional HTML form within our Remix component. We don't need a Remix Form because we're sending a request to an external service. Here's the basic structure of our Upload component:

export default function Upload() { const { presignedUrl, fields } = useLoaderData(); const fetcher = useFetcher(); const handleSubmit = async (event) => { event.preventDefault(); const form = event.currentTarget; const fileInput = form.elements.namedItem("file"); const file = fileInput.files?.[0]; if (file && presignedUrl) { // We'll implement the specific upload logic for S3/Hetzner and Cloudflare later // This is the same for both implementations const fetcherFormData = new FormData(form); fetcherFormData.append("key", loaderData.key); fetcherFormData.append("originalFileName", file.name); fetcher.submit(fetcherFormData, { method: "post" }); fileInput.value = ""; // Clear the file input } }; return ( <form onSubmit={handleSubmit}> <input type="file" name="file" accept=".txt" /> <button type="submit"> Upload to S3-compatible storage </button> </form> ) }

This component sets up the basic structure for our file upload functionality. The handleSubmit function will contain the logic for uploading the file, which we'll implement differently for S3/Hetzner and Cloudflare.

Now, let's look at the specific implementations for creating presigned URLs and uploading files for both S3/Hetzner and Cloudflare.

Creating the presigned URL

Let's start with the loader. This will generate a presigned URL that we can use to upload the file to S3-compatible storage.

S3 and Hetzner Implementation

Creating the presigned URL

import { createPresignedPost } from "@aws-sdk/s3-presigned-post"; export const MAX_FILE_SIZE = 1 * 1024; // 1KB export const ALLOWED_FILE_TYPES = ["text/plain"]; export const loader = async ({ request }) => { const session = await getSession(request); const userId = session.get("userId"); const bucketName = process.env.UPLOADS_BUCKET_NAME!; const { url, fields } = await createPresignedPost(s3Client, { Bucket: bucketName, Key: key, Conditions: [ // Max size ["content-length-range", 0, MAX_FILE_SIZE], // Only allow certain file types ["eq", "$Content-Type", ALLOWED_FILE_TYPES.join(",")], // Uploaded location must be prefixed with the specific user key ["starts-with", "$key", `user/${userId}/uploads`], ], Expires: 3600, }); return { presignedUrl: url, fields, } };

S3 and Hetzner can use the createPresignedPost() method to generate URLs. This comes with a few security and permission features baked in via conditions:

  • "content-length-range", 0, MAX_FILE_SIZE

    This ensures the uploaded file will be rejected if it exceeds the maximum allowed length (in our case 1KB)

  • "eq", "$Content-Type", ALLOWED_FILE_TYPES.join(",")

    This ensures only specific files types will be accepted (in our case text/plain)

  • "starts-with", "$key", `user/${userId}/uploads`

    This is an additional security measure to ensure that the path of the uploaded file in your storage bucket is prefixed with the current user's ID. By doing this each user's files are isolated in the bucket and other user's can't upload files to other's directories

Uploading the file

For S3 and Hetzner, we'll use a POST request to upload the file. Here's how we can modify the handleSubmit function in our Upload component:

const handleSubmit = async (event) => { // ... (previous code remains the same) if (file && presignedUrl) { // S3/Hetzner specific upload logic const formData = new FormData(); Object.entries(loaderData.fields).forEach(([key, value]) => { formData.append(key, value as string); }); formData.set("Content-Type", file.type); formData.append("file", file, file.name); await fetch(presignedUrl, { method: "POST", body: formData, }); // ... (rest of the function remains the same) } };

This will add all of the fields generated by the createPresignedPost() method to the form submission, comprising the file metadata and upload URL-specific permissions.

In practice, the fields will look like the following list, but you don't need to worry too much about it: Policy, X-Amz-Algorithm, X-Amz-Credential, X-Amz-Date, X-Amz-Signature, bucket and key.

Cloudflare Implementation

Creating the presigned URL

import { PutObjectCommand } from "@aws-sdk/client-s3"; import { getSignedUrl } from "@aws-sdk/s3-request-presigner"; export const ALLOWED_FILE_TYPES = ["text/plain"]; export const loader = async ({ request }) => { const session = await getSession(request); const userId = session.get("userId"); const bucketName = process.env.UPLOADS_BUCKET_NAME!; const command = new PutObjectCommand({ Bucket: bucketName, Key: key, ContentType: ALLOWED_FILE_TYPES[0], }); const url = await getSignedUrl(s3Client, command, { expiresIn: 3600, }); return { presignedUrl: url, } };

Cloudflare works slightly differently as it doesn't support uploading to presigned URLs via POST with the url and fields options. You have to generate a single URL which you eventually upload to with the PUT command.

Uploading the file

For Cloudflare, we'll use a PUT request to upload the file. Here's how we can modify the handleSubmit function in our Upload component:

const handleSubmit = async (event) => { // ... (previous code remains the same) if (file && presignedUrl) { // Cloudflare specific upload logic await fetch(presignedUrl, { method: "PUT", body: file, headers: { "Content-Type": file.type, }, }); // ... (rest of the function remains the same) } };

Note that this approach doesn't require the additional fields that S3 and Hetzner use.

Common steps

Saving file details

After the file is uploaded to the storage service, we need to save the file details in our database. By doing this you won't have to query your storage service to find the user's uploaded files. It also allows you to obfuscate the original file name in storage while preserving it's original name for presenting it to the user.

This step is common for both S3/Hetzner and Cloudflare implementations as we're no longer using any S3-specific functionality:

export const action = async ({ request }) => { const session = await getSession(request); const formData = await request.formData(); const key = formData.get("key"); const originalFileName = formData.get("originalFileName"); // Save userId, originalFileName and key combination in database somewhere return json( { success: true }, ); };

Improving the UX

Right now the only validation is done by the actual storage service, which isn't great UX. In reality you'll probably need to validate the file client-side before attempting to upload, and catching errors from the storage provider to show errors to the user if relevant.

This is out of scope for this article, but an example of you could implement validation on the file type and size would be in the handleSubmit method

const handleSubmit = async (event) => { // ... (previous code remains the same) if (file && loaderData.presignedUrl) { if (!ALLOWED_FILE_TYPES.includes(file.type)) { alert(`Only ${ALLOWED_FILE_TYPES.join(", ")} files are allowed.`); return; } if (file.size > MAX_FILE_SIZE) { alert("File size must be less than 1KB."); return; } // ... (rest of the function remains the same) } };

Similarly, it's good UX to support a file upload dropzone. With a dropzone users can drag files from their file explorer into the browser window instead having to click to open the file uploader and select manually.

Comparison of S3/Hetzner and Cloudflare Implementations

To summarize the key differences between the S3/Hetzner and Cloudflare implementations, here's a comparison table:

FeatureS3/HetznerCloudflare
Presigned URL generationcreatePresignedPost()getSignedUrl() with PutObjectCommand
Upload methodPOSTPUT
PermissionsGranular via conditionsLimited, simpler approach
Form data structureComplex (multiple fields)Simple (file only)
Client-side ImplementationRequires handling multiple fieldsStraightforward PUT request
File size/type restrictionsServer-side via conditionsClient-side validation required

If you're looking for a SaaS starter kit with S3-compatible storage, check out Launchway. As well as S3-compatible storage provider for file uploads, Launchway also offers authentication, subscriptions, database connections, tons of UI components and more to get your SaaS started.

me
Brian BriscoeCreator of Launchway