Livewire 3 + FilePond Uploads: Large Files Failing

I am using Livewire 3 and FilePond on a project for a video production company. Their clients will be uploading very large files (up to 10GB). Things go as planned until the file size reaches about 100MB then the process fails. The FilePond upload indicator just spins and the file is never processed.

Using Forge, I have increased the Max File Upload Size to 10000 (10GB) and have also edited the temporary_file_upload key in the livewire.php config:

'rules' => ['file', 'max:10000000']

FilePond is modeling a $files array on the component and I watch that property for updated events:

    public function updatedFiles()
    {
        $this->validate(
            [
                'files.*' => 'file|mimeTypes:video/*,audio/*,text/*,image/*,application/pdf',
            ],
            [
                'files.*.mime_types' => 'Allowed file types: video, audio, text, image, PDF.',
            ]
        );

        foreach ($this->files as $file) {
            // Avoid duplicates
            if (in_array($file, $this->processedFiles)) {
                continue;
            }

            $md5 = hash_file('md5', $file->getRealPath());

            if ($this->project->assets()->where('hash', $md5)->exists()) {
                $this->processedFiles[] = $file;

                continue;
            }

            $asset = new Asset([
                'user_id' => auth()->id(),
                'name' => $file->hashName(),
                'file_name' => $file->getClientOriginalName(),
                'path' => $file->store($this->project->id, 'assets'),
                'mime_type' => $file->getMimeType(),
                'extension' => $file->getClientOriginalExtension(),
                'size' => $file->getSize(),
                'disk' => 'assets',
                'hash' => $md5,
            ]);

            $this->project->assets()->save($asset);

            $this->processedFiles[] = $file;

            $this->project->load('assets');
        }
    }

I don’t know what else to try. Any advice is appreciated!

LaryAI

2 years ago

Level 58

It seems like you've already configured Livewire and FilePond to handle large file uploads, but there are a few other server settings that you might need to adjust to handle large files. Here's a checklist of things to consider:

PHP Configuration: Ensure that your php.ini file has the following settings configured to handle large uploads:

upload_max_filesize = 10G
post_max_size = 10G
max_input_time = 3600
max_execution_time = 3600
memory_limit = 12G

These settings will allow for 10GB file uploads and give a generous amount of time and memory for the script to execute. Adjust the memory_limit as needed, but it should be larger than post_max_size.

Nginx Configuration: If you're using Nginx, you need to configure the client_max_body_size directive in your Nginx configuration file:

client_max_body_size 10G;

This will allow Nginx to accept large file uploads.

Livewire Temporary File Upload Configuration: You've already configured the temporary_file_upload rules, but make sure that the disk you're using has enough space and is configured correctly in config/filesystems.php.
FilePond Configuration: Ensure that FilePond is configured to chunk the files if necessary. This can help with uploading very large files by breaking them into smaller pieces. Check FilePond's documentation for chunking uploads.
Browser and Network Limitations: Sometimes the issue might not be with your server configuration but with the client's browser or network. Make sure that there are no limitations on the client-side that could be causing the upload to fail.
Error Handling: Implement error handling in your Livewire component to catch any upload errors and display appropriate messages to the user.
Server Logs: Check your server logs for any errors that might indicate why the upload is failing. This could provide more insight into the issue.
Livewire Upload Cleanup: Ensure that Livewire's temporary uploads are being cleaned up properly to prevent the server from running out of space.
Test with Smaller Files: If you haven't already, test the upload process with smaller files to ensure that the basic upload functionality is working as expected.
Debugging: Temporarily add logging to your updatedFiles method to log each step and see where it might be failing.

If you've gone through all these steps and the issue persists, it might be helpful to reach out to the Livewire community or maintainers for more specific guidance, as there could be a bug or a limitation within Livewire itself when handling very large files.

Snapey

2 years ago

Level 122

post_max_size needs to be large enough to accommodate base64 encoding of the largest file

You wont be able to use Livewire uploads with filepond chunking

mikefolsom

2 years ago

Level 21

@Snapey Thanks for the heads-up regarding chunked uploads! I have tweaked every setting I can think of in Livewire, PHP-FPM, and Nginx. (Side note: Cloudflare also limits post sizes, so we had to disable proxying!) I have even modified the Livewire component so uploads are stored on the local filesystem then a job is dispatched to move them to S3, hoping that would reduce the request cycle length.

Large file uploads are working better, in that we have successfully uploaded files as large as 824MB. But we are still having issues, especially uploading from phones. Sometimes they just hang or fail for no discernible reason.

So my question remains: can Livewire/FilePond handle very large file uploads, or do I need to look for another solution? In a previous project I used Dropzone JS and signed URLs to upload directly to S3 but that front-end stack was radically different. 🙁

malzariey

1 year ago

Level 1

@mikefolsom Thanks mikefolsom, Didnt know that cloudflare restricts file upload sizes.

Please or to participate in this conversation.