Multipart request with Pool

Hi, I'm trying to do a function to load images into cloudinary in Laravel 8. As the SDK doesn't allow to batch upload, I thought I should give a try to concurrent requests using Http::pool(). I can send the requests sinchronously, appending images, generating signature, etc, etc. The problem arises when I start using Pool, that I get the same response 3 times (because I send 3 images). I already checked with logs that I'm getting the right image and generating the right payload, and also tested with another API (although they were only GET requests) that the logic for the pool generation is right (I got different responses for every promise, as expected).

public function uploadImages(array $images): array
    {
        $urls = [];
        $uploadPreset = env("CLOUDINARY_UPLOAD_PRESET");
        $cloudinaryFolderName = env("CLOUDINARY_FOLDER_NAME");
        $resourceType = "image";
        $uploadUrl = env("CLOUDINARY_UPLOAD_URL");
        $apiPublicKey = env("CLOUDINARY_API_PUBLIC_KEY");
        $transformations = "q_70,w_350";
        try {
            $callback = function (Pool $pool) use ($images, $uploadPreset, $cloudinaryFolderName, $transformations, $apiPublicKey, $resourceType, $uploadUrl) {
                $promises = [];
                foreach ($images as $image) {
                    $timestamp = Carbon::now()->timestamp;
                    $publicId = "Image" . $timestamp;
                    $signature = $this->generateCloudinarySignature(
                        [
                            "upload_preset" => $uploadPreset,
                            "folder" => $cloudinaryFolderName,
                            'transformation' => $transformations,
                            "public_id" => $publicId,
                            "timestamp" => $timestamp,
                        ],
                        $apiPublicKey
                    );

                    $imageExtension = $image->getClientOriginalExtension();
                    $base64Image = "data:image/" . $imageExtension . ";base64," . base64_encode($image->get());

                    $formData = array(
                        "file" => $base64Image,
                        "upload_preset" => $uploadPreset,
                        "folder" => $cloudinaryFolderName,
                        'transformation' => $transformations,
                        "public_id" => $publicId,
                        "resource_type" => $resourceType,
                        "timestamp" => $timestamp,
                        "api_key" => $apiPublicKey,
                        "signature" => $signature
                    );

                    $promises[] = $pool->asMultipart()->post($uploadUrl, $formData);
                }
                return $promises;
            };

            $responses = Http::pool($callback);

            foreach ($responses as $response) {
                if ($response->successful()) {
                    $responseBody = $response->json();
                    $urls[] = $responseBody;
                } else {
                    Log::error("Error uploading image", [$response->json()]);
                }
            }
        } catch (\Throwable $e) {
            Log::error("Error uploading images", [$e->getMessage()]);
            throw new \Exception("Error uploading images");
        }
        if (empty($urls))
            throw new \Exception("Error uploading images - No images were uploaded");

        Log::info("Finished uploading images", [$urls]);
        return $urls;
    }

Could it be somehow related to the Multipart request?

As I receive the exact same response for every request, I assume the promise is not being saved correctly. Has anyone faced a similar issue?

Tray2

2 years ago

Level 73

I played around with image uploads from regular JS, maybe you can use something like it-

function getElement(selector) {
    return document.querySelector(selector);
}

async function uploadFiles(e) {
    e.preventDefault();
    const images = getElement('#images');

    let stats = {
        noOfImages: images.files.length,
        loop: 1,
        counter: 0,
        maxFiles: 20,
    };

    while (stats.noOfImages > 0) {
        if (stats.noOfImages < stats.maxFiles) {
            stats.maxFiles = images.files.length;
            stats.loop = 1;
        }

        let formData = await createFormData(images, stats);
        await upload(formData);
        stats.noOfImages -= stats.maxFiles;
    }
    getElement('#images').value = null;
}

async function upload(formData) {
    try {
        const response = await fetch('/api/uploads', {
            method: 'POST',
            body: formData
        });

        const result = await response.json();
        displaySuccess(result);
    } catch (error) {
        displayError(error);
    }
}

async function createFormData(images, stats) {
    let formData = new FormData();

    formData.append('_token', getElement('#_token').value);
    formData.append('user_id', getElement('#user_id').value);

    for (let i = stats.counter; i < stats.maxFiles * stats.loop; i++) {
        formData.append('images[]', images.files[i]);
        stats.counter++;
    }

    stats.loop++;
    return formData;
}

What it does, is that it takes the files from a file input, then batch them twenty and twenty to keep the upload size and amount of files allowed inside the limits.

The html

<form action="{{ route('uploads') }}" method="post" enctype="multipart/form-data">
        <input type="hidden" name="user_id" value="1" id="user_id">
        <input type="hidden" name="_token"  id="_token" value="{{ csrf_token() }}" />
        <input type="file" name="images[]" id="images" multiple>
        <input type="submit" value="Upload" id="submit">
    </form>

The php used

 public function __invoke(UploadsFormRequest $request)
    {
        $validData = $request->validated();
        $images = $request->file('images');
        $imageData['user_id'] = $validData['user_id'];

        foreach ($images as $image) {
            $imageData['original_name'] = $image->getClientOriginalName();
            $imageData['stored_name'] = Str::replace('uploads/', '', $image->store('uploads'));
            Upload::create($imageData);
        }

        ProcessUploadedImagesJob::dispatch($validData['user_id']);

        return response()
            ->json([
                'status' => 'Success, your files will be available shortly',
            ]);
    }

Please or to participate in this conversation.