withoutSyncingToSearch() - algolia - laravel media library thumbnail image indexing issue

@trevorpan pass a variable to a callback by reference:

$imageUrl = null;

SlugJob::withoutSyncingToSearch(function () use ($job, &$imageUrl) {
   $imageUrl = $job->getMedia('document')->first()->getUrl('thumbnail');
});

What you are trying to do there? withoutSyncingToSearch method is used for executing the code without syncing model changes to the search index, but you're only getting the data here.

Hi thank you @sti3bas

Gave this a run and am getting this error still:

Too few arguments to function App\Job::App\{closure}(), 0 passed in /Users/trevorpan/code/bidbird/vendor/laravel/scout/src/Searchable.php on line 216 and exactly 1 expected

Now I'm thinking I'm not doing what needs to be done. The original goal withoutSyncingToSearch() was to find the job being stored in the form (prior to submitting to algolia) and then identify and attach it's path to the array which is sent as $imageUrl to algolia on the Job index not the Media index.

(not sure if you know that package but when using MediaLibary natively we can call $job->getMedia('document')->first()->getUrl('thumbnail'); and get the image in blade. But when sending it for index to algolia those relations are lost- which is why I tried concatenating things above in the image path)

The end goal is to be able to say something like this line in the Vue Jobs Search component

<img :hit="item" :src=“item.imageUrl” :hit="item" width=200 height=120   alt=""/>

This would be ideal as it only uses one algolia index Job.

The original goal withoutSyncingToSearch() was to find the job being stored in the form (prior to submitting to algolia) and then identify and attach it's path to the array which is sent as $imageUrl to algolia on the Job index not the Media index.

You should not have Media index if you haven't added Searchable trait to your Media model. So you could do whatever you want with Media model and the changes will not be synced to search index.

You just need to return an array from toSearchableArray method with the data you want to be synced to search index.

That's how I'm doing it in one of my projects:

public function toSearchableArray()
{
   $thumbUrl = optional($this->getMedia('profile-photo')->first())->getUrl('thumb');

   return [
      'id' => $this->id,
      'name' => $this->name,
      'photoThumbUrl' => $url ?: '/images/profiles/default-photo.svg',   
   ];
}

Then in my Vue component I can show the image:

<img :src="member.photoThumbUrl" alt="profile-photo">

Hi there @sti3bas

That's exactly what the goal is. Spent some time looking through Media model, actually had deleted the toSearchableArray(). So please pardon the confusion there. That was an initial attempt.

So pleased at what you've shared above. The optional image shows up just great and is indexed to Algolia.

However, the Spatie code returns null. I tried using self::getMedia etc but that returned null too.

Is the toSearchableArray() something that should be put to a queue to delay the function?

I'm thinking because the model may not be saved and presented with a view via a controller the getMedia line is not able to locate the path??

Thank you again, Trevor

Is the toSearchableArray() something that should be put to a queue to delay the function?

Are you using queue for your media conversions? If so, it will return null when creating new record because image is not processed yet.

You can add an event listener for Spatie\MediaLibrary\Events\ConversionHasBeenCompleted to update search record with $event->media->model()->searchable(); after conversation has been completed.

I found it hard to track all these automatic syncs when I was working on one of my projects. Some of my controller actions was triggering multiple syncs with the same data, which is bad because Algolia charges you for the records and operations. I ended up managing syncing manually and also wrote a package which helped me to write tests which ensures that syncing is only triggered once with the correct data. You can find it here, if you're interested: https://laravel-news.com/laravel-scout-array-driver-for-testing

Hi there. That's awesome, I'll check that out today.

Not currently using a queue, just thought it might be something worth looking at.

Had another possible solution was to use MySQL views: https://stitcher.io/blog/eloquent-mysql-views

Maybe sending the data we need to a view and then indexing that model is a possibility. Have a post on the MySQL forum - a regular there said MySQL views can be updated, unlike the blog above says.

I'll let you know what ends up working! Thank you -

Hi @sti3bas

I found this method wait() from a forum manager on Algolia and it allowed for the media to process, and by Joe, has sent the image path to algolia. I gave it a shot without using an image and your handy helper gave it the default image.

Is this too good to be true?

$thumbUrl = optional($this->getMedia('document')->first())->getUrl('thumbnail')->wait();

Not currently using a queue, just thought it might be something worth looking at.

I'm not sure what are you trying to solve as $this->getMedia('document')->first()->getUrl('thumbnail') should always return a url (if you upload the photo), because you're not using queue.

I found this method wait() from a forum manager on Algolia and it allowed for the media to process

getUrl method returns a string and I don't see any wait method in medialibrary package, so I'm not sure what you are talking about too.

@sti3bas actually, have implemented a queue since we last chatted..

Here's the wait() https://www.algolia.com/doc/api-reference/api-methods/wait-task/

I'm thinking that's why that method is available on my implementation. Not sure if it's a bulletproof use case, but I do know with certainty the $thumbUrl is sent to algolia, where before it was not pulling the url to send to the index.

Have you tried this out?

@trevorpan sorry, but I still don't understand. As I said getUrl return a string, so you can't call methods on it (also medialibrary has nothing to do with Algolia): https://github.com/spatie/laravel-medialibrary/blob/master/src/Support/UrlGenerator/DefaultUrlGenerator.php#L10

I think you're mixing something.

@sti3bas

What I'm doing is getting the path for the thumbnail and sending to algolia in the Job index. Algolia doesn't allow for two indexes to be searched in the way the DB does — with joins e.g..

So, what I was doing was delaying the index method on Job so Media could be stored then retrieving the string url so it could be sent to algolia under the one index Job (in this case).

Because I have the algolia-php package I believe that's why wait() allows the media to be stored before returning the index array.

I'm still not certain it will always work (in production) but at least in the few tests I've done it performs well. Does that help clarify? Are you able to do this to?

Well @sti3bas I come hat-in-hand. You were right. After clearing the cache:

Call to a member function wait() on string

So, I've gone back and trying to implement your code base Sti3bas / laravel-scout-array-driver.

I'm having a hard time figuring out how to upload an image in addition to creating the job.

e.g. spatie laravel media-library uses the Media model - because the image path is not saved in the job model (by Spatie design)

        $response = $this->actingAs($user)
            ->post('/jobs', $this->validParams(['document' => UploadedFile::fake()->image('lumber.jpg')]))
            ->assertRedirect('/searchjobs');

This error ErrorException: mb_strlen() expects parameter 1 to be string, object given is upset, I believe, that the file is an object and not the document name. It'd be easy to upload a string, but then the Media model would not have been created.

I've gone back to Adam Wathan's Test Driven Development course on uploading concert pictures, but the scenario is a bit different.

This is what I've got so far:

/** @test */
public function a_job_is_indexed_to_algolia()
{
$this->withoutExceptionHandling();
        Event::fake();
        Mail::fake();
        Mail::assertNothingSent();
        Search::assertEmpty();
        Storage::fake('media');

        $user = factory(User::class)->create();

        $job = $this->actingAs($user)
            ->post('/jobs', $this->validParams(['document' => UploadedFile::fake()->image('lumber.jpg')]))
            ->assertRedirect('/searchjobs');

        // this is where the error is occuring above.

        Storage::disk('media')->assertExists('lumber.jpg');

//        $thumbUrl = optional($job->getMedia('document')->first()->getUrl('thumbnail')); // your line

        $job2 = SlugJob::withoutSyncingToSearch(function () {
            return factory(SlugJob::class)->create([
                'body' => 'Concrete',
            ]);
        });

//        Search::assertSynced($job);
//        Search::assertEmpty($job2);


dd($job);

//
        Search::assertContainsIn('jobs', $job)
        ->assertNotContains($job2)
        ->assertContains($job, function ($record) {
            return $record['body'] === 'Sealants';
        })
//        ->assertContains($job, function ($record) {
//            return $record['thumbUrl'] === $thumbUrl;
//        })
        ->assertNotContains($job, function ($record) {
            return $record['body'] === 'Concrete';
        });
    }

I've gone back to laravel media library to try and see what they do, but if I'm not mistaken, I shouldn't be concerned about the uploading functionality, just when the search index is sent to algolia.

At this point, I've got queues working, which seem to have a greater ability to save the image path in time for indexing, but I'd like to see what you do with events and how you prevent the syncing before the entire index is ready to send to algolia - if you can share `

Thank you

@sti3bas geez, finally after weeks this has started to make sense.

Still don't have a full handle on your package, but can work on that in the future. (A few extra records charges won't matter at this stage of the site).

However (as you mentioned above)

// event service provider
        ConversionHasBeenCompleted::class => [
            AlgoliaJobIndexConversionHasBeenCompletedNotification::class,
        ]
    ];

// listener
    /**
     * Handle the event.
     *
     * @param ConversionHasBeenCompleted $event
     * @return void
     */
    public function handle(ConversionHasBeenCompleted $event)
    {
        $event->media->model()->searchable();
    }

A sincere thank you~