ashwinmram's avatar

Replacing Algolia with Typesense for fuzzy searching

I was using Algolia and then did some math and expected to pay more than I intend to charge monthly for my app to Algolia in the long run, so decided to look into alternatives. Elasticsearch was the first thing that came to mind, but it seemed to have a steep learning curve. Then I stumbled on Typesense and it was literally an answer to my prayers. I had prayed for a solution that will allow me to swap out Algolia without having to obtain a rocket science degree in the process, while also being able to use the Vue Instantsearch components I was already using in my app. Typesense and its typesense-instantsearch-adapter were the answers to my prayers. I spend a couple hours tinkering with the new tools this morning and swapped out Algolia and everything works just like it did with Algolia :)

I'm going to share the changes I made below.

0 likes
4 replies
ashwinmram's avatar

Here are the steps I went through to replace Algolia in my App.

Install the Scout adapter for Typsense composer require devloopsnet/laravel-typesense

Add the service provider in the providers array in config/app.php: Devloops\LaravelTypesense\TypesenseServiceProvider::class

Update .env to use the new driver: SCOUT_DRIVER=typesensesearch

I used Typsesense Cloud to spin up a server as it has a free tier. The paid tier is reasonable with no record/operation based charges or overages. If you wish to set Typesense up on your own server you can either use their docker image or install binaries for Os X or Linux. You can read more here:

https://typesense.org/docs/0.17.0/guide/#install-typesense

Spinning up a server is a no brainer and takes about 5 minutes, but let me walk you through what to do once it is spun up. You will need to click Generate API Keys and it will give you the information below.

=== Typesense Cloud: servername ===      

Admin API Key: 
adminkey-api-string

Search Only API Key: 
searchkey-api-string

Nodes:
- servernode-abcxyz.a1.typesense.net [https:443]

Publish the Scout Service Provider if you haven't already done so: php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"

You can then go to config/scout.php and add the following below the entry for Algolia:

    'typesensesearch' => [
        'api_key'         => 'adminkey-api-string',
        'nodes'           => [
            [
                'host'     => 'servernode-abcxyz.a1.typesense.net',
                'port'     => '443',
                'path'     => '',
                'protocol' => 'https',
            ],
        ],
        'nearest_node'    => [
            'host'     => 'servernode-abcxyz.a1.typesense.net',
            'port'     => '443',
            'path'     => '',
            'protocol' => 'https',
        ],
        'connection_timeout_seconds'   => 2,
        'healthcheck_interval_seconds' => 30,
        'num_retries'                  => 3,
        'retry_interval_seconds'       => 1,
    ]

The next steps are what took me a little bit of trial and error to figure out as not all parts were illustrated clearly in the Typesense driver package documentation.

I'm assuming you are already using Algolia so you should have a model that implements the Searchable trait. You now need to have the class also implement TypesenseSearch:

class Guest extends Model implements TypesenseSearch

You will need to add the following 3 methods with a snippet from my code base for illustration. The first is the getCollectionSchema method.

public function getCollectionSchema(): array {
      return [
        'name' => 'guests',
        'fields' => [
            [
                'name' =>'last_name',
                'type' => 'string'
            ],
            [
                'name' =>'first_name',
                'type' => 'string'
            ],
            [
                'name' =>'interests',
                'type' => 'string[]'
            ],
            [
                'name' =>'tenant_id',
                'type' => 'int32',
                'facet' => true // You can use facets in Typesense just like in Algolia by setting the facet boolean to true
            ]
        ],
        'default_sorting_field' => 'tenant_id' // This is required for now so I just used the tenant id
      ];
    }

The second method is the typesenseQueryBy method.

public function typesenseQueryBy(): array {
      return [
        'last_name',
        'first_name',
        'interests'
      ];
    }

You should already have a toSearchableArray() function, but I made the following tweaks. Typesearch expects most of the fields to be returned as strings, except for fields that are used for sorting as numbers (see tenant_id above) or as string arrays (see interests field above).

I tweaked my toSearchableArray function as follows:

    public function toSearchableArray()
    {
        $skipStringCoversion = ['tenant_id','interests'];
	
	return collect([
            'id' => $this->id,
            'tenant_id' => $this->tenant_id,
            'first_name' => $this->first_name,
            'last_name' => $this->last_name,
            'interests' => $this->interests()
        ])->map(function ($value, $index) use ($skipStringCoversion) {
            if(in_array($index, $skipStringCoversion, true)) {
                return $value;
            }

            return (string) $value;
        })->toArray();
   }

With this tweak you should be able import documents to your Typesense server.

php artisan scout:import App\\Guest

After that you Tinker and search:

Guest::search('Ashwin')->get();

That takes care of the back-end stuff... let's move to the front-end. I was using Vue Instantsearch after watching one of Jeffrey's videos and now just needed to swap in Typesense. In comes the adapter made by the Typsesense team.

npm install --save typesense-instantsearch-adapter

In my app I was injecting the searchClient while constructing my Vue root app:

import algoliasearch from 'algoliasearch/lite';
const searchClient = algoliasearch('ABC123DEF', 'algolia-search-api-key');

const app = new Vue({
   el: '#app',

   data() {
     return {
       searchClient
     };
   }
})

I replaced this code with the below:

import TypesenseInstantSearchAdapter from "typesense-instantsearch-adapter";

const typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
  server: {
    apiKey: "searchkey-api-string", // Be sure to use the search-only-api-key
    nodes: [
      {
        host: "servernode-abcxyz.a1.typesense.net",
        port: "443",
        protocol: "https"
      }
    ]
  },
  // The following parameters are directly passed to Typesense's search API endpoint.
  //  So you can pass any parameters supported by the search endpoint below.
  //  queryBy is required.
  additionalSearchParameters: {
    queryBy: "last_name,first_name,interests"
  }
});

const searchClient = typesenseInstantsearchAdapter.searchClient;

const app = new Vue({
   el: '#app',

   data() {
     return {
       searchClient
     };
   }
})

I made absolutely no other changes to my Laravel view where I leveraged the Instantsearch components for searching, refinement, results etc. and everything just worked exactly the way it did before. At this point my mind was blown!

The only issue for me was because I was using a global search client, the additionalSearchParameters were specific to my model. I have to figure out a way to generalize the queryBy fields... perhaps one of you can help me do this in a clean way without duplicating code.

I'm super excited about this and didn't want to waste any time before sharing it with this awesome community we are a part of :)

3 likes
ashwinmram's avatar

Some more lessons learned during my migration from Algolia to Typesense.

I was using the following configure code in Vue Instantsearch.

<ais-configure
          filters="hotel_id:1 AND status:checked_in OR status:due_out"
></ais-configure>

The current version of the typesense adapter doesn't convert filters, but you can use a facet-filters array instead.

https://www.algolia.com/doc/guides/managing-results/refine-results/filtering/in-depth/filters-and-facetfilters/

There are some subtleties around the syntax in conjunction with the typesense adapter, so the above code snippet translated into the one below.

<ais-configure
      :facet-filters.camel="['hotel_id:1',['status:checked_in','status:due_out']]"
></ais-configure>

I also didn't want to duplicate my searchClient code because I needed different queryBy fields for each model, so this is what I settled with instead.

  1. Create a generic component to use in my views
<template>
  <div>
    <slot v-bind:searchClient="searchClient"></slot>
  </div>
</template>

<script>
import TypesenseInstantSearchAdapter from "typesense-instantsearch-adapter";

export default {
  props: ['queryBy'],

  data() {
    return {
      typesenseInstantsearchAdapter: '',
      searchClient: ''
    }
  },

  created() {
    this.typesenseInstantsearchAdapter = new TypesenseInstantSearchAdapter({
      server: {
        apiKey: "searchkey-api-string", // Be sure to use the search-only-api-key
        nodes: [
          {
            host: "servernode-abcxyz.a1.typesense.net",
            port: "443",
            protocol: "https"
          }
        ]
      },
      // The following parameters are directly passed to Typesense's search API endpoint.
      //  So you can pass any parameters supported by the search endpoint below.
      //  queryBy is required.
      additionalSearchParameters: {
        queryBy: this.queryBy
      }
    });

    this.searchClient = this.typesenseInstantsearchAdapter.searchClient;
  }
}
</script>
  1. I then slipped in the query by parameters as a prop and pulled up the searchClient using object de-structuring like this:
<model-index v-slot="{ searchClient }"
  query-by="last_name,first_name,interests"
>
  <ais-instant-search
    :search-client="searchClient"
    index-name="guests"
  >
  // ...
  </ais-instant-search>
</model-index>

My original Algolia code was very similar without the model-index component wrapper.

  <ais-instant-search
    :search-client="searchClient"
    index-name="guests"
  >
  // ...
  </ais-instant-search>
ashwinmram's avatar

I just set up a local instance of the typesense server on my Mac and added the following because I am using Valet secure which creates a self-signed certificate for local development. I'm not sure if all the steps are required but this is what I needed to do.

  • touch /tmp/tyesense-data

  • ./typesense-server --data-dir=/tmp/typesense-data --api-key=specifywhateveryouwant --ssl-certificate=/Users/<username>/.config/valet/Certificates/domain.test.crt --ssl-certificate-key=/Users/<username>/.config/valet/Certificates/domain.test.key --enable-cors

The --enable-cors flag is crucial! Don't add a boolean at the end.

  • I actually had to update the /etc/hosts to update my domain

  • Add the certificate to OS X but using Finder and clicking Go to Folder under the go menu and typed ~/.config/valet/Certificates and then double clicking the crt file

  • I used Tinkerwell and executed openssl_get_cert_locations() to find where the .pem being referenced is located. In my case it was located here: /usr/local/etc/[email protected]/cert.pem

  • I then added the Keychain certs to this location

sudo sh -c 'security find-certificate -a -p /Library/Keychains/System.keychain >> /usr/local/etc/[email protected]/cert.pem'

sudo sh -c 'security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain >> /usr/local/etc/[email protected]/cert.pem'

I was then able to use Laravel Scout to update the documents into Typesense using the schema specified on the model

1 like

Please or to participate in this conversation.