Anyone still thinking about ElasticSearch now Jeff has started with Algolia?

It's the same as with any self-hosted vs SaaS discussion. Have self-hosted Beanstalkd vs Iron.io, have self-hosted ElasticSearch vs Algolia (or any other).

If you host it, then you have to set up everything. If you've got the money, just go with SaaS as it'll just work and you don't have to worry about it anymore!

I'm still on the ElasticSearch train and would love more on it. I have it working but I am not 100% happy with my implementation so far. Things like soft deletes, scopes, and other nuances are giving me problems in figuring out how to best implement them. Algolia looks awesome, but I have almost 1 million records to index (growing by 15k a month) so pricing would not be favorable. But I think the series will give me tips on how to better structure my implementation so watching attentively.

Algolia is amazing but you need to pay. Elastic is amazing but most probably will eat your memory....

If you just need a full text search that is fast and really simple consider using SphinxSearch (http://sphinxsearch.com/)

Not completely relevant to this topic, but since Laracasts is also covering some sidetracks related to development some sweet video's about https://prediction.io/ would also be fun to watch. To me atleast.

Just curious what is the difference between elasticsearch, and a reqular search of a database, like:

return $this->_db->select("SELECT * FROM ".PREFIX."pets WHERE petname like :search ORDER BY petname " . $pagingQuery, array('search' => $petsearch)); }

Then getting the results and looping over them in the view.

@jlrdw While MySQL is database management system itself, ElasticSearch is only a search engine. ElasticSearch is super-fast and provides much better full-text search support.

ElasticSearch is also more typo-friendly, which is a huge thing. If you search users table and search for user "Jonh" (and it should've been "John"), ElasticSearch will return all the relevant data nevertheless, based on a score-system.

Where MS SQL Server searched my data (by simple query) for more than 1 minute, ES is doing it in less than 20 ms (even by using very complex query).

I'll surely still on Elasticsearch. I currently have an 3 Million reccords index on ElasticSearch costing $40 mo hosted at DigitalOcean. With Algolia, maybe I have a simplest usage, simplest development, but costing $659 mo.

My application don't have the size to pay this only at search engine. No way.

@EliasSoares Yikes!! Yep you're right looking at the pricing structure.

Algolia are certainly an exciting company. They've just raised $18.3M of investment to expand internationally and develop products. Will be interesting to see if their pricing alters over time.

Source: http://www.marketwired.com/press-release/algolia-raises-183m-series-a-investment-led-by-accel-partners-2021539.htm

@jlrdw MySQL works fine when doing a simple search and you don't need fast responses.

My case, I have a City Search Typeahead that find a city in a 3 Million records city database.

Using MySQL, this search takes more then 5 seconds just for searching for "new york". Using ElasticSearch, I have responses in 50ms, with a very better relevancy order since I can do everything, including Fuzzy Search, autocompleting and more.

@olimorris Surely. Maybe thay don't want to have small clients like most of us. A single "shared" server for most of us is enough. :D

good point. i was leaning towards elasticsearch and still probably will, but will be interesting to learn about how to implement algolia.

Hey all. I'm the developer of the ES-PHP client and engineer at Elastic. This thread popped up in my inbox so I thought I'd swing by and make myself available for questions.

@noeldiaz Feel free to ping me here or on twitter (@ZacharyTong) if you're having issues with Elasticsearch, would be more than happy to help out.

Obviously, I work at Elastic so I'm a bit biased. So take my opinion with a grain of salt :)

Now Jeff is covering Algolia, I'm struggling to see (apart from the pricing) why I would consider using ElasticSearch.

I don't have direct experience with Algolia, but it looks like a nice product. They have tackled a relatively narrow, but very important, problem: search + autocomplete and a suite of integrations to make implementing search relatively plug-and-play. Combined with the SaaS offering, this makes implementing basic search very easy...especially if you just want to implement a search solution for a relatively small dataset.

Because Elasticsearch is a server, not a service, you have complete control over your implementation. It strives to give sane defaults so you can get "Good" results with minimal configuration. But critically, you can tweak your queries/data to get "Great" or "Outstanding" results. It does come with the cost of a learning curve, but there are fewer limitations than closed-box solutions that you don't have control over.

Many people aren't aware of the cool, custom functionality ES is capable of. Off the top of my head, a few interesting use cases:

• Several news agencies use Elasticsearch to rank news articles such that their relevance is exponentially weighted against the age of the article. This is super important, since articles that are more than just a few hours old start losing relevance very quickly in the news business.
• Related, the Guardian gave a very cool talk about their in-house tool, Ophan, which uses ES to track engagement. It's basically a Google Analytics on steroids, allowing all their editors and journalists to monitor article activity in realtime and adjust how they promote articles through various channels.
• Hotel rental sites are using ES to sort results such that lodgings are exponentially less important as they fall away from a geo-point, but are also boosted if they have XYZ features (wifi, etc), a better star rating, or an active promotion.
• One company wrote a custom tokenizer to turn images into tokens (edges, features, etc), which allows them to do image search
• Another company is serving machine learning recommendation models out of Elasticsearch using the function_score query (by basically combining a few dozen linear functions as coefficients to their model). Results are found via regular full-text search, then the top-N results are re-sorted using the Rescore Query based on the ML model to provide personalized recommendations
• Various companies only use ES for the percolator feature, which is basically like an alerting engine. One in particular replaced a large enterprisy rules engine with ES percolator. On a plane ride once, I built a random-forest decision tree ML model using the percolator
• Innumerable companies are using ES as a log storage and analytics engine in combination with Logstash and Kibana. My largest customer has nearly a petabyte of log data in ES, and there are other clusters in the wild with more than that.

So yeah, I'd say there is still plenty about Elasticsearch that might interest you. It just depends how much control and performance you need over your data. There isn't really a right or wrong answer, just whichever tool makes the most sense for your problem :)

As an aside, Elastic recently acquired Found which provide a SaaS version of Elasticsearch. And it probably goes without saying, but since Elasticsearch is OSS (Apache 2 license) the company could go belly-up tomorrow and the software would survive on.

Thanks for the information @polyfractal. No real issues with Elastic so far. We have been running it now for around 6 months with great success. Moving over from Solr, the web team really likes how easy it is to get projects going with it. What I mostly meant on my previous message was that our implementation is very cobbled up right now (mostly in how we do queries and the faceting part) so we are working on streamlining the code and making it more modular to be able to reuse it better. But it works. And it is very fast. Granted, I am sure we are just scratching the surface of what it can do, but we have a few ideas for other implementations that might get interesting (mostly in the geo coding side of things). Got way more ideas than time. :)

@noeldiaz Cool, glad to hear it! Keep me updated as you guys progress, or if you run into troubles. We love to hear what people are using ES for.

And I absolutely know what you mean about more ideas than time. It feels like plane rides are the only time I have to work on fun proof-of-concepts :)

Since you mentioned geo, we semi-recently hired a guy who is working on Geo stuff full time, both at the ES and Lucene level. He's doing a lot of really cool stuff. Recently improved geo-point compression by something ridiculous, like a 40 fold increase. I think he's currently working on new geo aggs and a modular projection system, so you aren't stuck with whatever the default projection is (which will also allow people to start using 3D coordinates, polar coords, etc). Lotta neat stuff :)

@noeldiaz @polyfractal I read all the posts,and I am doing some work and want to introduce fulltext search,so I found elasticsearch and algolia,at first glance,I chosed algolia,it's really easy to use and have good performance ,documents and tools. I have to say it's a very good product for normal project.

but,but after several days,when I have read most of algolia documents and examples and finished most codes,I found it's price is expensive,very expensive,for a startup business that's unacceptable ,because maybe several month I can't make money,but have to pay lots of money to SaaS,unbearable!

So now I turn back to Elasticsearch,and being prepare to read the document and guide,though now I have to work more than algolia,hope to it's worth to do these in the long term. Good luck for me :)

I'm getting ready to start looking into setting up elasticsearch at work.

We're migrating a bunch of sites from drupal and joomla to a custom CMS and I want to index all the articles from all the sites in elasticsearch.