Using Redis as main data source

Hi KONSTANTINRACHEV,

There is a lot to think about if you plan on using Redis. I'm no expert but I will try and answer you to the best of my knowledge.

First thing first, Redis is an in memory key value store. It's not capable of being queried in the same way a MySQL database can be. So keeping this in mind, you need to know the key of what ever you want to retrieve. A lot of thought will need to be put into your naming convention because of this.

Some gotchas or caveats, as it's in memory, the amount of memory available on the server will limit the amount of data you can store. Out of the box, it also doesn't write anything to disk. You can enable snapshots which will write a copy of what's stored to disk on a time schedule or when a certain number of changes have been made. Without this, if your server reboots, you will lose everything in memory. With it, it will take a while for your Redis server to recover when it restarts. You will lose anything that was changed since the last snapshot.

Redis is generally used for caching and not a standalone storage service. My use of Redis uses MySQL as the main storage engine and Redis for reading. So the workflow would be:

Write to MySQL followed by writing to Redis. So anything that gets updated in MySQL also gets updated in Redis. From a reading point of view everything is read from Redis first, if it isn't there I will read it from MySQL and write it to Redis. With this method it takes less time to recover if your Redis server goes down. Reads will be a bit slower to start with, but at least the data will be available straight away.

Redis is a fantastic layer to add for caching, a typical MySQL query might take 200ms, pulling the same data from Redis will take 10-20ms, so it is great for speeding sites up.

Hopefully this gives you some idea of how to get around failures. If you have any further questions, I will try to answer them.

Neil

@neilherbertuk thanks! After a dozen of articles about Redis and the combined usage with MySQL, I changed my mind and take about the road of using traditional SQL with Redis as caching engine.

Two days ago, there were some issues with Redis as the attempted connection on my hosting was not active and unable to read the data. So, going with Redis and MySQL.

I am interested, do you use Redis to cache pages as pure HTML, as it is speeding up the render.

Thanks in advance.

That's not something I use it for, but there is no reason why you can't. However, it would make your writing mechanisms longer as you would need to regenerate your entire HTML to include the change, otherwise the end user wouldn't see it. Laravel has its own caching system which works with Redis out of the box - https://laravel.com/docs/5.5/cache - though you do need to pull in another package to enable it.

I might add that Redis might be overkill for your needs. Oh and btw the workflow I mentioned is how Facebook works, though they use memcached instead of Redis.

Neil

Thanks! I am currently using the Cache facade over the Redis one. Definetely, Redis is speeding the situation up, as the site is over 250 pages for now and more and more are going to be added.

I know, 250 are nothing, but the architecture is more complicated and the need for speed(huh :)) was something I intended to look for.

The HTML caching is something I would like to use for pages, that are not target of everyday updates - the about us one, the contacts one, etc.

I am also interested, as I am newbie to Redis itself, which one do you prefer - caching all the records from the database for limited time or forever is the first one or just letting them cache when someone tries to open the record and then you just remember from the database to the cache instance?

It would completely depend on use case and what your internal requirements are. What sort of data are you using, how often it would be accessed.

If your need for Redis is about hitting below a certain time to respond to the user, then yes, cache everything. However, it would be pointless to cache data that doesn't get used very often, as you would be dedicating extra resources to store it both in MySQL and again in cache.

Problem is as your data grows, so will your caching infrastructure. Which will lead to potentially unnecessary cost of servers, the more resources you throw at Redis, obviously the higher the operating costs become. If you want to hit sub certain times across multiple regions you'll also need to consider Redis clustering and routing to the nearest data centres etc.

I don't personally prefer either way. It really does depend on use. Such as multi-tennant applications, it makes sense to cache data that will be accessed globally such as config or site specific branding or such forever, but caching data that a single user might access once a day, or only a few times a week doesn't make much sense to me. In that case, cache for a limited amount of time, or don't bother at all.

Neil