Kai-T's avatar

Are Queues a solution to my problem?

I'm working on an application, where users can book a slot on an event. These event's have

  • maximum users
  • start and end of the event
  • start and end of the booking window

There will be moments in Time, where users want to join these events en masse. My current implementation lags out if arround 1.5k users try to join in the same moment. As the page is slow, they tend to refresh more often, creating more load. I'm currently optimising my join process and the generation of the List-Pages.

But I'm asking myself, if a queue will be a solution or if they will hurt more. For testing I've created a Job that checks if the user is allowed to join (for example: user can't join if already in another overlapping event) and if so, dispatches another job that creates all the notifications (E-Mail, Push, Reverb, based on user choices). The "join job" takes about 100ms according to the debug statements of the worker. But the worker only takes a job every 500 - 1000 ms.

I'm using the database driver for testing. Will using redis on linux (for production) speed the polling up? Are there other ways to decrease the "idle time" except starting more workers? How are multiple workers handling jobs in the queue? Will ever worker pick the next from start of the queue (I would think so). My Idea to use only one worker for the joining was to minimize database collissions if 1.5k users try to join the same events (which have less slots than needed).

I think that I'm not the first with this problem, but as search engines are getting worse at delivering meaningfull information, I wasn't able to find something usefull.

Any recomendations for what to do and what not to do regarding this problem?

Regards kai

0 likes
4 replies
Glukinho's avatar

If you deal with thousands of simultaneous requests I don't see how jobs would help with performance. They can help with organizing actions but not provide faster response.

In the first place you should have fast and finely tuned web server(s) / DB server(s) that can carry such load.

My Idea to use only one worker for the joining was to minimize database collisions if 1.5k users try to join the same events (which have less slots than needed)

Database collisions should be handled on database level - using database transactions, constraints etc. Queues are not for that purpose. Also, you app should act equally no matter you have 1 worker or 10 - otherwise queue becomes a source of problems instead of solving them.

If your users need to see response quickly (I think they do) maybe queued jobs with non-guaranteed time of starting is not your solution at all. Maybe you should handle all logic in one request, providing fast response to a user.

Tray2's avatar

This doesn't sound like something that should be a job, this is simply an insert into a table with a limitation on how many "seats" there are for each event.

1500 users at the same time isn't that much, I would suggest l looking into your database settings and increase the number of simultanious connections.

https://serverpilot.io/docs/guides/mysql/config/max-connections/

You should also probably increase the cpu and memory for your database, and as a last resort look into load balancing your database, but I don't think that is really necessary.

I'm guessing here that you have quite a few indexes on the tables involved, and that is most likely also a reason for it being slower than preferred, MySQL actually lock the table when it updates the indexes, and if you are doing lots of updates, well then you will get delays in the database, these are called blocking locks, and in worst case deadlocks.

One option could also be that you just insert into a temporary table with only a unique index, and then have a job move the data into an properly indexed table.

Kai-T's avatar

Thanks for your feedback. I'm back from vacation and will have a look into both your suggestions on the weekend

Kai-T's avatar

The database setup (concerning this problem) is very small. There is one table with the events a user can join that has an integer column with the maximum slots (and many other columns which aren't used here). The table where the user is entered has an uuid for the entry, a foreign uuid for the event and a string field if a slot is manually blocked and not by a user directly.

Joining is done after some checks (and I think I can optimize here):

  • is the user already joined to the event
  • is joining for this event generally allowed
  • is the current time inside the set timeframe for joining
  • is the user in any other event at the same time
  • are there any free slots

All these checks are done using the models, but I would creating custom database querys that return true or false be faster than loading the data and performing the checks in php? (Database is a MySQL or to be more precise MariaDB 11 (.8 I think) on the same host as the laravel application)

Please or to participate in this conversation.