NameUnknown's avatar

Redis repeatedly saves the queue database (horizon) resulting in loss of pending jobs

When we used Redies for queued jobs we faced the following problem: Redis constantly overwrites the database almost every second. Here's the log:

30:C 21 Oct 2023 09:24:21.091 * DB saved on disk
30:C 21 Oct 2023 09:24:21.091 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
1:M 21 Oct 2023 09:24:21.189 * Background saving terminated with success
1:M 21 Oct 2023 09:25:22.045 * 1 changes in 60 seconds. Saving...
1:M 21 Oct 2023 09:25:22.045 * Background saving started by pid 31
31:C 21 Oct 2023 09:25:22.048 * DB saved on disk
31:C 21 Oct 2023 09:25:22.049 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
1:M 21 Oct 2023 09:25:22.146 * Background saving terminated with success
1:M 21 Oct 2023 09:26:23.018 * 1 changes in 60 seconds. Saving...
1:M 21 Oct 2023 09:26:23.018 * Background saving started by pid 32

If a job is delayed for long time (like 30 minutes) it is just erased from the queue. We ran the app in Docker with following container settings:

redis:
      image: redis:alpine
      ports:
          - "6379:6379"
      command: redis-server --appendonly yes   --appendfsync everysec 
      restart: unless-stopped
      volumes:
          - selenium_redis:/data/

We set --maxmemory to 1024mb and more but it helps only for small amount of time and then the looped overwriting starts again.

We have changed the queue driver which solved the problem, but now we are curious: what is the issue with Redis?

0 likes
0 replies

Please or to participate in this conversation.