I automatically clear cache on deployment and found out it kills "withoutOverlapping()" scheduled tasks. Ideas ?

So my deployment process automatically clears the cache to avoid issues after a deployment when the format of the cached data has changed so the updated code receives unexpected data from the cache and crashes.

We dont cache a whole lot of things and we usually only deploy a few times a week so it's not an issue performance-wise and it's worked pretty well for a couple of years.

The issue is that some of our tasks need to run continuously so we scheduled them to run every minute withoutOverlapping. And it turns out this relies on the application cache to know wether a task is already running... So it gets ran again the minute after the deployment even if it was not finished.

Am I just wrong to automatically clear the cache upon deployment ? Is there a way to clear our own application cache (and of some packages like spatie/laravel-permission) but keep the cache used internally by Laravel ?

I can't be the only one to face this issue, what's the recommended way to go about it ?

LaryAI

1 year ago

Level 58

The issue you're facing is due to the fact that Laravel's withoutOverlapping() feature for scheduled tasks relies on the cache to track whether a task is currently running. When you clear the cache during deployment, this information is lost, causing the task to potentially run again even if it hasn't finished.

Here are a few strategies you can consider to address this issue:

Selective Cache Clearing: Instead of clearing the entire cache, you can selectively clear only the parts of the cache that are likely to cause issues. This can be done by using cache tags or keys to target specific cached data. However, this requires that your application and any packages you use support cache tagging.
```
// Example of clearing specific cache tags
Cache::tags(['your-tag'])->flush();
```
Use a Different Cache Store for Overlapping Control: You can configure a separate cache store specifically for handling the withoutOverlapping() feature. This way, when you clear your main application cache, the cache store used for task overlapping control remains intact.

In your config/cache.php, you can define a new cache store:
```
'stores' => [
    'overlapping' => [
        'driver' => 'file',
        'path' => storage_path('framework/cache/overlapping'),
    ],
    // other stores...
],
```
Then, in your App\Console\Kernel.php, specify the cache store for the withoutOverlapping():
```
$schedule->command('your:command')
         ->everyMinute()
         ->withoutOverlapping(10)
         ->onOneServer()
         ->store('overlapping');
```
Graceful Deployment Strategy: Consider implementing a more sophisticated deployment strategy that doesn't require clearing the cache. For example, you could use feature flags or versioned cache keys to manage changes in cached data structures.
Custom Overlapping Logic: If none of the above solutions are feasible, you might consider implementing custom logic to handle task overlapping. This could involve using a database table or another persistent storage mechanism to track running tasks.

By using one of these strategies, you can maintain the integrity of your scheduled tasks while still managing your cache effectively during deployments.

1 like