Does horizon support enqueuing jobs from non-horizon codebases via illuminate/queue?

To answer my own question: I'm now running this combination in production since ~12 hours, so far looking good.

Over 24 hours, it's still running 👍

I guess this answers my question whether this works or not 😀

I still wish I could make the jobs be properly realized by Horizon, but besides reverse engineering this it would require installing Horizion which doesn't work. Horizon requires the Laravel Application which is the service container + stuff on top of it, which isn't available when just using the illuminate/* packages.

Hi,

We're facing now exactly the same issue. We want to have Horison to process queue, but we want to dispatch jobs from non-laravel codebase, from plain php framework. We will try to follow the same approach with Illuminate/Queue package. Is still working fine for you?

Hi @antonk ,

yes, absolutely! Works like a charm. I did it for two non-Laravel code bases (both are CakePHP 2…).

Integration is probably very specific but if you want to know more information, I can give an outline what I had to do; just let me know.

cheers

Thanks for the reply @max_well . What I achieved so far is that I can push message via queue to redis from non-laravel codebase. It appears in redis under the key queues:default and the data looks like this

{
  "uuid": "061b8839-5f7e-48d9-8e9c-d9c865a5982b",
  "displayName": "App\Queue\Job\SendSmsJob",
  "job": "App\Queue\Job\SendSmsJob",
  "maxTries": null,
  "maxExceptions": null,
  "delay": null,
  "timeout": null,
  "data": {
    "phone_number": "+123123123213",
    "sms": "Hello from the queue!"
  },
  "id": "4FVJHX4JBZIuDtUDg4y8yEcKiRvGQTPf",
  "attempts": 0
}

I also setup horizon and so far it doesn't see such job. When I push message from horizon it appears under laravel_horizon key. And there are a lot of other keys in redis for horizon.

So the main question now what configuration you have from non-laravel side that horizon treats this as native queue massage?

This is how I push message from non-laravel side

$queue = new Queue;

        $container = $queue->getContainer();
        $container['config']['database.redis'] = [
            'cluster' => false,
            'default' => [
                'host' => 'redis',
                'password' => null,
                'port' => 6379,
                'database' => 0,
            ],
        ];
        $container['config']["queue.connections.redis"] = [
            'driver' => 'redis',
            'connection' => 'default',
            'queue' => 'default',
            'retry_after' => 30,
        ];

        $container->singleton('redis', function ($container) {
            return new RedisManager(Container::getInstance(), 'predis', $container['config']['database.redis']);
        });

        $queue->addConnection([
            'driver' => 'redis',
            'connection' => 'default',
            'queue' => 'default',
            'retry_after' => 30,
        ]);

        $queue->setAsGlobal();

        Queue::push('App\Queue\Job\SendSmsJob', ['phone_number' => '+31321312323', 'sms' => 'Hello from the queue!']);

After some tweaks on horizon side with queue prefixes, I managed to push message from non-laravel codebase to the horizon queue. Job is also being processed. But the problem that I still have that on horizon dashboard I can't see these messages. I found out that this because of the redis key content. Jobs that are being pushed by horizon have 5 more fields like connection, id, name, queue, etc. Don't know yet is it possible to fix this issue from horizon side.

@max_well did you also face the same issue?

Yes, I also think I don't see the jobs.

I wasn't a concern for me as I rarely use the dashboard.

Since all the information is in redis, I wrote some custom adapters for 3rd party metric / graphing system which did integrate this with many other graphs (cpu, disk, ram, etc.).

Some differences in our code I see is:

• I'm using \Illuminate\Queue\Capsule\Manager to set it up, e.g.

        $manager = new QueueManager($container);
  
        $manager->addConnection([
            'driver' => 'redis',
            'host' => 'localhost',
            'queue' => 'default',
        ]);
  
        return $manager;
  

  (but looking at your code, I think your Queue is the Capsule manager…)
• I did not manually bind the redis stuff, I just called \Illuminate\Redis\RedisServiceProvider::register
• I don't use the global stuff; never been I fan of this. I get the instance from the container and call->pushOn()
• I use actual classes to push jobs which are in fact mirrored from the destination code base For example if the code base running is in the name space CoreServiceApp then there are the regular local jobs in CoreServiceApp\Jobs I did mirror this namespace in the other projects so I cold present jobs as actual classes and there constructor takes the parameters => same as it works in Laravel To me this has the benefit of more strictness/type safety but also requires placing those "shadow classes" somewhere and fiddling with composer to autoload them correctly

cheers!

Yeah, nice. Just our of curiosity - why then you use horizon if you are not using it's dashboard? For us this is the main reason why we have chosen horizon. Otherwise it can be standard redis queue from laravel without UI interface.

Horizon is more than just the dashboard.

• you can programmatically define the queues, something with regular queues would require manual config of e.g. supervisord or whatever one is using.
• it allows balance strategies on queues.
• out of the box support for different environments: I might run 10 workers on a queue in production but only 2 in dev

I guess people are dazzled by the first sentence "Horizon provides a beautiful dashboard…" but it's much more ;)