Iron.io queues with Laravel 5.1

Thats's right! I wait for the answer!

@socieboy guess it's just the two of us :P

Any information on this would be helpful. I want to make sure that this project runs smoothly when it goes to full production. Just seems like a big loose end.

I know bro!

+1

Isn't it this what you are looking for? http://laravel.com/docs/5.1/queues#running-the-queue-listener

@bobbybouwmann I don't know? I don't think so? With Iron.io in all previous versions of Laravel that used it (that I'm aware of), you subscribed to a Push Queue on Iron. Laravel then sent the data to Iron which in turn sent it back to the server for processing - hence Queue::marshal() being placed in a route accessible via POST. To my knowledge, using Push Queues, php artisan queue:listen was not required - I never ran it myself, anyway.

Now we're being told that Queue::marshal() is deprecated and to use plain Iron.io queues, but I have no idea how to set one of those up or how it will interact with my project. Also, without anything being changed in my 5.1 project, the old Iron configuration still works - but apparently I'm now doing it wrong.

I really hope that the answer isn't using something like queue:listen because I was using Iron to avoid commands like that, other queue options (Beanstalkd in particular) really did not seem to like Forge/DO and Envoyer and would stop working constantly, despite being started with Forge which is supposed to restart them if they stop (tried both with daemon and without).

So, I'd really like to know exactly how we're supposed to interact with Iron in 5.1.

Surely a lot of people are using queues with Forge/DO/Envoyer and surely lots of people use Iron.io. How are you supposed to deal with queues with Envoyer? Should I have a webhook to run php artisan queue:listen every time I deploy? Because Beanstalkd became totally unstable when I started using Envoyer, it seemed like after every deployment it would just stop working - even though Forge said they were still running. That's why I switched to Iron, resolved the issue because I wasn't relying on something running on the server to handle queued jobs.

If I can get something other than Iron.io to be reliable when used with Forge/DO/Envoyer, I'll use that. But there really needs to be some documentation for queues as they relate to Iron for 5.1. @JeffreyWay @TaylorOtwell, any input?

+1

That's right!, L4 use

Queue::marshall()

to process the Job that was send to iron.. I don't like the idea to run the command line to run the job.s

@socieboy so did Laravel 5.0

Push queues will be removed in December 2015 with L5.2. I hope that there will be optional package / drive which will replace push driver seamless.

I think that push queues scale very well for multi tenant applications which work with dedicated / isolated database, encryption key, queue tubes and etc per tenant .. and same codebase

Are there any easy way to run queues in that case which do not include running daemon for every tenant and monitor it with supervisord .. eg. 1000 tenants === 1000 daemons === harder maintenance and higher resource usage

@drekinov I must have missed that. So you have any suggestions on how to keep Beanstalkd running between Envoyer deployments? Or how it needs to be restarted? I'm a bit worried about breaking someone if I am running artisan queue commands while users are actively sending queued jobs.

Here's my two cents...hopefully it's worth at least that.

From what I understand Iron.io was one of the only people that actually had "Push Queues". No other service was really doing it. It also added layers of complexity to developing smaller apps because smaller devs like myself use my local Homestead to develop and using Iron.io push queues was impossible since my app wasn't accessible from the web.

All a "Push Queue" was (from my limited understanding) was essentially a Queue job that would then get a ping back from Iron.io to a queue worker that would handle the queued job.

The artisan command of queue:listen replaces that in that it just looks for any queued jobs and processed them automatically thus not requiring the "push" from Iron.io.

In regards to deployment... I use Forge. In Forge you can setup a Daemon that points to the current release directory and fires up the queue listen command with a few flags (delays, sleep, tries, quiet). Forge in turns sets up this command via Supervisord so that it can be maintained as a single process.

I have not had to re-trigger that command ever. Once I setup the Daemon on Forge my queues work, even after deployment. Queue Listen re-loads the entire app every time it processes a queue job, so the changes would be reloaded the next time it processed a job.

Laravel 5.1 Docs have instructions on setting up Supervisord for Queue Listen for all of you non Forge folks. http://laravel.com/docs/5.1/queues#supervisor-configuration

Hopefully that's useful to you. Not sure how to sort that N+1 queue problem for multi-tenants.

Hi @BENderIsGr8te - first, love your nickname and avatar ;) As I mentioned a couple times in this thread, I am using Forge with a Digital Ocean server and deploy code with Envoyer.

No matter what I tried, the queue workers (daemons or otherwise) would just stop working after I deployed code via Envoyer. They would say that they were running but queued jobs were not processed unless I went back to Forge and manually restarted the workers. There was not an issue when I was pulling code from my repository directly in Forge, something happened when Envoyer entered the mix. Something, somewhere, is getting reset when I deploy.

Has anyone else encountered such an issue? I am building a mission critical infrastructure for a government client and I can't just "hope" that Beanstalkd is actually working - queues are used heavily.

Below is a screen shot of how my Forge is setup with Artisan Queue. Like I said I haven't had a problem with Queues not running.

@BENderIsGr8te is there an advantage to doing it this way instead of starting a worker under Queue Workers? Do you use Envoyer as well? And lastly, did you install Supervisor on your server or should it be there by default with a Forge provisioned DO server?

I just fired up a queue worker running as a daemon using the Queue Workers tab, deployed through Envoyer, and everything is working fine - so far.

This may have been a poor understanding of how Beanstalked works on my part. I had been running three queue workers (all as daemons) thinking that more is better. Could this have been causing some sort of conflict? Is there an advantage in using more than one? What exactly happens if several queued jobs are fired off at once?

Queue Workers

Much less CPU since they cache the framework on initial run and then use the cached version of the framework to process each job. You can set them up with a --daemon flag as well to force them to keep running. However, the drawback is that since the app is cached, if you push out changes to your code, they queue worker won't see it because it's still using the cached version. So, after your deployment is finished, use a deployment hook (in Envoyer) to run php artisan queue:restart to gracefully restart the queue worker using the new files. A second drawback is that because the framework is not flushed and restarted between each job if you have any memory intensive tasks as part of your queue, you'll have to make sure to free up the resources at the end of the job or you may find yourself running out of room.

Queue Listen

Unlike the worker, queue listen is a little more CPU intensive as it reloads a fresh copy of the app each time it processes a queue job. The advantage to this of course is that you can deploy updates and don't have to re-start anything. However, the queue:listen command does not have a daemon flag, which is where Supervisord comes into play. The Laravel docs show you how to setup Supervisor. But if you deployed your code with Forge, then Supervisor was setup on the server during deployment and it's good to go. Just adding the command like in my screen shot in my previous post will set it all up.

Which one?

It depends on your needs. queue:worker is much less processor intensive and easier to setup a daemon for. However you need to make sure your jobs are flushing resources it uses and you need to restart them with each code deployment.

Whereas queue:listen uses more processing power, but it doesn't need to be re-run between each deployment to pickup code changes. But you do need to user Supervisord to setup a daemon if you want to keep it running.

For me I decided right now while my app is relatively light in use and I have many deployments for changes until it's "prime time" to use the queue:listen command with a Supervisord daemon. However, once my code changes less frequently and my system is getting many more visitors and there are many more queue jobs running, I will move to a queue:worker approach to help maximize my system resources.

I am not a queue expert, that's just my understanding of the system from this page:

http://laravel.com/docs/5.1/queues#running-the-queue-listener

Also, to answer @opheliadesign question about multiple queue workers... (well attempt to provide guidance more than "an answer")... Just like a database where the table is locked while a record is entered, meaning that only one record is entered during that millisecond, queueing jobs is similar. 10 jobs may hit the queue at once, but they are added one at a time a millisecond apart and in turn they are processed one at a time.

From what I imagine happens is that a job is seen, locked, processed, then deleted. This would allow a second daemon to do the same thing, and if a job is locked, it skips to the next job. If you look at the link in my previous post it talks about setting up Supervisord and it appears to mention the number of processes you can setup. In the example it's 8. I take that to mean that you can have up to 8 processes running that command...or in other words 8 different threads processing the queues. That means 1 daemon is running 8 queue workers. That should also solve some of your issues.

I apologize for this. But why is Laravel creating API's for specific Vendors like Iron.io?

It's not so much an API as it is a driver. But it seems like it's a little of both. Iron.io was one of the first to offer 3rd party queue handling and I think it was integrated way back in Laravel 3. So it's just been legacy. With 5.1, that specific Iron.io functionality is gone in favor of a built in functionality which is (a) the right direction and (b) the reason this thread exists...the question being now that Iron.io push queues are not supported out of the box how should queues be handled for the folks that were using that for the past few years.

@BENderIsGr8te wow, wealth of information right there! Thank you so much for taking the time to break it all down.

Indeed this explains quite of a few of the problems I was encountering previously with Beanstalkd and I REALLY wish that Taylor had explained to me when I submitted a support request that the running workers were using cached code from my project. I was getting really weird errors even though everything worked fine locally and in production if I used Iron.

And on that note, there have been times when developing locally when I have tried to use Beanstalkd and something breaks during the execution of a queue while php artisan queue:listen is running. Even after I fix the bug, it seems like the queue keeps on using the same code and hitting the error indefinitely - even after I stop php artisan queue:listen and try every queue command I can think of. What causes this and how can I prevent it?

Years ago, in a Laravel 3 app, I was using Twilio to send group text messages to about 200 people. Somehow an invalid phone number slipped by and I did not know about exception handling at the time, so the queue kept running into it and starting over. As a result, about 70 people received the same message almost a hundred times until I literally had to restart the server. This was with an Iron.io push queue, mind you, and back then $job->delete() seemed to have some sort of bug. So I'm a bit overly cautious with queues in Laravel - I need reliability and no surprises.

+1

Contrary to how it may look I am definitely not an expert on queues. I have just spent a lot of time over the past few month's studying them and trying out different things. So all of that information is just my understanding of how they work. Truth is I only got into Laravel around February of this year for the first time.

Having said that @opheliadesign the only thing I can think of causing the problem you described are failed jobs. If you don't have your system setup to allow jobs to fail then the same jobs will keep re-trying even after you deploy new code. There is an artisan command for installing a failed jobs table in your database. There is also a flag --tries=3 that you can setup with both queue:worker and queue:listen that will record the job in the failed table after (in this case 3) failed attempts. If you have jobs that are failing (crashing) and haven't let them fail into a table, then regardless of code deployment they are going to continue to fail and cause you issues, even after a full restart.

An example where I ran into this was that I started queuing several emails, but I didn't realize when you are going to queue emails that the when I would pass in a Model it would be passed as an array (since serializing closures is complicated an array is used instead of an object). So my code which was referencing $variable->property kept getting me errors. Eventually I realized this and updated all my code to use array accessors $variable['property']. However I still ran into errors because even though I changed my code the queued job still had my old code and was still failing. Once I put the --tries flag in there and allowed them to fail properly (which they did) then everything started to work great after that.

@BENderIsGr8te I had the failed_jobs table and they were not being entered into it. There's also a command to delete a job but requires an ID to do so, I could not find any way of retrieving said ID - again, nothing was going to the database.

I think you have given me some very valuable information to use as I move forward with this and other projects. Very impressive knowledge for being so new to Laravel, great job!

I also posted this question to Stack Overflow - http://stackoverflow.com/questions/30793682/laravel-5-1-and-iron-io and have not received any answers or comments. Could you head over there and summarize what you have shared on Laracasts? I'll give you a +1 and selected answer.

Thanks!

@opheliadesign I just finished updated my app from laravel 4.2 to 5.1 and the push queue work as normally.

@socieboy yes they still work but Queue::marshal() says it is deprecated now.

@BENderIsGr8te okay, so I tried what you suggested by entering a regular daemon on the server to keep php artisan queue:listen running.

I entered exactly what you had in your screenshot and it threw a fatal error saying that the command could not be executed. However, if I prefixed it with php, it worked.

I also noticed that, after a deployment, I absolutely had to restart the daemon for the queue listener to recognize new code. Also, trying to figure out why URLs formed with route('members.foobar') are returning with localhost/... rather than the domain name in production - not sure if this is related to queued event listeners. Comment? Suggestions?

Thanks :)

I can't tell you as to why you had to restart the queue listen command, I haven't had to yet, but maybe I haven't made any changes to any files that the queue worker uses.

As far as the second issue, if you are calling a route generator from a queued job then you are getting the correct localhost response. Remember, the domain name (HOST) is provided by the web server (nginx, apache, etc..). Queued jobs are handled from Artisan...the command line, so there is no web server passing a HOST to the app. As a result Laravel looks to your config/app.php file for the URL you have configured there (which by default is 'localhost'). So assuming you don't have to fire off jobs for multiple sub domains or multiple domains, then setting the base URL in the config/app.php file should probably solve that issue.