dbrmr's avatar

File processing queue

Hello Laracasts forum,

I am working on a project, which allows many users to upload files (csv,xls,xslx,xml), the files should be validated, and if valid, the data is stored in the database.

The files have strict naming convetion and it will be validated during upload. The file content could be very long, so it cannot be validated during a request, this task will be pushed to a queue.

I would like to give the users some UI, where they can see their uploaded files. It is a table with a Status column, which is changing like: Uploaded, Validating, Importing to DB, Success, or possible Error after each step.

My questions is how can I monitor the jobs and display their status ?

0 likes
3 replies
brandonviles's avatar

I'm going to take a crack at this having never done it.

Why not have a jobs table in a database, that your pending jobs will write their status to as it hits certain points in it's lifecycle.

Then you can fetch this data from the database, asynchronously even, and display it to the user.

2 likes
dbrmr's avatar

So, I have a file_proc_jobs table

  1. when a file is uploaded, I should insert a row in this table with status uploaded and also push a job to the queue

  2. when the job code runs, it updates the table with the status

This sounds simple enough.

Thank you @brandonviles

pmall's avatar

I do a lot of file processing an this is exactly what I do.

Please or to participate in this conversation.