SQLSTATE[HY000] [2006] MySQL server has gone away

Also it does appear that my payload is being generated properly just from looking at the error. https://i.imgur.com/OAIxHRy.png

That is exactly why I've switched to SQLite or Docker depending on the need for matching the production as closely as possible.

So would you recommend I just move to Redis or something? Is this just a limitation of MySQL? Just looking for a practical solution to this problem. I have never used Redis before but I am sure it can't be that hard to swap over.

Not Redis, but SQLite or MySQL under Docker. PostgreSQL is also a great option, but also under Docker.

So to confirm, I just moved to SQLite to test it. The job processes just fine. However it adds entries at like 1/1000th of the speed that it was adding entries to MySQL. Is it supposed to be slow? I have to enter 40,000 entries and that would've normally been done at (somewhat close to) 100 records per second on MySQL. SQlite is going roughly 10 records or less a second.

EDIT: Also if it is processing the Job TablePlus seems to repeatedly lose its connection to the database file while it is in mid-entry. This does not seem like a better way to go about it. I feel like I should be fixing the MySQL problem rather than creating new different problems.

For me SQLite works very well. It is not something that I would use on production, but for local env I had no issues. Do you have fast SSD? For me it is not slow at all.

I also use TablePlus and it never dropped connection with SQLite. Not sure what your workflow is but mine always worked out of the box.

Yeah I have a standard HDD which is probably the difference we are experiencing. For those of you who are googling and reading form the future (hello future people) I resolved this issue by modifying my MySQL config file.

I changed max_allowed_packet under [mysqld] to 10M though, depending on the size of the data you are passing who knows you may need more. 10mb in a single MySQL field is a lot of data though so if you are passing more than that into a single field maybe its time to re-evaluate what your process is

I did also add default_authentication_plugin=mysql_native_password to the same section as per the advice of the article linked above. Though, I don't think that actually resolved or changed anything. That was the first thing I tried and it didn't actually work until I changed the max_allowed_packet to be larger. I am leaving it in there since it doesn't seem to hurt anything and I am not curious to find out how everything behaves if I remove it. 99.99% sure the packet size was what fixed it.

EDIT: Just to further clarify so you aren't going "ok well that fixes it by why did it fix it". I believe what was happening was that my Job was pretty dang large. Not only was it a lengthy complex operation itself but the dataset it ultimately needed to process through was very large. MySQL could not physically enter all the information required into the payload field of the Jobs table because there was a limitation on the size of what could be stored in a MySQL field. Bumping that size up allowed me to store my Job properly in the Jobs table.