DoeJohn's avatar

Unable to JSON encode payload. Error code: 5

I'm working on Laravel (v5.7) app that converts uploaded CSV (with contacts) into array that is then passed as argument when job class is being dispatched.

Here is the example of CSV file (format that is supported):

123456,Richard,Smith
654321,John,Doe

Uploaded (CSV) file is handled like this:

$file_path = $request->file_name->store('contacts');
$file = storage_path('app/' . $file_path);

$contactsIterator = $this->getContacts($file);

$contacts = iterator_to_array($contactsIterator); // Array of contacts from uploaded CSV file
    protected function getContacts($file)
    {
        $f = fopen($file, 'r');

        while ($line = fgets($f))
        {
            $row = explode(",", $line);

            yield [
                'phone'     => !empty($row[0]) ? trim($row[0]) : '',
                'firstname' => !empty($row[1]) ? trim($row[1]) : '',
                'lastname'  => !empty($row[2]) ? trim($row[2]) : '',
            ];
        }
    }

Finally, $contacts array is passed to a job that is dispatched:

ImportContacts::dispatch($contacts);

This job class looks like this:

    public function __construct($contacts)
    {
        Log::info('ImportContacts@__construct START');
        $this->contacts = $contacts;
        Log::info('ImportContacts@__construct END');
    }

    public function handle()
    {
        Log::info('ImportContacts@handle');
    }

... and everything worked fine (no errors) until I've tried with this CSV:

123456,Richardÿ,Smith
654321,John,Doe

Please notice ÿ So, when I try with this CSV - I get this error exception:

/code_smsto/vendor/laravel/framework/src/Illuminate/Queue/Queue.php | 91 | Unable to JSON encode payload. Error code: 5 

... and my log file looks like this:

  error local   2019-11-11 17:17:18     /code_smsto/vendor/laravel/framework/src/Illuminate/Queue/Queue.php | 91 | Unable to JSON encode payload. Error code: 5
  info  local   2019-11-11 17:17:18     ImportContacts@__construct END
  info  local   2019-11-11 17:17:18     ImportContacts@__construct START 

As you can see - handle method was never executed. If I remove ÿ- no errors and `handl is executed.

I've tried to solve this, but without success:

  1. Move "handling" (converting to array) of uploaded CSV file into @handle method of a Job class (but the same issue still exists).
  2. Apply utf8_encode:
    protected function getContacts($file, $listId)
    {
        $f = fopen($file, 'r');

        while ($line = fgets($f))
        {
            $row = explode(",", $line);

            yield [
                'phone'     => !empty($row[0]) ? utf8_encode($row[0]) : '',
                'firstname' => !empty($row[1]) ? utf8_encode($row[1]) : '',
                'lastname'  => !empty($row[2]) ? utf8_encode($row[2]) : '',
            ];
        }
    }

... and it works (no errors, no matter if there's that ÿ, but then Greek and Cyrillic letters are turned into question marks. For example, this: `Εθνική? become ???????.

I also tried with mb_convert_encoding($row[1], 'utf-8') - and it doesn't turn Greek or Cyrillic letter into question marks, but this ÿcharacter will become ?.

DEBUGGING:

This is what I get from dd($contacts);:

enter image description here

So, it has that "b" where ÿis. And, after some "googling" I found that this "b" means "binary string", that is, a non unicode string, on which functions operate at the byte level (https://stackoverflow.com/questions/4749442/what-does-the-b-in-front-of-string-literals-do).

What I understand is this: When dispatching Job class, Laravel tries to "JSON encode" it (passed arguments/data) but it fails because there are binary data (non-unicode strings). Anyway, I was not able to find a solution (to be able to handle such CSV file with ÿ.

I am using:

  • Laravel 5.7
  • PHP 7.1.31-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Aug 7 2019 10:22:48) ( NTS )
  • Redis powered queues
0 likes
1 reply
Jaytee's avatar

You may get away with json encoding the data before you pass it to the job, and then decoding it in the handle method. It'll be encoded twice, but Laravel checks for any json errors, where as if you do it manually, you won't be, and it'll convert the characters

Please or to participate in this conversation.