PDO - InvalidArgumentException in JsonResponse.php line 58: Malformed UTF-8 characters, possibly incorrectly encoded

In my production and local environment I have the MSSQL DB config shown below.

    'recipeDB' => [
        'driver' => 'sqlsrv',
        'host' => env('DB_HOST', 'host'),
        'database' => env('DB_DATABASE', 'database'),
        'username' => env('DB_USERNAME', 'username'),
        'password' => env('DB_PASSWORD', 'password'),
        'charset' => 'UTF-8',
        /** Updating collation does not appear to do anything */
        'collation' => 'SQL_Latin1_General_CP1_CI_AS',
        'prefix' => '',
    ],

Selecting from this database and converting to JSON works perfectly in production (PHP5), however, in my test environment (PHP7) I get the following error:

 Malformed UTF-8 characters, possibly incorrectly encoded

If I changed my PDO charset from UTF-8, I can no longer connect.

EDIT

The Collation for the database is SQL_Latin1_General_CP1_CI_AS - I'm not sure how that translates to charset.

More Details

PHP Version: PHP 7.0.8-3ubuntu2 (cli) ( NTS )
Framework: Laravel 5.2
PDO & Driver Config: Shown Above

Data Returned from dd(DB::connection('recipeDB')->table('Recipe')->first())

  {#322
  +"recGUID": b"›E•_K¿\x1CHŸü\x00\x00ât\x03Ž"
  +"location": "France"
  +"locationid": 3
  +"description": "Tangerina"
  +"publishedname": "Tangerina"
  +"instructions": ""
  +"alcohol": 1
  +"web": 0
  +"PublicationExists": 0
  +"GlobalMobileApp": 0
  +"credat": "2009-12-16 00:00:00"
  +"updat": "2011-05-19 12:14:21"
  +"servingsize": null
  +"isCoffee": 1
  +"isFood": 1
  +"isHome": 1
  +"isBar": 1
  +"colorid": b"+Üp{b%ÃB┤ä┘Ãú÷┌ü"
  +"equipmentid": b"–lOW¾…yDˆÜ¦ù§\x03E!"
  +"bevtypeid": b"ã®få†$ÃD¹êò\x15@×K•"
  +"serveid": b"═HØ»\x19Ô8Aûv<╝¡zÎ\v"
  +"glassid": b"O¾\x16C<i‡G±!³#GÇ\x15É"
  +"methodid": null
  +"servingsizeuomid": null
  +"LastEditDate": "2013-10-30 20:05:06"
  +"CreationDate": "2010-06-10 17:08:46"
  +"proprietary": 1
  +"Rating": -1
  +"LevelOfDifficulty": 4
  +"Enable": 1
  }

Error:

  InvalidArgumentException in /home/user/PhpStorm/Projects/recipeApi/vendor/laravel/framework/src/Illuminate/Http/JsonResponse.php line 58:
  Malformed UTF-8 characters, possibly incorrectly encoded

Error Trace:

   1. in JsonResponse.php line 58
   2. at JsonResponse->setData(array('data' => array(array('recipeId' => '�N��D��E�MH�4>�', 'alt_id'=>null...

waled

9 years ago

Level 2

Hello, did you find a solution? I have the same issue.

JoolsMcFly

9 years ago

Level 2

As stated in the doc ``json_decode``` only deals with UTF-8.

It means you either need to convert from whatever collation you have to UTF-8 and then call json_decode.

Does your DB have the same collation on prod and in your test environment?

Dev could fail when it finds a non UTF-8 value, accentuated characters for example, something your prod might not have?

If I were you I'd make sure DB columns were using UTF-8 to avoid collation issues.

tylerssn

9 years ago

Level 3

Strangely, this only appears to impact GUIDs. This is occurring during my attempt to upgrade my site to PHP 7 and my research seems to be pointing to lack of UTF-8 support in drivers as the culprit.

julesbloemen

9 years ago

Level 2

I have the same issue with the Microsoft driver. Any solution yet?

tylerssn

9 years ago

Best Answer

Level 3

Yes! And I can finally move forward with PHP 7.

Summary, Cause, and General Fix

See my Laracast post and answer here for how I setup Laravel DB and Eloquent classes to automatically resolve this.

From my StackOverflow Answer

As explained in this pull request, the cause of this issue in PHP 7 is that you have to instruct PDO to convert the GUIDs from binary to a string.

This should do it for anyone struggling with this.
    /** @var \PDO $pdo */
    $pdo->setAttribute(\PDO::DBLIB_ATTR_STRINGIFY_UNIQUEIDENTIFIER, true);

1 like

Please or to participate in this conversation.