staticcode's avatar

Incorrect string value

Hi guys, this error is driving me nuts! I am trying to import CSV data. When I manually add it in Phpmyadmin then it works fine, but when I add it through Laravel I end up with the error below.

Tried everything but nothing solved this problem, but stuck on this for over a week now. Does anyone know a solution maybe? as said, I can add it manually in SQL no problem, just through Laravel it gives me this error so I'd say it has to do with Laravel?

My row:

article_name varchar(64) utf8mb4_unicode_ci

SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\xFD)' for column 'article_name' at row 1121 (SQL: update `products` set `fk_id` = 64356, `status` = 0, `brand` = 15, `article_number` = JN001, `color` = aqua, `size` = S, `article_name` = Round-T-Medium (150g/m�), `description` = Ronde hals met elastaan en halstape geen zijnaden Dubbel gestikt op schouders hals en mouwboorde, `material` = Buitenstof 150 g/m 100% katoen, `category` = T-shirt, `washing_instructions` = Maximale wastemperatuur 40 C normale was, `color_id` = blauw, `photo_1` = art79\detail_1005.jpg, `photo_2` = art79\detail_7264.jpg, `photo_3` = art79\detail_7264_highres.jpg, `photo_4` = art79\detail_7265.jpg, `photo_5` = art79\detail_7265_highres.jpg, `photo_main` = artfarbe847\master.jpg, `weight` = 0, `price` = 3.55, `updated_at` = 2018-03-24 16:42:23 where `sku_id` = 68824)
0 likes
8 replies
ejdelmonico's avatar

That error would point me towards the utf8mb4_unicode_ci. When you import the CSV manually, are you setting the same character set?

rin4ik's avatar

You need to ensure the file you are importing is actually in UTF-8 format, and not ANSI for instance, as it will cause the error described.

staticcode's avatar

@rin4ik The CSV itself is saved in UTF-8 format :) so that should be good on that part.

@ejdelmonico The row collition is set to utf8mb4_unicode_ci, so all I do manually is (in short)

insert into description values ("description here")

and that works perfectly, just not when doing it with Laravel :)

My query in Laravel:

$query = Products::where('sku_id', '=', $items['sku_id'])->update([
                                   'fk_id' => $items['fk_id'],
                                   'status' => $items['status'],
                                   'brand' => $brandid,
                                   'article_number' => $items['article_number'],
                                   'color' => $items['color'],
                                   'size' => $items['size'],
                                   'article_name' => $items['article_name'],
                                   'description' => preg_replace("/[^a-zA-Z0-9%\/\s]/", "", $items['description']),
                                   'material' => preg_replace("/[^a-zA-Z0-9%\/\s]/", "", $items['material']),
                                   'category' => $items['category'],
                                   'washing_instructions' => preg_replace("/[^a-zA-Z0-9%\/\s]/", "", $items['washing_instructions']),
                                   'color_id' => $items['color_id'],
                                   'photo_1' => $items['photo_1'],
                                   'photo_2' => $items['photo_2'],
                                   'photo_3' => $items['photo_3'],
                                   'photo_4' => $items['photo_4'],
                                   'photo_5' => $items['photo_5'],
                                   'photo_main' => $items['photo_main'],
                                   'weight' => $items['weight'],
                                   'price' => $items['price'] * 1,
                                ]);
ejdelmonico's avatar

That error is usually because unicode processing so that is why I mentioned about you must have database charset set to utf8mb4 and collation to utf8mb4_unicode_ci in both cases. Mysql version should be current.

staticcode's avatar

Ah good point @ejdelmonico , can you tell me how I can do that? all I know is that I set my table on utf8mb4_unicode_ci and that in database.php my charset is utf7mb4 and collation utf8mb4_unicode_ci :)

I just tried turning "strict" off (setting it to false). Maybe this will provide a solution as well :) Will let you know the outcome!

ejdelmonico's avatar
Level 53

https://dev.mysql.com/doc/refman/5.7/en/charset-unicode-conversion.html

basically, run this SQL statement:

ALTER DATABASE
    database_name
    CHARACTER SET = utf8mb4
    COLLATE = utf8mb4_unicode_ci;

OR

ALTER TABLE
    table_name
    CONVERT TO CHARACTER SET utf8mb4
    COLLATE utf8mb4_unicode_ci;

OR

ALTER TABLE
    table_name
    CHANGE column_name column_name
    VARCHAR(191)
    CHARACTER SET utf8mb4
    COLLATE utf8mb4_unicode_ci;
1 like
staticcode's avatar
'strict' => false

in my database.php fixed the issue as well. Honestly, probably not the best way to solve an issue like this but it worked lol.

@ejdelmonico thanks for the SQL command, Ill try running it later tonight and see if I can turn the strict method back on :)

2 likes
ysanchez's avatar

I had the same problem.

Error:

General error: 1366 Incorrect string value: '\xE9phane' for column 'name' at row 1 (SQL: update `users` set `edu_id` = [email protected], `name` = Stéphane, `surname` = Dubois,
...

Code that produces the error:

$user = User::updateOrCreate(
    ['id' => env('mail')],
    [
        'edu_id' => env('uniqueID'),
        'name' => env('givenName'),
        'surname' => env('surname'),
        'id' => env('mail'),
    ]
);

Error appends when a user with accents in its name (Stéphane) tries to log in.

I suspected a problem with inputs : env('givenName')

Code that corrected the problem:

$user = User::updateOrCreate(
    ['id' => env('mail')],
    [
        'edu_id' => utf8_encode(env('uniqueID')),
        'name' => utf8_encode(env('givenName')),
        'surname' => utf8_encode(env('surname')),
        'id' => utf8_encode(env('mail')),
    ]
);
3 likes

Please or to participate in this conversation.