Eloquent save converting unicode characters

What is the field type of location?

Varchar and Text have their own collation and charset. Make sure it is utf8mb4. :)

Yeah, should've posted that. Sorry.

location is a varchar field. Its collation is 'utf8mb4_unicode_ci' and its charset is 'utf8mb4'.

Can it be text? I default to text and only varchar when forced to. In text, I have been able to store all sorts of characters including emojis with utf8mb4.

Thank you for the suggestion. I changed the field type to text and tried it again. Same result. Bummer.

what about

CHARACTER SET utf8mb4 COLLATE utf8mb4_bin

Check the mysql settings in config/database.php if they match your DB actual values.

mysql settings in config/database.php

'mysql' => [
    'driver' => 'mysql',
    'url' => env('DATABASE_URL'),
    'host' => env('DB_HOST', '127.0.0.1'),
    'port' => env('DB_PORT', '3306'),
    'database' => env('DB_DATABASE', 'forge'),
    'username' => env('DB_USERNAME', 'forge'),
    'password' => env('DB_PASSWORD', ''),
    'unix_socket' => env('DB_SOCKET', ''),
    'charset' => 'utf8mb4',
    'collation' => 'utf8mb4_unicode_ci',
    'prefix' => '',
    'prefix_indexes' => true,
    'strict' => true,
    'engine' => null,
    'options' => extension_loaded('pdo_mysql') ? array_filter([
        PDO::MYSQL_ATTR_SSL_CA => env('MYSQL_ATTR_SSL_CA'),
    ]) : [],
],

if you paste 'Bordeaux–Mérignac Airport' via another client, does it save correct?

First, THANK YOU for taking time out of your day to help me. I love the Laravel community.

Just ran a couple of tests...

If I paste 'Bordeaux–Mérignac Airport' directly into the location field of my MySQL client (Table Plus on Mac), it saves just fine.

Here's the weird(er) part: I wanted to see if Eloquent was doing anything to the string to prep it prior to saving to MySQL, so I created a very simple web form to edit the SomeModel location field. When presented in the browser, the location field text input element contains the large space character (as expected). If I highlight that and then type an en dash character (option-dash on the Mac), then save, the en dash saves perfectly.

public function update(SomeModel $someModel, Request $request)
{
    $input = $request->validate([
        'location' => 'required|string|max:255'
    ]);

    $someModel->update($input);
}

So, I don't think it's a MySQL collation or charset issue, and I don't think it's an Eloquent issue.

Now I'm really stumped.

for my knowledge, if the field is set to unique and will contain any non-number / non-english

you should use

$table->string('name')->collation('utf8mb4_bin')->unique();

for example try to insert two difference name to table when use utf8mb4_unicode_ci

you will get error duplicate entry

森下えりか
森下エリカ

https://mathiasbynens.be/notes/mysql-utf8mb4

i don't know character-set-client-handshake will cause that problem too

.php is UTF-8 ?

@twopiers Your client test shows it is not a MySQL issue. So that leaves only Laravel and PHP.

1. If you write a raw query, does it save correctly? Be sure to paste Bordeaux–Mérignac Airport from here, the dash in question is – and not a typical - (hyphen/minus on our keyboard).
2. If raw query works, then only Laravel remains as the sole suspect.

I know it's late but I think it will help in future.

First of all, It's all doing Eloquent, not a database. If you do insert manually it works normally.

The problem that Eloquent calls json_encode without the flag JSON_UNESCAPED_UNICODE. If you don't have this flag, it will convert multibyte symbols to unicode. So to solve this, you just have to use mutations:

class Order extends Model
{

    protected $fillable = [
        'name',
        'equipments',
        'documents',
    ];

    protected $casts = [
        'id'                => 'integer',
        'equipments'        => 'array',
        'documents'         => 'array',
    ];

    public function setEquipmentsAttribute($value)
    {
        $this->attributes['equipments'] = json_encode($value, JSON_UNESCAPED_UNICODE);
    }

    public function setDocumentsAttribute($value)
    {
        $this->attributes['documents'] = json_encode($value, JSON_UNESCAPED_UNICODE);
    }
}

So, now it will work normally. No need to thank ^)