Malformed UTF-8 characters, possibly incorrectly encoded

Published 1 week ago by lnb596844

When I'm trying to return the result from my controller, I'm getting this error "Malformed UTF-8 characters, possibly incorrectly encoded"

$bank_receipts = DB::table('bank_receipts')
                ->where('transaction_id', 1);

$receipts = DB::table('cash_receipts')
            ->where('transaction_id', 1)
            ->unionAll($bank_receipts)
            ->get();

return response()->json($receipts, 200);
kfirba
kfirba
1 week ago (203,855 XP)

@eDesignary I believe the source of this error message is when Laravel tries to convert the $receipts to JSON. What does the $receipts variable looks like? Can you try and post a dd() here?

Please ensure that your database driver is using some kind of utf8 encoding.

lnb596844

@kfirba I think it is due to the receipt_date in my collection, "receipt_date": b"á\x07\x08\x08

kfirba
kfirba
1 week ago (203,855 XP)

@eDesignary Seems like it. It looks corrupted. What does this field should look like?

Anyway, try to exclude the receipt_date from the response and see if you get that error

lnb596844

@kfirba without receipt_date there is no error. It is working normally when I return the result from joins and normal query, but when I try unionAll I'm getting this error, but it is working with union

kfirba
kfirba
1 week ago (203,855 XP)

@eDesignary What is the scheme for the receipt_date? Also, what's your database driver?

Actually, it would help if you share both bank_receipts and cash_receipts table schema. The Union operator has a few rules (W3):

  1. Each SELECT statement within UNION must have the same number of columns
  2. The columns must also have similar data types
  3. The columns in each SELECT statement must also be in the same order

Can it be that you lack something?

lnb596844

@kfirba I'm using mysql and these are the table schemas

kfirba
kfirba
1 week ago (203,855 XP)

@eDesignary That's weird...

Does the data for the receipt_date seem to be valid on your database side?

lnb596844

@kfirba sorry, I don't understand what you are asking.

kfirba
kfirba
1 week ago (203,855 XP)

@eDesignary When you view your database using the console or some other program, does the data seem to be valid? Does it show the proper date as you expect?

Also, try running the query manually against your database using the console or some other program and see what the returned data looks like.

I'm trying to see where exactly the data corruption is happening.

ftrillo
ftrillo
1 week ago (12,140 XP)

It might not have anything to do with the problem but I don't see the txn_id field in any of the tables.

Also, do all the items in the collection have the corrupted receipt_date like that? Could you check if they have a correct date before returning them as json?

lnb596844

@kfirba yes it is working, I've tested it manually from phpmyadmin.

lnb596844

@ftrillo it is transaction_id not txn_id, edited my question, and it is not the problem. and yes all the items in the collection have the corrupted receipt_date and they are good before returning them as json.

lnb596844

there is no error when I use union, but when I try unionAll, I'm getting the above error

kfirba
kfirba
1 week ago (203,855 XP)

@eDesignary Well, I'm not sure why it's happening.

Can you try running the query manually from PHP using the raw PDO object?

You can get the PDO object like so:

DB::connection()->getPdo();

If you don't remember how to interact with PDO directly, check out the docs for PDO's statements

Please sign in or create an account to participate in this conversation.