1 year ago

How to receive and process request known to be in non UTF-8 charset?

Posted 1 year ago by pilat


Here's the issue: one of sites sends data to my app in a non-UTF-8 charset. That site is not under my control, so, let I it be as it is.

Now, when I'm trying to process this data, I have few issues.

  1. I cannot log the request data.

    Log::debug("[[email protected]{$site}] request", [ $request->all() ]);
    Log::debug("[[email protected]{$site}] request", [ $_REQUEST ]);

    Both lines produce the following in the log file:

    [2018-04-09 15:40:05] production.DEBUG: [[email protected]] request

    (there's not even "[]" after the "message" part and it's strange…)

  2. Later on, when I try to dispatch a Job with this data as payload, I get the following error:

    [2018-04-09 15:40:06] production.ERROR: exception 'InvalidArgumentException' with message 'Unable to create payload: Malformed UTF-8 characters, possibly incorrectly encoded' in /home/i/ivento/kartazamerov.ru/public_html/mappanel/vendor/laravel/framework/src/Illuminate/Queue/Queue.php:94

  1. Error stays the same even if I try to covert this data from the original charset to UTF-8:
        $payload = $request->all();
        if ('ThatDinozaurus' === $SiteName) {
            $payload = iconv('CP1251', 'UTF-8', json_encode($payload));
            $payload = json_decode($payload, true);

The task: to be able to handle those requests as well as other, good'ol utf-8 ones. I know which site sends non-standard charset. I have iconv at my disposal. But still something's wrong…

Please sign in or create an account to participate in this conversation.