Minimal bootstrap for webhooks processing

Hi,

Is there a way to load "as less Laravel as possible" when processing a webhook? The objective here is to send the response asap otherwise the external system would send the same webhook again and eventually would disable webhooks at all.

Note: I'm using queued jobs already, so my controller looks like this:

class WebhooksController extends Controller
{
    public function process(Request $request)
    {
        if ($request->has('leads.update')) {
            // do not process this kind of hooks…
            return response("Ok", 200);
        }

        try {
            // Cache-related routines:
            $job = new \App\Jobs\UpdateLeadJob($request->all());
            $this->dispatch($job);
        } catch (\Exception $e) {
            Log::critical("[webhooks] dispatching primary job failed: {$e->getMessage()}", $request->all());
        }

        try {
            // Custom webhook processors:
            $customJob = $this->getCustomJob($request);
            if ($customJob) {
                $this->dispatch($customJob);
            }
        } catch (\Exception $e) {
            Log::critical("[webhooks] dispatching custom job failed: {$e->getMessage()}", $request->all());
        }

        $executionTime = microtime(true) - $_SERVER["REQUEST_TIME_FLOAT"];
        if (1.9 < $executionTime) {
            Log::error("[webhooks] webhook processing took more than 2 secs!", $request->all());
        }

        return response("Ok", 200);
    }

    // ...

}

Still, it happens sometimes that it's not fast enough. So, I can usually see several "webhook processing took more than 2 sec" messages in the logs.

So, probably I need a way to disable middlewares or something else… Is there any way to reduce the application loading time for this single action?

Note: I'll still need means to analyse request (like that if ($request->has(..))) and to dispatch queued jobs.

lostdreamer_nl

7 years ago

Level 53

I think you should be checking into your server configuration / HD IO etc.

I'm getting < 40 ms response on normal pages with a fully loaded laravel app including queries etc. without caching on a 'regular' vm (1 CPU / 2GB RAM / SSD)

If you're getting 2 second response times I can only assume you are doing some heavy jobs within your UpdateLeadJob or your $customJob and have setup your laravel to handle queues synchronisely.

pilat

7 years ago

Level 41

Both the Jobs are queued. So, the controller's job is only to receive the webhook, analyse it a little bit (reject 'leads.update' ones; find if there is a custom job available) and dispatch queued jobs. The dispatched jobs are then executed in a separate thread/process (in a worker).

Overall, the server is pretty loaded, yes. But… I think that upgrading hardware is not a proper way to solve this certain issue :-) There is no heavy-loading at all, in this specific action: as it only needs to add a record to "jobs" table and immediately response with 200 - Ok status. Just wondering if I really need to load "bootstrap whole laravel application" for doing so…

lostdreamer_nl

7 years ago

Level 53

"Just wondering if I really need to load "bootstrap whole laravel application" for doing so…"

Not really needed (check out lumen for instance), you could remove a lot of the providers in config/app.php and then only load them via your own AppServiceProvider if the route does not match one of your API routes.

But I still think this is more about HD IO (or something else).

Have you tried the laravel debugbar to see where the time is most spent? Any page that takes more then a second is not working OK imo.

I've had a lot of servers (mostly cloud based) where the HD would be on another part of the network then the CPU / DB. having a slow HD will seriously increase your load times.

Perhaps running the optimization commands might already give you a nice boost:

https://laravel.com/docs/5.7/deployment#autoloader-optimization

composer install --optimize-autoloader --no-dev
php artisan config:cache
php artisan route:cache

Please or to participate in this conversation.