How to Centralize Laravel Logs from Multiple EC2 Servers (Managed via Forge) into a Single Server or Location?

Level 58

Certainly! Centralizing logs from multiple Laravel servers is a common need as your infrastructure scales. Here’s how you can achieve centralized logging for your Laravel apps running on multiple EC2 instances managed by Forge:

Solution Overview

The most common and robust approaches are:

Use a Log Aggregation Service (e.g., Papertrail, Loggly, Datadog, AWS CloudWatch Logs)
Self-host an ELK Stack (Elasticsearch, Logstash, Kibana) or use Graylog
Centralize to a Remote Syslog Server (using syslog or rsyslog)

Below are practical steps for each approach, focusing on Laravel configuration.

1. Using a Log Aggregation Service (Recommended for Simplicity)

Example: Sending Laravel Logs to Papertrail

Step 1: Create a Papertrail account and get your log destination (host:port).

Step 2: Update your Laravel logging configuration (config/logging.php):

Add a new channel:

'channels' => [
    // ... existing channels ...
    'papertrail' => [
        'driver' => 'monolog',
        'handler' => Monolog\Handler\SyslogUdpHandler::class,
        'handler_with' => [
            'host' => env('PAPERTRAIL_URL'),
            'port' => env('PAPERTRAIL_PORT'),
        ],
    ],
],

Step 3: Set your .env variables:

LOG_CHANNEL=papertrail
PAPERTRAIL_URL=logsN.papertrailapp.com
PAPERTRAIL_PORT=XXXXX

Step 4: Deploy this config to all your servers.
Now, all logs will be sent to Papertrail and you can view/search them in one place.

2. Using AWS CloudWatch Logs

If you want to keep everything within AWS, use the CloudWatch Logs Agent to ship logs from each EC2 instance.

Step 1: Install and configure the CloudWatch Logs agent on each EC2 instance.

Step 2: Configure the agent to watch storage/logs/laravel.log.

Step 3: View all logs in the AWS CloudWatch Logs console.

3. Self-hosted ELK Stack

Step 1: Set up an ELK stack (can be on a dedicated EC2 or managed via AWS Elasticsearch Service).

Step 2: Install Filebeat on each Laravel server to ship logs to Logstash.

Step 3: Configure Filebeat to watch storage/logs/laravel.log and send to your ELK server.

4. Centralize to a Remote Syslog Server

You can configure Laravel to send logs to a remote syslog server.

Step 1: Add a syslog channel in config/logging.php:

'channels' => [
    // ... existing channels ...
    'syslog' => [
        'driver' => 'syslog',
        'host' => env('SYSLOG_SERVER_HOST'),
        'port' => env('SYSLOG_SERVER_PORT'),
        'facility' => LOG_USER,
    ],
],

Step 2: Set your .env variables:

LOG_CHANNEL=syslog
SYSLOG_SERVER_HOST=your-central-log-server-ip
SYSLOG_SERVER_PORT=514

Step 3: Set up a syslog server (rsyslog, syslog-ng, or Graylog) to receive logs.

Best Practice

Use a managed log aggregation service for simplicity and reliability.
If you want to self-host, ELK or Graylog are popular choices.
For AWS-centric setups, CloudWatch Logs is seamless.

Summary

For most teams, using a service like Papertrail or CloudWatch Logs is the fastest and easiest way to centralize logs from all Laravel servers.
Just update your config/logging.php and .env as shown above, and you’ll have all logs in one place, searchable and filterable.

Let me know if you need a step-by-step for a specific service or more details on any of the above!

1 like