How to fetch and normalize data from multiple supplier APIs (JSON/XML) in parallel using Laravel?

Certainly! Here’s a comprehensive solution for your scenario, focusing on clean architecture, scalability, and robust error handling.

---

1. Parallel HTTP Requests in Laravel

Laravel’s HTTP client supports concurrent requests via Http::pool(). This is efficient for your use case.

use Illuminate\Support\Facades\Http;

$responses = Http::pool(fn ($pool) => [
    'supplier1' => $pool->timeout(15)->get('https://api1.example.com/search', ['q' => $query]),
    'supplier2' => $pool->timeout(20)->get('https://api2.example.com/search', ['q' => $query]),
    'supplier3' => $pool->timeout(10)->get('https://api3.example.com/search', ['q' => $query]),
    // Add more as needed
]);

• Timeouts: Set per-request to avoid slow APIs blocking the whole process.
• Fallbacks: If a request times out or fails, you can skip or log it.

---

2. Handling JSON and XML Responses

Create a Supplier Adapter for each API. Each adapter handles:

• Making the request
• Parsing the response (JSON/XML)
• Normalizing to a common structure

Example Adapter Interface:

interface SupplierAdapter
{
    public function search(string $query): array;
}

JSON Adapter Example:

class JsonSupplierAdapter implements SupplierAdapter
{
    public function search(string $query): array
    {
        $response = Http::timeout(15)->get('https://api1.example.com/search', ['q' => $query]);
        if ($response->successful()) {
            $data = $response->json();
            return $this->normalize($data);
        }
        return [];
    }

    protected function normalize(array $data): array
    {
        // Map $data to your unified structure
    }
}

XML Adapter Example:

class XmlSupplierAdapter implements SupplierAdapter
{
    public function search(string $query): array
    {
        $response = Http::timeout(20)->get('https://api2.example.com/search', ['q' => $query]);
        if ($response->successful()) {
            $xml = simplexml_load_string($response->body(), "SimpleXMLElement", LIBXML_NOCDATA);
            $json = json_decode(json_encode($xml), true);
            return $this->normalize($json);
        }
        return [];
    }

    protected function normalize(array $data): array
    {
        // Map $data to your unified structure
    }
}

---

3. Aggregating Results in Parallel

You can use Laravel’s Bus::batch() for true parallelism (jobs/queues), but for most cases, Http::pool() suffices.

Example Aggregator:

class SupplierAggregator
{
    protected $adapters;

    public function __construct(array $adapters)
    {
        $this->adapters = $adapters;
    }

    public function search(string $query): array
    {
        $results = [];

        // Run all adapters in parallel
        $responses = Http::pool(fn ($pool) => array_map(
            fn ($adapter) => $pool->async(fn () => $adapter->search($query)),
            $this->adapters
        ));

        foreach ($responses as $response) {
            // Each adapter returns [] on failure
            $results = array_merge($results, $response ?? []);
        }

        return $results;
    }
}

---

4. Timeouts, Fallbacks, and Error Handling

• Each adapter sets its own timeout.
• If an API fails or times out, return an empty array or log the error.
• The aggregator merges only successful results.

---

5. Scalable Design Pattern

• Adapter Pattern: Each supplier gets its own adapter class implementing a common interface.
• Service Container: Bind adapters in Laravel’s service container for easy extension.
• Configuration: Store supplier configs (URLs, timeouts) in config files.

---

6. Example Controller Usage

public function search(Request $request)
{
    $query = $request->input('q');

    $aggregator = new SupplierAggregator([
        new JsonSupplierAdapter(),
        new XmlSupplierAdapter(),
        // Add more adapters as needed
    ]);

    $results = $aggregator->search($query);

    return response()->json([
        'results' => $results,
    ]);
}

---

7. Real-World Tips

• Logging: Log slow or failed suppliers for monitoring.
• Caching: Optionally cache results to reduce load.
• Queueing: For very slow APIs, consider dispatching jobs and returning partial results.

---

8. Summary

• Use Http::pool() for concurrent requests.
• Use the Adapter pattern for supplier-specific logic.
• Normalize all responses to a common structure.
• Handle timeouts and errors gracefully.
• Easily add new suppliers by creating new adapters.

---

This approach gives you a clean, scalable, and maintainable system for aggregating data from multiple, heterogeneous APIs.