How to detect memory leak source with Octane?

Hi,

We're running a production app using Octane (Swoole) on Forge.

I have been getting few Sentry reports on memory leakage exceptions once every few days on random API routes. The exceptions are neither happening on consistent times or routes, which makes this difficult for me to find the piece that break the puzzle.

Example:

Allowed memory size of 268435456 bytes exhausted (tried to allocate 8192 bytes)

I have checked the routes at which the exception is thrown, and there is nothing on them that may produce Octane memory leak. Some of them is very primal API routes that doesn't do much work.

For context, we're using Laravel as an API, a Filament dashboard, and we cache lots of stuff, but the cache is on Redis. The overall server memory usage never exceeds 40%.

From the logs, the error is coming from these sources. I understand Redis is involved, but I don't understand why.

PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 356352 bytes) in /home/forge/x/vendor/predis/predis/src/Connection/StreamConnection.php on line 346
   INFO  {"status":500,"message":"Allowed memory size of 268435456 bytes exhausted (tried to allocate 356352 bytes)","errors":[]}.  

PHP Fatal error:  Allowed memory size of 268435456 bytes exhausted (tried to allocate 16384 bytes) in /home/forge/x/vendor/laravel/framework/src/Illuminate/Cache/RedisStore.php on line 488
   INFO  {"status":500,"message":"Allowed memory size of 268435456 bytes exhausted (tried to allocate 16384 bytes)","errors":[]}.

Aterniad

2 weeks ago

Level 1

Hi, I've found your topic while trying to solve the same problem on Octane (Swoole) app.

Memory leaks aren't easy to debug, but I'll try to point few key steps that helped out project:

If caching is involved (Memcached in our case but still), make sure you store scalar types or simple arrays. Serializing/deserializing values during I/O increases memory usage, especially with large datasets.
Check the "max-requests" setting when running Octane. If this value is too high, memory leaks can occur in wseemingly unrelated parts of the code even if there aren’t many leaks overall. I'd compare working with "max-requests" setting to a train, where one request can reserve seats for any number of people. Two heavy requests might fill the train to 98% capacity, and then one simple request might temporarily require 3%, overflowing the memory and causing the error to point to unrelated code.
Look for any usage static variables usage (simple pattern "::$"), local caches, identity maps and similar. They surely will grow in size between different requests, avoid such cases.

Hope this helps someone. Also, sorry for my English

Please or to participate in this conversation.