load test failing for 500 concurrent users?

So my setup is:

1. load balancer server (512 MB)
2. site1 & site2 = 1GB each (separate servers)
3. Cache on site 1 using REDIS.. Site2 connected to the redis on Site1

i have a url such as api.com/users/me which hits Wordpress API (But there is a cache on my side so after the first request the response time is < 40 ms) ( i am using laravel instead of lumen..)

i have tried 100 users and it seems to be fine with 0 error rate. but when i increase this to 500 users the test server says aborting due to high error rate (error rate means there was no response)...

Any suggestions? should i be adding more servers for 500 users (ie my expectations are high)

Here is the result if interested: http://bit.ly/2d6bgaQ