Estimating IOPS

I am having issues with my RDS instance maxing out its IOPS when we are competing long running report operations. I am currently only running a single script, 1 at a time and I am surprised that I am hitting 3000 IOPS over a prolonged period of time (hours) and getting bursts up to 7000 IOPS!

As far as I understand, an input/output operation to the RDS instance is a read or write query?

I have checked my script locally with the Laravel debug bar, and it contains 446 queries for the 1st run and then 204 queries for all subsequent runs.

I have it running in a queue with just a single worker (i.e. concurrency of 1) and checking on my logs and over the course of 10 minutes we can run 282 of these jobs.

By my calculations that is 408 + (204 x 281) = 57,732 queries in 10 minutes, 5,773 per minute or 96 per second.

What am I missing here to account for the 3000 - 7000 IOPS as registered by AWS monitoring?

booni3

6 years ago

Level 3

OK, so I see that a single query is not actually a single IOP.

The table I am inserting into has 7 columns excluding the create/update and it has 4 indexes. From what I have read, indexes have at least 1 IOP each and there will be a few other operational IOP's. Even If I said 10 IOP per transaction, that still only calculates to around 960 IOPS - much less than what the RDS monitoring is telling me.

Is there anything else I can check or optimise to try and reduce this. I really do not want to have to switch to a provisioned IOPS instance.

Please or to participate in this conversation.