Am I doing this right

Hi I have just implemented a rating system for myself.
A have 3 model and tables: venue, user, review.
think of venue as a score.
my rating table looks like this:

Schema::create('reviews', function (Blueprint $table) {
            $table->increments('id');
            $table->integer('user_id')->unsigned();
            $table->foreign('user_id')->references('id')->on('users')->onDelete('cascade');
            $table->integer('venue_id')->unsigned();
            $table->foreign('venue_id')->references('id')->on('venues')->onDelete('cascade');
            $table->tinyInteger('quality');
            $table->tinyInteger('cost');
            $table->timestamps();
        });

Here the problem starts:

every time a user adds a review for a venue,I just sum all venues ratings, count all records number, sum venue's ratings,count venue's record numbers and calculate the rating in a special way and store venue's rating in an attribute called rating in venues table.
but this seems really inefficient without I just inserted 10000 rows in table in 2min without averaging and 7min with the above method.
How can I store a venues rating in a table and how to update it

Things I have tried but didn't help

1- Caching queries (reduced just 1min) 2- Making a queued job (reduced 3min)