Eloquent User Provider Causing Duplicate Query

I'm required to calculate the current 'rating' of a user (model) based on a relationship with Activity(model). I therefore have the following in my User model:

protected $with = ['lastActivity'];
protected $appends = ['rating'];

public function lastActivity() {
  return $this->hasOne(Activity::class, 'user_id')->latest();
}

public function getRatingAttribute() {
    $user_points = $this->lastActivity->total;
        return ($user_points >= 1000) ? "Level 1" : "Level 2";

}

Because this needs to be used site-wide, I've used $appends. The only problem is that this is causing duplicate query on my blog show pages. The first one is for rendering the user in the site's top navigation

/vendor/laravel/framework/src/Illuminate/Auth/EloquentUserProvider.php:52 :

Query: 
select * from `activity_points` where `activity_points`.`user_id` in (1) order by `created_at` desc

and the second query is being run when I'm loading blog post along with replies: -

My statement -
$post   =   Post::where('id', $post)->with(['author',  'replies.author', 'replies.likes.giver'])->withCount('likes')->first();


app/Http/Controllers/Blogs/PostsController.php:44
Query:
select * from `activity_points` where `activity_points`.`user_id` in (1) order by `created_at` desc

What could be a possible fix for this?