Limit eager loaded relationships

My posts have a comments relationship, and the comments have a recursive relationship to themselves:

 public function comments()
    {
        return $this->morphMany(Comment::class, 'commentable')-with('comments');
    }

This will get a post's comments and all the comment's replies.

I wish to constrain the nested relationship, e.g.

 public function comments()
    {
        return $this->morphMany(Comment::class, 'commentable')->with(['comments' => function($query) {
            $query->orderBy('id')->cursorPaginate(5);
        }]);
    }

This is so I only load 5 replies to any comment. With cursor pagination I can then have the user hit a load more button and continue loading comments in a particular chain.

Unfortunately the above does not work. I have also looked at https://github.com/staudenmeir/eloquent-eager-limit but it does not work with pagination.

Can anyone point me in the right direction?