Sorting by the count of related models

I have two models:

class Photo extends Model
{
    public function likes()
        {
            return $this->morphMany('App\Like', 'likeable');
     }
}

class Like extends Model
{
    public function likeable()
        {
            return $this->morphTo();
        }
}

Here is a method for getting the paginated and ordered list of photos:

$photos = Photo::orderBy($request->input('sort', 'id'), $request->input('order', 'asc'))
                        ->paginate($request->input('limit', 20));

And now I want to get the list of Photos ordered by the number of likes: /photos?sort=likes&order=desc

The most simple way I think is to get all photos from the DB and apply the sortByDesc method to the Collection. But it could be too hard for the server if the list of photos is too big and also there is a problem with pagination.

Is there an elegant way to reach this?