BelongsTo relationship with pivot table from BelongsToMany relationship

I have a User model, which has a BelongsToMany lessons relationship:

class User extends Authenticatable
{
    /**
     * @return BelongsToMany
     */
    public function lessons(): BelongsToMany
    {
        return $this->belongsToMany(Lesson::class)->withTimestamps();
    }
}

In my Lesson model I have a BelongsTo user relationship:

class Lesson extends Model
{
    /**
     * @return BelongsTo
     */
    public function user(): BelongsTo
    {
        return $this->belongsTo(User::class);
    }
}

In my case every Lesson belongs to 1 User and a User belongs to many Lessons.

So I created a pivot table called lesson_user for this:

return new class extends Migration
{
    /**
     * @return void
     */
    public function up(): void
    {
        Schema::create('lesson_user', function (Blueprint $table) {
            $table->id();
            $table->foreignId('lesson_id')->unique()->references('id')->on('lessons');
            $table->foreignId('user_id')->references('id')->on('users');
            $table->timestamps();
        });
    }

    /**
     * @return void
     */
    public function down(): void
    {
        Schema::dropIfExists('lesson_user');
    }
};

Right now I am stuck on the user BelongsTo relationship in the Lesson model. Because normally it would require adding an user_id column to the lessons table.

But because I already have the pivot table for the lessons BelongsToMany relationship. It doesn't make sense to me to not make use of it for the BelongsTo relationship as well.

So my question is, how do I make sure that I can retrieve the user from the lesson, using the pivot table and not adding an user_id to the lessons table?

Like so:

Lesson::find(1)->user

Or is my approach wrong?