Use a JOIN rather than using the relationship method for the ordering, e.g.
$users_with_lastest_notification = User::with('latest_notification')
->select('users.*) // only want user fields in User model instances
->join('notifications', 'notifications.user_id', 'users.id')
->orderByDesc('notifications.created_at')
->get();
Aside, you can redefine the relationship using the one-of-many approach:
public function last_notification(): HasOne
{
return $this->hasOne(Notification::class, 'sender_id', 'id')->latestOfMany();
}