I got it working by extracting the code for client generation from the passport command and adding it to the test:
$clientRepository = new ClientRepository();
$client = $clientRepository->createPersonalAccessClient(
null, 'Test Personal Access Client', 'http://localhost'
);
DB::table('oauth_personal_access_clients')->insert([
'client_id' => $client->id,
'created_at' => new DateTime,
'updated_at' => new DateTime,
]);
This creates an access client and stores it in the database. Since I am using in-memory sqlite for unit tests, this needs to be done every time the unit test is executed.