OrWhereIn on Pivot Table Using Query Builder

I have a table of 'projects' with each project belonging to a primary group (e.g. belongsTo(Group::class)) associated by the column 'group_id' of which any users in that group have access to those projects. However, each project can also be shared with other groups, and these (id's) are stored in the pivot table 'group_project' with the two columns 'group_id' and 'project_id'. Whilst it's easy to retrieve all projects associated with the primary group using query builder, I don't know how I can also return projects that are also shared with other groups (i.e. having those projects visible to other users who are not members of the primary group, but are in any of the shared groups).

Here is the first part of the query:

Project::select('projects.*')
->whereIn('projects.group_id', auth()->user()->groups()->pluck('id')->toArray()))

But I now need to add an additional 'orWhereIn' clause on the pivot table that will also pull in all projects are shared with other groups e.g. something like...

->orWhereIn('pivot.group_id', auth()->user()->groups()->pluck('id')->toArray()))

All help gratefully received!