QueryException General error: 1096 No tables used

I am trying to get all users but the condition is that they must have a valid role, the selected user should not be logged in user and the third condition is in whereNotIn clauses closure function. every thing else is working fine but for whereNotIn clauses, I'm getting an error like 1096 No tables used. I don't know what's wrong I'm doing.

here, is my code snippet.

                $allUsers = User::orderBy('users.created_at', 'asc')
                            ->join('role_users', 'users.id', '=', 'role_users.user_id')
                            ->join('roles', 'role_users.role_id', '=', 'roles.id')
                            ->where('role_users.role_id', '=', '3')
                            ->where('users.id' , '!=', $userData['user_id'])
                            ->whereNotIn('users.id', function($query) use ($addressId) {
                                $userId = PAddressDetail::find($addressId)->colistAgent()->get()->pluck('id')->toArray();
                            })->get()->toArray();
                dd($allUsers);