You are working with Collections here; just use the Collection diff method instead of array_diff:
$notActivatedUsers = $users->diff($activatedUsers);
However, it would be more performant to make just one query (basically the opposite of User::role('activated')->get() however that is implemented, e.g.
$notActivatedUsers = User::hasntRole('activated')->get()