Apparently looking at Laravel/Fortify/Actions/ConfirmTwoFactorAuthentication.php we can see that Fortify puts the error in confirmTwoFactorAuthentication error bag as below
throw ValidationException::withMessages([
'code' => [__('The provided two factor authentication code was invalid.')],
])->errorBag('confirmTwoFactorAuthentication');
Which I'm not sure why the need for the error bag if anyone could enlighten me. But for now I'll just have to update my code to below.