I don't think you can override the HTML without overriding the underlying Vue component. It's pretty hard to do this right because it will be overridden on the next release.
If you only want to change the styling you can create your own theme or load an extra CSS file that overrides the styling of that component.
Theme: https://nova.laravel.com/docs/3.0/customization/themes.html#theme-css
For a custom CSS file, you can do this
// app/Providers/NovaServiceProvider
public function boot(): void
{
parent::boot();
Nova::style('admin', public_path('css/admin.css'));
}
Then you can simply create your own admin.css file that is generated by Laravel Mix or you can use plain CSS