You should probably rethink the approach. Place the CSS file alongside the component file, and import it into the component itself. That way the CSS is only included in builds when the component is used, and each component is a reusable little package.
Since the CSS isn't relevant outside of the component, it shouldn't be included as a global style. It might still get bundled into the same CSS file, depending build settings, but now Vite knows that the CSS is tied that component. It's subject to tree shaking and can be split into a different CSS bundle if needed.