Using html-loader or vue-html-loader for Vue files with Laravel Mix

I am trying to figure out how I would add a loader to *.vue files (either the html-loader, or vue-html-loader).

As I have understood it, vue-loader is pulled in by default, so what I would like to do is to extend the rule that applies this and add another loader here, whilst still keeping the original configuration.

Anyone that could point me in the right direction?