Test failed when running queries in Service Provider

It is not necessarily bad; obviously you have decided that this Config needs to be stored in the database, but then also needs to be available relatively early in the request lifecycle whenever the app is running normally. Testing is a special scenario, so the same expectation does not apply. You need to find a way to setup the application with a valid config (e.g. from a fixture array) if it is needed in this circumstance. But how you solve this very much depends on how you use that config.

Before trying to use data from a table that might not exist, you can check for the testing app environment:

if (app()->environment('testing')) {
	// do stuff
}

or, perhaps better to check for the presence of the table

if (Schema::hasTable('config')) {
	// do stuff
}

Is running db query in service provider a bad practice?

Yes.

I want to have configuration (stored in the database) set & available everywhere

Instead, consider creating a repository class that you can inject in classes that need to make use of your database-backed configuration settings. You can wrap it in caching so you’re not hitting your database on every request just to read some strings from your database on each request.