Accessors using Query Builder

Hey, I have been trying to use an accessor with query builder, but no luck so far.

I have tables that have a column called user_id and others client_id that relate to this polymorphic table.

With eloquent, since I have appends and accessors I have no problems but with query builder I get that the column does not not exist.

This is how I am trying with query builder:

$query->whereHasMorph('target', '*', function ($query) use ($request) {
                $query->where('user_id', $request->user()->id);
            });

Any way to make this work with either user_id or client_id column? Thanks in advance.