I'm not sure, but I think you have to set two hasMany relationships in the User model and naming the foreign keys explicitly. Like so:
function playerone() {
return $this->hasMany('App\Games', 'foreign_key', 'local_key');
}
function playertwo() {
return $this->hasMany('App\Games', 'foreign_key', 'local_key');
}
However, this won't group the results, but since they come from both datasets they should be easy to merge together (https://laravel.com/docs/master/collections#method-merge).
GroupBy is a method you can use on an Eloquent collection if I recall correctly (https://laravel.com/docs/5.1/collections#method-groupby)