@zness21 Hey Zac,
It sounds like you're almost there - you just need to take an extra step to get the data you want!
I'd double check that you're indeed requesting the OAuth scopes required to traverse the Google Apps directory stuff: https://laravel.com/docs/5.5/socialite#access-scopes
Once the user has granted access, you'll get a $token variable you can use to make future requests using Google's own PHP SDK:
$user = Socialite::driver('google')->user();
$token = $user->token;
// TODO: Install google/api-client via Composer
$client = new Google_Client();
$client->setApplicationName("My Application");
$client->setDeveloperKey("MY_SIMPLE_API_KEY");
$client->setAccessToken($token);
$service = new Google_Service_Directory($client);
// use $service to access your directory
Google has more docs on their Getting Started guide here: https://developers.google.com/admin-sdk/directory/v1/quickstart/php