Activity Streaks: How to return ONLY collection of Carbon objects from query scope?

I'm going to need to compare timestamps from multiple tables, the goal of which is to identify activity streaks.

For each table, I can of course grab all of the user's records, pull out the timestamp, add to array, compare, etc...

There are a lot of records though, so I'm looking to optimize where possible.

An example query scope:

    public function scopeAnsweredByUser($query, User $user)
    {
        return $query->where([
            ['user_id', $user->userID],
            ['answered_at', '!=', null],
        ]);
    }

And an example of its use:

$tableDates == Table::answeredByUser($user)->get(['answered_at'])->values();

This returns a collection of Table objects that contain only the answered_at attribute. What I'd like to get is a collection of Carbon objects that are not keyed by "answered_at", but rather just indexed.

I guess I could iterate over this collection, add the Carbon objects to an array, and then re-collect them? I'm looking for a way to end up with that result without that overhead though.

Is this a thing? Am I just looking to over-optimizing here?

The idea is, I'll run this type of query on a few tables, combine the results to get an ordered collection of distinct dates, and look for streaks of consecutive days.