Cast a date for Laravel Nova, but return my own format

I have a Model that I want to use in Nova that has some data and datetime fields. Nova keeps complaining that I need to cast it to a Date type in my model, but I want to always return the format of dates as "Y-m-d" and datetimes as "Y-m-d H:i:s". I tried to cast the fields, and then edit the get attribute methods as follows, but the getAttribute methods seem to overwrite the casts:

protected $casts = [
    'date' => 'date',
    'start_time' => 'datetime',
    'end_time' => 'datetime',
    'actual_start_time' => 'datetime',
    'actual_end_time' => 'datetime'
];

...

public function getDateAttribute($value)
{
    return Carbon::parse($value)->format('Y-m-d');
}

public function getStartTimeAttribute($value) {
    return Carbon::parse($value)->format('Y-m-d H:i:s');
}

public function getEndTimeAttribute($value)
{
    return Carbon::parse($value)->format('Y-m-d H:i:s');
}

...

How can I cast to a date so Nova will be happy, but also always return in my desired format when returning the model instance to endpoints?