I looked around more, and one Alternative way could be using media collection with strict convention. Extending, model and filesystem only, is not enough in most cases to add soft delete ability.

// 1. Upload Media to 'files' collection:
$post->addMedia(request()->file('file'))->toMediaCollection('files');

// 2. When Fetching Media to show. Only load 'files' collection:
$post->getMedia('files');

// 3. When Deleting, perform soft delete. By moving to 'trash' collection:
$file = $post->getMedia('files')->first();
$file->move($post, 'trash');

// 4. Later, you can also easily clear 'trash' collection at once using:
$post->getMedia('trash'); // Gets all the trashed media
$post->clearMediaCollection('trash');