Level 80
@nolannordlund If you need to copy images from one S3 bucket to another, then why not just use the AWS CLI to do this?
May 12, 2021
12
Level 4
Save an image from a URL to S3
I have an application deployed on Vapor that needs to copy large numbers of images from a publically accessible S3 bucket to another under my own account. I'm getting the image URLs via API calls. It seems like this should be pretty easy to do but I don't see anything like Storage::put($filename, $url). Is there an easy way to do this or do I need to read the contents of the file and then upload to S3?
Level 4
This is something that I'll need to do consistently, possibly every hour. I could write a script to do that but I'd rather have that code within my application if possible.
Level 80
@nolannordlund Then why not just upload the files to the bucket you need them in? Instead of uploading them to bucket A, downloading them from bucket A just to upload it to bucket B. There’s not very efficient or cost effective.
Level 4
@martinbean I'm building a realty website. I'm accessing data through an MLS which hosts images of the listings in an S3 bucket. Listings are updated, added, and removed frequently. Whenever they do, I need to pull the images from their S3 bucket into my own. They don't allow serving images directly from their bucket.
Level 80
Can you not just set up a CloudFront distribution that uses their bucket as the origin? That way you’re not having to copy their images, but you’re also not serving directly from that bucket as requested.
Level 4
Maybe? I don't have much experience with CloudFront. Vapor sets up a CloudFront distribution but I assume this would require another distribution, correct? Does CloudFront care how many images are in a distribution?
If CF prices based on data or objects, it could be cost-prohibitive to do that because of the massive amount of listings in the MLS. Only a small portion of them matter for my client.
Level 80
CloudFront is a CDN. It’s not going to cost any more to serve an image through CloudFront as it is to upload and store an image in an S3 bucket 🙃
CloudFront will serve as many images as you need. It will also only serve the images you tell it to. The first time you request an image, it will grab it from the origin you specify (i.e. the service’s S3 bucket) and then serve subsequent requests for that image from its edge caches.
Level 4
Set it up this morning and it is working great. Thanks for the help
1 like
Level 4
The best solution I have to this right now is Storage::put($filename, file_get_contents($url));
Level 75
@nolannordlund does the multiple listing service have an API you can hook into to just show the images directly?
Level 4
No, they require that I have my own copy of the image.
Level 80
Which is what you would have if you served it from your own CloudFront distribution…?
Please or to participate in this conversation.