How to automated web scraping with laravel

I know that goutte is a package for web scraping. But I need to automated scraping everytime the targeted sites updated. What comes to my mind is to use goutte and cronjobs, is this possible?? or to use Scrapy framework which is python.

cojamav720

5 years ago

Level 1

This Will Help You.

Here's my seeder, which populated my database ready for the main data extraction;

mabdullahsari

5 years ago

Level 16

Well, since Laravel uses Symfony behind the scenes you don't need to require any package to crawl web pages inside your code. Take a look at: https://symfony.com/doc/current/components/dom_crawler.html

Add use Symfony\Component\DomCrawler\Crawler; inside your job and you're good to go.

1 like

Please or to participate in this conversation.