Are you trying to scrape pages that require authentication? If so, you're going to need to supply the necessary cookies, headers, etc. in your cURL requests.
Nov 6, 2016
2
Level 22
Url Scraping like facebook in Laravel 5.3
Hello Guys i am using https://github.com/FriendsOfPHP/Goutte to scrape urls like facebook in laravel 5.3 but my trails are failing i had tried both http and https urls http gives a empty result and htttps gives curl 60 error can anyone help me, i have the following code in my routes.
use Illuminate\Http\Request; use Goutte\Client; Route::get('getFromUrl',function (Request $request){ if(Auth::check()){ $client = new Client(); $url = $request->url; $crawler = $client->request('GET', $url); return response()->json($crawler); } });
Please or to participate in this conversation.