Automate filling forms external websites

Dusk uses a browser to run commands. It isn't just for testing. It's basically a scraper.

Do you know any tutorials where the use Dusk not as testing but for scaping sites? I found some sample code:

namespace App\Http\Controllers\Backend;

use Illuminate\Http\Request; use App\Http\Controllers\Controller; use App\Contract;

use Laravel\Dusk\Browser;

use Symfony\Component\Process\Process; use Symfony\Component\Process\ProcessBuilder; use Facebook\WebDriver\Chrome\ChromeOptions; use Facebook\WebDriver\Remote\RemoteWebDriver; use Facebook\WebDriver\Remote\DesiredCapabilities;

class WaterController extends Controller {

public function index()
{
    // Start chromedriver
    $chromeProcess = $this->buildChromeProcess();
    $chromeProcess->start();

    // Create a RemoteWebDriver Instance
    $driver = retry(5, function () {
        return $this->driver();
    }, 50);

    $browser = new Browser($driver);

    $browser->visit('https://www.website.nl/service/signup/sigugupproces#?step=1')
        ->type('PostalCode', 'Test')
        ->type('HouseNumber', 1);
}

public function driverSuffix() {
    switch (PHP_OS) {
        case 'Darwin':
            return 'mac';
        case 'WINNT':
            return 'win.exe';
        default:
            return 'linux';
    }
}
public function buildChromeProcess() {
    $driver = realpath(__DIR__.'/bin/chromedriver-'.$this->driverSuffix());
    if (realpath($driver) === false) {
        throw new RuntimeException("Invalid path to Chromedriver [{$driver}].");
    }

    $env = ['DISPLAY' => ':0'];
    if (PHP_OS === 'Darwin' || PHP_OS === 'WINNT') {
        $env = [];
    }
    return (new ProcessBuilder())
        ->setPrefix(realpath($driver))
        ->getProcess()
        ->setEnv($env);
}

public function driver() {
    return RemoteWebDriver::create(
        'http://localhost:9515', DesiredCapabilities::chrome(), 5000, 10000
    );
}

}

I run into the following problem:

Failed to connect to localhost port 9515: Connection refused

1/1) WebDriverCurlException

Curl error thrown for http POST to /session with params: {"desiredCapabilities":{"browserName":"chrome","platform":"ANY"}}

Failed to connect to localhost port 9515: Connection refused

I am working in an wamp environment on windows. Normal dusk testing works fine, but inside the controller I can't get it to work.

Yeah it's probably not the best idea to use it on the web. You also shouldn't use Dusk in production.

If you want to grab data from a URL, just curl the page and manipulate the contents that uses something like this https://github.com/paquettg/php-html-parser

Hmm oke.

But any other solutions to makes this manual job automated? It's a recurring job and the have to copy and past the data from the system to the supplier page to singup a person. So I dont want to scape anything I want to submit real data on the website.

I don't really know what you're asking now.

I will try to explain it.

We receive water contracts for houses. These contracts have to be registered at the water company that provides the water to the houses. This is an manual action for the company that we word for. So every day they receive around 30 new water contracts and they have to register them manually.

So normally we would think let's use there API so we can automate this process. But the problem is, these companies don't have API's.

That’s the reason we are looking for a solution to automate this. Dusk does this very well, except like you mentioned it shouldn’t be used on production.

What other options do we have?

Guzzle.

@Erwin Oh, you want to populate form inputs on another website. Right.

1. Try something like this: https://github.com/duncan3dc/dusk
2. Use Laravel Dusk and just have a local build that you can actually run commands from a database. Can use groups as well php artisan dusk --group=filler:water

Thx @bashy. Will test it tomorrow.