Accessing HTTPS Homestead sites with Laravel Dusk

@ORRD Were you ever able to resolve this? I'm having the exact same problem and it's proving rather frustrating. I'm surprised that more people don't have this problem, given that Homestead uses TLS for all sites by default currently.

I, too, am using a wildcard certificate, and while it is self-signed, it is trusted at the operating system level in Homestead (I add the CA to the trusted store upon provisioning).

From what I recall, I don't think we were able to find a solution.

Ours was a fully valid certificate, but I think it may be even less likely to work if it's self-signed. I'm pretty sure the Dusk environment doesn't use the same CA info as your host system, so I think that probably wouldn't work. I don't know that for sure, so I'm just making assumptions based on what I think I know about how Dusk works.

Thanks for the reply! I really appreciate it.

After further testing, I concur. It seems that the standalone "chromedriver" that ships with Dusk either lacks support for TLS entirely or lacks awareness of Ubuntu's CA certificate store (it may be using a separate Java store). It's also possible that the binary includes only common CA certificates, which would, of course, preclude the use of a self-signed certificate.

My hunch is that it will be necessary to install Selenium and chromedriver manually, or perhaps even build one or both from source, to add support for custom CAs.

I have no choice but to figure it out, so I'll post back whenever that may be.

I finally figured this out.

Apparently, on Ubuntu, Chrome looks to an obscure and completely non-obvious location for TLS certificate authorization overrides. Basically, the trick here was to determine how to replicate trusting a self-signed certificate in the GUI version of Chrome, but via the CLI.

(See this helpful article for more information: https://leehblue.com/add-self-signed-ssl-google-chrome-ubuntu-16-04/ )

For self-signed certificates, such as those that Homestead generates during provisioning, it is necessary to add them to this arcane sqlite database in the effective user's (in this case, the vagrant user's) Home directory before Chrome (and in turn, the standalone chromedriver) will trust the certificates.

Chromium's (and therefore Chrome's) certificate handling is described in moderate detail at https://chromium.googlesource.com/chromium/src/+/lkcr/docs/linux_cert_management.md .

Firstly, certutil must be installed:

$ sudo apt-get install libnss3-tools

The following commands must be executed as the vagrant user, assuming a Homestead environment (otherwise, as whomever the chromedriver will be running).

On a new system, it's possible that the certificate database does not yet exist, in which case it is necessary to create it before performing the subsequent steps:

$ mkdir -p $HOME/.pki/nssdb

$ certutil -N -d $HOME/.pki/nssdb --empty-password

(the --empty-password switch is necessary to automate this process)

To add a given certificate:

$ certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n /etc/nginx/ssl/site.example.com.crt -i /etc/nginx/ssl/site.example.com.crt

To confirm the addition (list all "accepted" overrides):

$ certutil -d sql:$HOME/.pki/nssdb -L

And if for any reason one desires to remove an override:

$ certutil -D -d sql:$HOME/.pki/nssdb -n /etc/nginx/ssl/site.example.com.crt

Now, my tests pass when the APP_URL is https://site.example.com!

In your case, given that you have a valid wildcard that is not self-signed, it must be that your certificate does not contain the SubjectAltName field, which is now required as of Chrome 58, or you're missing an intermediate CA or two in the trust chain (can occur with less common CAs).

If your certificate does include SubjectAltName, then you should use curl from the CLI in Homestead (or whatever environment you're running Dusk from within) to determine which intermediate CA certificate is missing from the trust chain.

@cbj4074 Very nice work, and helpful explanation. You should mark the post as solved so others can find it who might have a similar problem.

@Cronix Thank you! I'm not the OP, so I can't mark the thread as Resolved, but maybe @ORRD will. :D

Hmm, I'm not completely satisfied with this solution just yet.

For whatever reason, the certificate overrides are effective only when chromedriver is started from the command line, manually, e.g. /home/vagrant/code/laravel/vendor/laravel/dusk/bin/chromedriver-linux, and not when it's started from within Laravel via this method:

    /**
     * Prepare for Dusk test execution.
     *
     * @beforeClass
     * @return void
     */
    public static function prepare()
    {
        static::startChromeDriver();
    }

As yet, I have no idea why this is, given that PHP is running as the vagrant user when the Dusk test is invoked, and should therefore have access to the certificate database.

To workaround this bizarre limitation, I've commented-out the above line and am starting chromedriver in the background before running my Dusk tests.

I'd love to nail-down this odd anomaly!

I've reduced the issue noted in my previous post to the fact that the HOME=/home/vagrant environment variable appears not to be set when chromedriver is started using the startChromeDriver() method.

I had hoped that adding the following to the relevant phpunit.xml would fix the issue:

<php>
    <env name="HOME" value="/home/vagrant"/>
</php>

But no such luck. And I don't like having to hard-code that path in several places.

I ended-up leaving that line commented-out, installing chromedriver globally, and then adding it to the Supervisor config, in /etc/supervisor/conf.d/chromedriver:

[program:chromedriver]
environment=HOME=/home/vagrant
process_name=%(program_name)s_%(process_num)02d
command=/usr/local/bin/chromedriver
autostart=true
startretries=99
autorestart=true
user=vagrant
numprocs=1
redirect_stderr=true

Note the environment=HOME=/home/vagrant. This is the special sauce that enables chromedriver to find the user-trusted certificates in $HOME/.pki/nssdb.

With all of this in place, my Dusk tests pass with HTTPS enabled, even on a freshly-provisioned Homestead instance. :)

A couple points of note:

I have no idea why it is insufficient simply to add the self-signed Homestead CA cert (/etc/nginx/ssl/ca.homestead.homestead.crt) to the operating system's trusted CA store. Chrome seems not to look there... so, where from is it getting its "built-in" list of trusted CAs?

Similarly, adding the aforementioned file to the user-specific certificate store doesn't do the job, either; it is necessary to add each individual certificate to be trusted, even though they are all issued by the Homestead CA.

My understanding, based on https://chromium.googlesource.com/chromium/src/+/lkcr/docs/linux_cert_management.md , is that a CA can be trusted with the C type (in contrast to P). For example:

certutil -d sql:$HOME/.pki/nssdb -A -t "C,," -n /etc/nginx/ssl/ca.homestead.homestead.crt -i /etc/nginx/ssl/ca.homestead.homestead.crt

But that doesn't seem to do the job.

If either of these measures could be made to work, it would be simpler to trust every Homestead-CA-issued certificate on the box.

In the meantime, I've added the following to my Vagrant provisioning process:

#!/bin/sh

# Install the latest Chrome version to prevent "Chrome version must be >= X"
# when running Dusk tests.

sudo curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add
sudo sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
sudo apt-get -y update
sudo apt-get -y install google-chrome-stable

# Install the latest chromedriver globally; using the Composer-installed version
# from any given project is less reliable if using a mounted filesystem.

CHROME_DRIVER_VERSION=`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`
wget -N -nv http://chromedriver.storage.googleapis.com/$CHROME_DRIVER_VERSION/chromedriver_linux64.zip -P ~/
unzip -qq ~/chromedriver_linux64.zip -d ~/
rm ~/chromedriver_linux64.zip
sudo mv -f ~/chromedriver /usr/local/bin/chromedriver
sudo chown root:root /usr/local/bin/chromedriver
sudo chmod 0755 /usr/local/bin/chromedriver

# Add all Homestead-generated certificates to the "vagrant" user's trusted
# certificate store for Chrome. For more info:
# https://chromium.googlesource.com/chromium/src/+/lkcr/docs/linux_cert_management.md

sudo apt-get -y install libnss3-tools

# We don't want a password on the certificate database; see:
# https://bugzilla.redhat.com/show_bug.cgi?id=1401606

if [ -d "$HOME/.pki/nssdb" ]; then
    rm -rf $HOME/.pki/nssdb
fi

mkdir -p $HOME/.pki/nssdb
certutil -N -d $HOME/.pki/nssdb --empty-password

# Iterate through all Homestead-generated certificates and add them to the trusted
# store. Many of the certificates may be identical (because Homestead generates
# wildcards), in which case "certutil -d sql:$HOME/.pki/nssdb -L" command will
# show only one instance of each identical cert.

for file in /etc/nginx/ssl/*.crt; do
    [ -e "$file" ] || continue
    
    certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n "$file" -i "$file"
done