How to copy directory from local to remote server?

I have a website to automatically build a new website for users. User only have to submit their VPS host, port, user, password, theme name. I have prepare different pre-build theme files together with configured CMS. Once they submit the VPS infos, the rest of procedure is fully automatic.

Currently this code works but its slow because it copies file one by one.

Any better way to perform this? I tried to use rsync but as I cannot create SSH keys, even if I have password, it will pump "save the fingerprint" confirmation. Please advise, thank. Its also ok to use package for this.


    public function copy_theme_files()
    {

        // create path /www/wwwroot/$this->cname
        $this->source_dir = '/www/templates/maccms/' . $this->manual_order->theme->name;
        $this->target_dir = '/www/wwwroot/' . $this->manual_order->domain;

        $local_server = new Filesystem;
        $remote_server = Storage::createSftpDriver([
            'driver'   => 'sftp',
            'host'     => $this->manual_order->ip,
            'port'     => (int) $this->manual_order->port,
            'username' => $this->manual_order->root_user,
            'password' => $this->manual_order->root_password,
        ]);
        
        if(!$this->copy_directory($local_server, $remote_server, $this->source_dir, $this->target_dir)){
            dd('copy failed');
        };

       dd('DONE');
    }

    public function copy_directory($local_server, $remote_server, $source_dir, $target_dir)
    {
        // Check if the source directory exists locally
        if (!$local_server->exists($this->source_dir)) {
            $this->show_info("Source directory does not exist");
            return;
        }

        // Check if the target directory exists remotely
        if (!$remote_server->exists($this->target_dir)) {
            $this->show_info("Target directory does not exist");
            $remote_server->makeDirectory($this->target_dir);
            $this->show_info("Created target_dir in remote server");
        }

        // Copy the source directory to the target directory
        // $remote_server->copyDirectory($this->source_dir, $this->target_dir);

        // Get all files and sub-directories in the source directory
        $files = $local_server->allFiles($this->source_dir);

        // Loop through each file and sub-directory and upload them to the remote server
        foreach ($files as $file) {
            $relative_path = substr($file, strlen($this->source_dir) + 1);
            $remote_path = $this->target_dir . '/' . $relative_path;

            // Check if the file already exists remotely
            if ($remote_server->exists($remote_path)) {
                $this->show_info("File already exists on remote server: " . $relative_path);
                continue;
            }

            // Upload the file to the remote server
            if ($local_server->isFile($file)) {
                $remote_server->put($remote_path, $local_server->get($file));
                $this->show_info("File uploaded to remote server: " . $relative_path);
            }
            // Create sub-directories remotely
            elseif ($local_server->isDirectory($file)) {
                $remote_server->makeDirectory($remote_path);
                $this->show_info("Directory created on remote server: " . $relative_path);
            }
        }

        $this->show_info("Directory transfer complete");
        return true;
    }

LaryAI

3 years ago

Level 58

One way to improve the performance of copying files from local to remote server is to use the scp command instead of copying files one by one. Here's an example of how to use scp in PHP:

public function copy_theme_files()
{
    // create path /www/wwwroot/$this->cname
    $this->source_dir = '/www/templates/maccms/' . $this->manual_order->theme->name;
    $this->target_dir = '/www/wwwroot/' . $this->manual_order->domain;

    $command = sprintf(
        'scp -r -P %d %s %s@%s:%s',
        $this->manual_order->port,
        $this->source_dir,
        $this->manual_order->root_user,
        $this->manual_order->ip,
        $this->target_dir
    );

    exec($command, $output, $return_var);

    if ($return_var !== 0) {
        dd('copy failed');
    }

    dd('DONE');
}

This code uses the exec function to run the scp command with the -r option to copy directories recursively, and the -P option to specify the port number. The source directory and target directory are passed as arguments to the command, along with the remote server's IP address and root user credentials. The output and return value of the command are captured in variables for error handling.

earnbyshare2016

3 years ago

Level 1

I also tried to zip the file. It copies quickly but I don't know how to unzip on a remote server without SSH keys (only have password)

psrz

3 years ago

Level 10

Instead of looping over $files = $local_server->allFiles($this->source_dir); split that in chunks and send each chunk to a job

That should make it faster since there'll be as many processes Horizon is set to manage transfering files, but also depends how big the files are and how fast the internet connection is...

Please or to participate in this conversation.