Developer654079525's avatar

MySQL sync

What is the prefered options of syncing a local MySQL db with the production one that sits on a shared hosting? We are using GIT, ssh and Composer to sync the framework, but the MySQL sync is a bit of a pain. Currently, we are using PHPMyAdmin to manually export on the dev machine and drop and import on a live hosting account. How can this be automated so that we don't have that brief downtime while syncing. Both the dev and the hosting machines are running Linux.

The shared hosting allows remote MySQL connections and has all the bash goodies as well. Is mysqldump in combination with scp and ssh the way to go?

0 likes
14 replies
Snapey's avatar

Your database changes should always be via migrations.

What scenario requires you to take the development database and overwrite production?

1 like
Developer654079525's avatar

I see. Well, the data is entered exclusively on a dev machine through an admin section. That's just a requirement we have. I should clarify, It's not the db structure that changes, but rather the db records/content itself.

JussiMannisto's avatar

Development and production databases should be completely separate. There should be no data transfer between them.

Schema changes are done through migrations, like @snapey said. You run migrations whenever you deploy changes to production.

How would "syncing" even work when there's production data? Let's say someone registers on your site. If you import a MySQL dump from development, that user account will vanish.

1 like
Developer654079525's avatar

I see, I get a clear picture now. It seems like the current workflow can/should be improved. Currently, we do not have user registration yet, and are using SQL dumps on a dev machine, the node machine, and manually importing them on live server.

But, if we were to sync in one direction only, what approach would you recommend? The mysqldump route?

Glukinho's avatar

Development and production databases should be completely separate. There should be no data transfer between them.

There are different requirements and dev workflows, maybe OP has non-public corporate app without registering at all. I can imagine cases where his pattern is applicable.

1 like
JussiMannisto's avatar
Level 50

You could write a bash script that generates a dump, transfers it to the production server, and imports it. But you can't restore a dump without any downtime.

You could develop directly on the production DB by connecting the dev app to it via an SSH tunnel. You'd get rid of downtime that way, but I wouldn't recommend it. If you goof something up, it's immediately felt in production.

Replication is used to keep multiple production databases in sync in real time, but it's not feasible here. Unless the dev app runs on its own server, so you can maintain a persistent connection with TLS.

I can't recommend a good solution because this isn't how live databases are designed to operate. I guess a script to automate the import is the least crazy option.

DigitalArtisan's avatar

To me, you could set up replication with master slave. One database as the source(dev), the other as the replica(prod).

Very common when you want or need to have a 'copy' of you data.

1 like
JussiMannisto's avatar

Very common? In what world?

How would you even set up replication from a local dev machine without a static IP?

DigitalArtisan's avatar

To clarify, I said replication is very common when you want a copy of your data. I wasn’t suggesting that syncing dev → prod in a shared hosting environment is common practice.

My point was simply that replication is widely used for maintaining data copies, and it’s a more appropriate term than “sync” in this context.

There are multiple ways to setup replication from a dev box without a static IP: VPN, SSH tunneling, Master/Slave, Pull/Push based, etc.

Anyways, I do not recommend what the OP is doing, nor would I want to.

I would use migrations and seeding to populate the production db.

Glukinho's avatar

Master-slave replication is not trivial to set up and brings problems if not constantly overseen. This is not for dev workflow, in my opinion.

OP should use simple script with backup-restore. Mysqldump is fine if database is not very large and some interruption is acceptable. Otherwise there is xtrabackup and other tools.

Snapey's avatar

I have a scenario with a 'warm' standby server located on a VPS in a different datacentre. It does not need to be synced in real time as all data is transient, but it does need to be daily synced.

A console command on the Live system uses Spatie DBDumper to create a dump of certain tables into a file on an S3 bucket. Both systems have access to the same bucket. At a time after the dump was created, the standby system pulls the file from S3 and applies it to its copy.

I've pasted the two commands below. They are pretty simple to understand.

DumpDB.php

PullDb.php

2 likes
ajinaniyan's avatar

If you’re trying to sync MySQL changes across Laravel environments, the usual approach is to rely on Laravel migrations rather than trying to manually sync the database. Make sure both environments are pointing to the correct database in the .env file, then run php artisan migrate to apply any pending schema changes. If things got out of sync badly, you can use php artisan migrate:status to see what’s missing, or in development use php artisan migrate:fresh to rebuild the schema from migrations. For data movement between environments, using mysqldump or Laravel seeders is usually safer than trying to copy tables manually.

Also check that both environments are running compatible MySQL versions and the same charset/collation settings, because mismatches there can cause sync issues.

If the sync problem is happening because some tables became corrupted or MySQL cannot read them properly, then native tools may fail to export them. In that situation, try some third party recovery tools like Stellar Repair for MySQL can sometimes help extract data from damaged tables before reimporting them into a clean database.

Snapey's avatar

The question was about syncing DATA not STRUCTURE

Please or to participate in this conversation.