Robstar's avatar
Level 50

Partially restoring a large compress database dump

At the moment I'm taking compressed backups (.sql.gz) of a MySQL database. This works fine.

Does anyone have advice on how I could extract a single table from the backup? When the backup is uncompressed it is very larger (currently ~37GB).

At the the monment doing this is pretty slow.

FYI I did the fopllowing without luck:

# Compress file
sed -n -e '/CREATE TABLE.*`users`/,/Table structure for table/p' db.sql.gz > table.sql
# On the un compressed file
zcat db.sql | awk '/CREATE.*TABLE.*users/,/^)/' > table.sql
0 likes
1 reply
LaryAI's avatar
Level 58

To extract a single table from a compressed MySQL database dump, you can use a combination of zgrep, awk, or sed to filter out the contents of the specific table from the compressed file without having to decompress the entire file. Here's a step-by-step solution:

  1. Use zgrep to find the starting line of the table creation and the ending line of the table dump.
  2. Use sed to extract the lines between the start and end found by zgrep.
  3. Optionally, pipe the output to mysql to import the table directly into your database.

Here's an example of how you might do this:

# Define the table name you want to extract
TABLE_NAME="your_table_name"

# Define the database dump file name
DUMP_FILE="your_dump.sql.gz"

# Find the line number for the start of the table dump
START_LINE=$(zgrep -n "Table structure for table \`${TABLE_NAME}\`" $DUMP_FILE | cut -d':' -f1)

# Find the line number for the end of the table dump
# This assumes that the dump for the next table or the end of the file follows the dump of your table
END_LINE=$(zgrep -n -m 1 "Table structure for table" $DUMP_FILE | grep -A 1 "$START_LINE" | tail -n1 | cut -d':' -f1)

# If the table is the last one in the dump, you might not find an end line. In that case, you can omit the end line from the sed command.

# Extract the table dump using sed. If an end line was found, use it, otherwise print to the end of the file.
if [ -n "$END_LINE" ]; then
  zcat $DUMP_FILE | sed -n "${START_LINE},$((END_LINE-1))p" > ${TABLE_NAME}.sql
else
  zcat $DUMP_FILE | sed -n "${START_LINE},$p" > ${TABLE_NAME}.sql
fi

# Now you have the table structure and data in a file named `your_table_name.sql`
# You can import this into your database using the mysql command:
mysql -u username -p database_name < ${TABLE_NAME}.sql

Please note that this script assumes that the table dumps are in a specific order and that the "Table structure for table" line is unique to the start of each table dump. If your dump has a different format, you may need to adjust the grep patterns accordingly.

Also, be aware that this method can still take some time to process, especially for very large tables, because it involves scanning through the compressed file. However, it should be faster than decompressing the entire 37GB dump file.

Please or to participate in this conversation.