In this blog post, we’ll find out how to use Percona XtraBackup on a MySQL instance with a large number of tables.
As of Percona Xtrabackup 2.4.5, you are required to have enough open files to open every single InnoDB tablespace in the instance you’re trying to back up. So if you’re running innodb_file_per_table=1 , and have a large number of tables, you’re very likely to see Percona XtraBackup fail with the following error message:
If you run into this issue, here is what you need to do:
I would add at least another 1000 to this number for system tablespace and other miscellaneous open file needs. You might want to go even higher to accommodate for a growing number of tables.
Alternatively, you can pass it as a command-line option, or run ulimit -n 2000000 before running the backup command.
You need to be sure your user account has permissions to set open files limit this high. If you are doing backups under the “root” user, it shouldn’t be a problem. Otherwise, you might need to adjust the limits in /etc/security/limits.conf :
Specifying a “soft” limit in this file eliminates the need to run ulimit before Percona XtraBackup, or specifying it in the configuration.
The limit I have on this system is slightly above 1 million. You can increase it using the following:
With these configuration adjustments, you should be able to use Percona XtraBackup to backup MySQL instances containing millions of tables without problems.
What if you can’t allow Percona XtraBackup to open that many files? Then there is the option –close-files that won’t normally require increasing the limit to the number of open files. Using this option, however, might cause the backup corruption if you’re doing DDL operations during the backup.
From where does this strange limitation requiring you to keep all tablespaces open come? It comes from this issue . In some cases, DDL operations such as RENAME TABLE might cause the wrong file to be copied, and unable to be caught up by replying to InnoDB redo logs. Keeping the file open clearly shows which file corresponds to a given tablespace at the start of a backup process, and gets handled correctly.
This problem is not unique to Percona XtraBackup. If anything, Percona Xtrabackup goes the extra mile to ensure database backups are safe. For comparison, MySQL Enterprise Backup 4.0 simply states :
“ Do not run the DDL operations ALTER TABLE , TRUNCATE TABLE , OPTIMIZE TABLE , REPAIR TABLE , RESTORE TABLE or CREATE INDEX while a backup operation is going on. The resulting backup might become corrupted .”
Peter managed the High Performance Group within MySQL until 2006, when he founded Percona. Peter has a Master's Degree in Computer Science and is an expert in database kernels, computer hardware, and application scaling.
Percona’s widely read Percona Database Performance blog highlights our expertise in enterprise-class software, support, consulting and managed services solutions for both MySQL® and MongoDB® across traditional and cloud-based platforms. The decades of experience represented by our consultants is found daily in numerous and relevant blog posts.