Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
All types of backups and restore
1. Contents:
A. Backup a single Database
B. Backup Multiple Databases
C. Backup all the Databases
D. Restore all the Databases
E. Restore a Single Database
F. Backup a specific Table
G. Import data in MySQL.
H. Backup (Script on Linux)
I. Import and Export on MySQL Database
J. MySQL Backups on Windows
K. Different types of Backups for MySQL databases
L. Dump MySQL database into a new database on another server
M. How to dump local MySQL data to remote MySQL server through terminal?
N. Loading a SQL file into MySQL
O. Loading CSV or tab delimited files into MySQL
P. Mysqldump Upgrading to 4.1
Q. Simple backup options
2. (A) Backup a single database:
Using mysqldump, you can backup a local database and restore it on a remote database
at the same time, using a single command. In this article, let us review several practical
examples on how to use mysqldump to backup and restore. For the impatient, here is the
quick snippet of how backup and restore MySQL database using mysqldump:
Backup: #mysqldump –u root –p[root password] [database name] >dumpfilename.sql
Restore: #mysql –u root –p[root password] [databasename] < dumpfilename.sql
This example takes a backup of sampledb database and dumps the output to dbdump.sql
[mysql@localhost]# /usr/bin/mysqldump –u root –pmysql sampled > /tmp/dbdump.sql
[mysql@localhost]# mysqldump –u root –p[rootpassword] [databasename] >
/tmp/dbdump.sql
http://www.techiecorner.com/31/how-to-restore-mysql-database-from-sql-dump-file/
http://www.cyberciti.biz/faq/mysql-datadir-files-stored-unix-linux/ (location of data-dir)
After you press enter to execute the command, the backup will be generated. If you
browse to this path: /tmp, then you should see dbdump.sql in there.
(B) Backup multiple databases:
If you want to backup multiple databases, first identify the databases that you want to
backup using the show databases as shown below:
# mysql -u root -ppassword
mysql> show databases;
3. +--------------------+
| Database |
+--------------------+
| information_schema |
| newdb |
| mysql |
| sampledb |
For example, if you want to take backup of both sampledb and newdb database, execute
the mysqldump as shown below:
[mysql@localhost]# mysqldump –u root –p[rootpassword] --databases
sampledbnewdb> /tmp/dbdump.sql
Verify the dbdump.sqldump file contains both the database backup.
#grep –I “Current database:” /tmp/dbdump.sql
--Current database: ’newdb’
--Current database:’sampledb’
(C) Backup all the databases:
The following example takes a backup of all the database of the MySQL instance.
# mysqldump -u root -ptmppassword --all-databases > /tmp/all-database.sql
(D) Restore all the databases:
(How to restore mysqldump –all-databases backup ?)
[mysql@INVIRH54DB3 ~]$ mysql -u root -p < /tmp/alldbs55.sql
Enter password: mysql
Or
Mysql –u root –ppassword@<alldatabases.sql
4. (E) Restore a singledatabase:
In this example, to restore the newdb database, execute mysql with < as shown below.
When you are restoring the dumpfilename.sql on a remote database, make sure to create
the newdb database before you can perform the restore
# mysql -u root -p[root_password] [database_name] <dumpfilename.sql
5.
6.
7. (F) Backup a specific table:
In this example, we backup only the ta2 table from sampledb database.
#mysqldump –u root –ptemppasswordsample ta2 > /tmp/nwedb_ta2.sql
(OR)
#mysqldump –c –u username –ppassworddatabasenametablename>
/tmp/databasename.tablename.sql
9. (G) Import data in MySQL
Create an employee table and employee.txt data file.
For the examples, let us create a very simple employee table with three columns–
employee number, employee name andjob.
(1)Mysql -u root -ptmppassword
Mysql> use test
Database changed
mysql> create table employee (empnoint,enamevarchar(15),job varchar(10));
(2)Create a test datafile employee.txt with fields delimited by tab as shown below.
Cat employee.txt
100 John Doe DBA
200 John Smith Sysadmin
300 Raj Patel Developer
(3)Upload tab delimited datafile to MySQL table
Use mysqlimport to import the employee.txt datafile to employee table in test database,
as shown below:
mysqlimport -u root -ptmppassword --local test employee.txt
test.employee: Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
(4)Verify that the records got uploaded successfully.
mysql -u root -ptmppassword
mysql> use test;
mysql> select * from employee;
Note: In mysqlimport, the name of the datafile should match the name of the table. The
extension of the datafile can be anything. In the above example, only employee.* datafile
10. can be used to upload data to employee table. You’ll get the following error message
when the filename is not same as tablename:
mysqlimport -u root -ptmppassword --local test emp.txt
mysqlimport: Error: Table 'test.emp' doesn't exist, when using table: emp
[Note: The table name is employee. So, datafile name should be employee.*]
(5) Import multiple datafiles into multiple MySQL tables:
The following example uploads data from two different datafiles to two different tables.
I.e. It uploads employee.txt to employee table and manager.txt to manager table.
mysqlimport -u root -ptmppassword --local test employee.txt manager.txt
(6) Use LOAD DATA LOCAL INFILE to upload data to MySQL tables:
The mysqlimport client is simply a command-line interface to the LOAD DATA LOCAL
INFILE SQL statement. Most options to mysqlimport correspond directly to clauses of
“load data local infile” syntax. You can perform the same upload explained in example#1
using “load data local infile” instead of mysqlimport as explained below:
mysql -u root -ptmppassword
mysql> use test;
mysql> LOAD DATA LOCAL INFILE '/home/ramesh/employee.txt' INTO TABLE employee
FIELDS TERMINATED BY 't' LINES TERMINATED BY 'n' (empno, ename, job);
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from employee;
+-------+------------+-----------+
| empno | ename | job |
+-------+------------+-----------+
| 100 | John Doe | DBA |
| 200 | John Smith | Sysadmin |
| 300 | Raj Patel | Developer |
+-------+------------+-----------+
3 rows in set (0.00 sec)
Load a CSV file into a table.
11. Mysql> LOAD DATA INFILE ‘/tmp/filename.csv replace into table *tablename+ fields
terminated by ‘,’ lines terminated by ‘n’ (field1,field2,field3);
(7)Most frequently used mysqlimport options:
The most frequently used mysqlimport options are shown in the example below. Most of
these options are self-explanatory.
Compress: Compress all information sent between the client and the server
delete: This option is very handy when you want to empty the table before importing the
text file local: Read input files locally from the client host
lock-tables: Lock all tables for writing before processing any text files. This ensures that all
tables are synchronized on the server.
1. mysqlimport
--user=root
--password=tmppassword
--columns=empno,ename,job
--compress
--delete
--fields-optionally-enclosed-by='"'
--fields-terminated-by='t'
--fields-escaped-by=''
--lines-terminated-by='n'
--local
--lock-tables
--verbose
test employee.txt
Output of the above mysqlimport command:
Connecting to localhost
Selecting database test
Locking tables for write
Deleting the old data from table employee
12. Loading data from LOCAL file: /home/ramesh/employee.txt into employee
test.employee: Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
Disconnecting from localhost
Issue while taking a backup on mysql
http://itknowledgeexchange.techtarget.com/security-admin/resolving-mysql-error-1146-
table-doesnt-exist-when-doing-backup/
(H) Backup (Script on Linux) :
[root@mysql backup]# cat dbbackup.sh
MAIL_ADMINS="DEEPAK.KACHOLE@yahoo.com"
LOGFILE=/backup/dbbackup.log
#Clear the Log file
rm -f $LOGFILE
#Backup MySQL Database
if
/usr/bin/mysqldump --verbose --log-error=$LOGFILE -u root -p'abcd' --all-databases |
gzip> /backup/database_`date '+%m-%d-%Y'`.sql.gz
then
mail -s "$HOSTNAME MySQL Backup Sucessful" $MAIL_ADMINS < $LOGFILE
exit 0
else
mail -s "$HOSTNAME MySQL Backup Failed" $MAIL_ADMINS
exit 1
fi
Clean Up all files older than 10 days-
find /backup/ -mtime +10 -exec rm -Rf {} ;
(I) How to import and Export MySQL Database.
To export MySQL DB:
1.Type the below command to export MySQL db.
#mysqldump –u root –p DBName>dbname11092012.sql
To import MySQL DB:
13. 1. Go to file location where you saved backupdb and type following command.
2. Login into MySQL and create new database.
Create database newdb
3.exit from sql and type as following
#mysql –u root –p newdb<db_04102012.sql
[where newdb is newly created db and db_04102012-0700.sql is going to restore newly
created db.]
(J)MySQL backups on windows:
14. On Windows cmd prompt using the following command on taking backup.
On Windows cmd prompt using the following command on restoring the tables or
databases.
15. (K)Different types of Backups for MySQL databases
(1) Logical Backup
It is done by dumping the contents of database into text files containing SQL statements
which can be used to rebuild the database. They can be used even to build it on a host
with different architecture and with different engine. But they are generally slower than
the raw kind, for both backup and recovery operations. And they can be larger than the
actual data too. It can be performed using the mysqldump utility.
(2) Raw (Binary) Backup
It involves only file copy operations, so can be very fast to perform backup or recoveries.
The engine type of tables cannot be changed in this method, as it preserves the actual
data format. Depending on engine type, it can be used to take Cold, Hot or Warm
backups. Various tools are available for this operation, which will be covered in a slide
ahead.
(3) Snapshot Based Backup
16. This kind of backup uses external utilities to take file system snapshot of MySQL. For
InnoDB, a hot backup can be performed using this, and for other engines, a warm backup.
(4) Replication-Based Backup
A copy of the Primary installation can be maintained on a separate server using a
Replication setup. Using this method, an exact replica of the databases can be maintained
separately, and can be used for backup purposes. But it is comparatively an expensive
option.
(5) Incremental Backup
The binary logs, which contain a record of all changes done on the database, can be
backed up for the purpose of doing point-in-time-recovery. Using tools like mysqlbinlog,
databases can be recovered from Binary logs till a certain point in time, or certain log
position also. Binary logging should be enabled on MySQL servers
(6) Backup Tools
Though backups can be taken without the use of any tool, there are some tools designed
specifically to make the operation easier.
(7) Mysqlhotcopy
Provided with the MySQL distribution, this is useful to perform raw backup of MyISAM
type tables only. Contrary to its name, it is not exactly a “hot” backup, as database is not
fully available during the operation, but is available only for read. It is locked and cannot
be altered. So, it is also called a “warm” backup.
(8) MySQL dump
It is provided with the MySQL distribution. It helps in taking logical backups for any kind of
engine. Our current masters are designed to automatically enable this backup in cron at
the time of installation.
(9) MySQL Administrator
17. It is available as a separate GUI download from MySQL, and can be used to perform
logical backups. It can also be used with any database engine, and has some tracking
capabilities also.
(10) InnoDB Hot Backup (ibbackup)
It is a commercial product available from InnoDB. It performs raw backups of MySQL
databases using InnoDB engine only. It is in fact a real “hot backup” as databases are
completely available for read and update during the operation.
(11) Third Party Tools
Among the many commercially available third party tools available, there is one called
Zmanda Recovery Manager that can perform both logical and raw backups for all engine
types. It can do both warm and hot backups, and can drive Replication and Snapshot
based backups too. It takes consistent backups, and has extensive reporting and Tracking
capabilities
(L) Dump MySQL database into a new database on another server :
Here I am trying to achieve is to dump a MySQL database over to a new database on a
different server; I also wanted to do it with limited write to the original server.
The plan is to dump the old MySQL database, and import the output into the new
database on the second server.
Here is what I’ve got; I've done it this way so that in theory the original server will only be
doing a read, whereas the new server will be doing the write.
$mysqldump –uusername –ppassworddbname | sshuser@newhostmysql –u user –
ppassword newdbname
(http://stackoverflow.com/questions/10104725/dump-mysql-database-into-a-new-
database-on-another-server )
18. (M) How to dump local MySQL data to remote mysql server through
terminal?
I’m currently working on some local development, which often needs to update the
remote database with my own local development database.
This is what I try to do , dump the local database and ssh to remote and update the db.
mysqldump -ulocaluser -plocalpasslocaldb | ssh user@255.255.255.255 "mysql -
uremoteuser -premotepassremotedb"
It seem like completed without any error, but checking on the remote db, it seem like the
old table never drop and replace with the new table data.
Check the output ofmysqldump -ulocaluser –plocalpass localdbDoes this contain DROP
statements?
(N)Loading a SQL file into MySQL:
Import SQL file from MySQL client command line:
mysql> source file.sql
OR
mysql> . file.sql
The SQL file may have schema generation statements like CREATE TABLE ... or data
load statements like INSERT INTO. The statements in the SQL file will be executed
as if they were being specified at the MySQL client command line interface. One
may import data into the MySQL database from SQL files or "load" data from CSV
or tab delimited files using the LOAD command.
(O)Loading CSV or tab delimited files into MySQL
"LOAD DATA LOCAL INFILE" vs "LOAD DATA INFILE": The term "LOCAL" pertains to
19. whether the file is local to the MySQL client. Without the keyword "LOCAL", the data file
must reside on the same computer as the database server. The location of the client in
this case would be irrelevant. The "LOAD DATA INFILE" has many file permission pitfalls
and is thus tricky. In fact I have never been successful using this method with a user
directory.Load a tab delimited file into the database:
Command:LOAD DATA LOCAL INFILE 'file.dat' INTO TABLE employer;
Input tab delimited file: file.dat
Note: The number of tab delimited fields MUST match the number and order of fields in
the database. Load a comma delimited file (CSV) into the database:
Command:LOAD DATA LOCAL INFILE "/tmp/TableData.csv" INTO TABLE employer
FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY """" LINES
TERMINATED BY "rn" (Name, Dept, jobTitle);
Note:MS/Windows generated files will have lines terminated by "rn".
Linux/Unix generated files will have lines terminated by "n".
File locations on database server must be absolute path names, relative path or relative
to the mysqld process owner's home directory (typically /var/lib/mysql/). File locations on
the client may be fully qualified or relative to the current mysql client directory.
Fully qualified: /tmp/TableData.csv
Relative to current mysql client directory: ./TableData.csv (Verify current directory:
mysql> ! pwd)
Database process owner home directory: TableData.csv(Actual:
/var/lib/mysql/TableData.csv)
Text strings often are encapsulated by quotes so that the strings may contain a comma
without representing a new field.
20. ERROR 13 (HY000): Can't get stat of '/tmp/TableData.csv' (Err code: 13)
The fils is local and you have not specified the "LOCAL" directive.
ERROR 29 (HY000): File '/var/lib/mysql/test/TableData.csv' not found (Errcode: 2)
Error from command LOAD DATA INFILE 'TableData.csv' INTO ... where the file is assumed
to be read from the /database-process-home-directory/mysql-database-
name/TableData.csv
(Note: Database name "test" is being used.)
ERROR 1045 (28000): Access denied for user 'user1'@'%' (using password: YES)
OR
ERROR 2 (HY000): File '/tmp/TableData.csv' not found (Err code: 2)
Error from command LOAD DATA INFILE '/tmp/TableData.csv' INTO .... This is a common
pitfall, trying to load a file located on the database server remotely from a client. Getting
the file permissions correct is difficult. Avoid this method. Use the LOAD DATA LOCAL
INFILE instead of the LOAD DATA INFILE method (it is so much easier).
Also look at the MySQLimport command.
Dump/Backup/Transfer Database:
The mysqldump command will read the MySQL database and generate a SQL command
text file. This allows data to be migrated to other versions of mySQL (i.e. upgrade from
typical Red Hat (RH7.x to FC3) MySQL release 3.23.58 to a more advanced MySQL 4.1 or
5.0) or to other SQL databases. SQL command file generated can create tables, insert
data.
Option Description
-A--all-databases Dump all the databases.
-B--databases Dump the specified databases.
-h--host= Specify host to connect to.
-p--password= Specify password. If you do not specify a password, then you will be
queried.
21. -u--user= Specify user. Defaults to current user logged in.
-a--all Include all mySQL specific SQL "create" options.
-e--extended-insert Allows utilization of the new, much faster INSERT
Syntax. Database you are migrating to must support this notation.
-q--quick Don’t buffer query, dump directly to stdout.
-l--lock-tables Lock all tables for read.
-?--help Display command line options.
(P) MysqldumpUpgrading to 4.1:
Upgrading mySQL to 4.1 from 3.23
Use the command:
mysql_fix_privilege_tables --password=root-password
This allows you to use the new GRANT command.
Restore MySql Database:
Restore using dump generated by mysqldump above:
mysql -h host-name -u user-id -psupersecretpassword< total-db-dump-file.sql
On Source Server :
22. Mysql database-name -h host-name -u user-id -psupersecretpassword<db-
Dump-file.sql
On Destination server :
23. (Q) Simple backup options
Backup all databases uncompressed from the command line (not from within MySQL):
Backup:mysqldump -u root -pmypass --all-databases >alldatabases.sql
Restore full:mysql -u username -pmypass<alldatabases.sql (no space in between -p and
mypass)
Restore single:mysql -u username -pmypassmydb<mydb.sql (no space in between -p and
mypass)
Backup all databases compressed from the command line (not from within MySQL):
With bzip2:mysqldump --all-databases | bzip2 -c > databasebackup.sql.bz2 (use bunzip2
to uncompress)
With gzip:mysqldump --all-databases | gzip> databasebackup.sql.gz (use gunzip to
uncompress)
Backup a specific database only:
mysqldump -u username -pmypassdatabasename>backupfile.sql
Backup database structure only:
mysqldump --no-data --databases databasename>structurebackup.sql
Backup a specific database and specific tables within that database only:
mysqldump --add-drop-table -u username -pmypassdatabasename table_1 table_2
>databasebackup.sql
The syntax for the command to issue is:
mysqldump -u [username] -p[password] [databasename] [table1 table2 ....]
>backupfilename.sql
24. Backing up only the database structure of specific databases, not the actual data:
mysqldump --no-data --databases db1 db2 db3 >structurebackup.sql
@@@@@@@@@@@@@@@@@@@@@
BACKUP AND RECOVER MySQL DATABASE
It is important to back up your databases so that you can recover your data and be up and running again in case
problems occur, such as system crashes, hardware failures, or users deleting data by mistake.
Types of Backup:
logical backup
This type of backup is created by saving information that represents the
logical database structures using SQL statements like create database, create
table and insert. This type of backup is ideal when you want to upgrade from
one version of MySQL to another however it is a slower method of backing
up.
physical
backup
This type of backup is a backup of the actual database files or disk partitions,
this type of backup can be very fast to backup and restore.
full backup
A full backup is a standalone backup containing everything in the database,
this could then be restored on another server. A full backup can be either
logical or physical.
incremental
backup
This type of backup only contains the data that has changed from the last
backup. The advantage of this type of backup is that it is faster as there is
not some much data to backup, however the disadvantage is that it takes
longer to recover.
consistent
backup
This is a backup at an exact moment in time, generally you shutdown the
database (or quiescent mode) then take the backup.
hot backup
This type of backup is taken when the database is running, during the backup
both reads and writes are not blocked
25. warm backup
This type of backup is taken when the database is running, however reads
are not blocked but writes are prohibited from making any modifications to
the database.
cold backup
Similar to a consistent backup as the database is shutdown before the
backup begins
point-in-time
restore
It is a restoration of a database to a specified date and time , some
databases use a full backup and recovery logs to restore to that point-in-
time, others can only use the last full backup which means that data might
have to be re-keyed into the system.
Types of Backup tools:
Backup tools for MySQL
Backup method
Storage
engine
Impact
Backup
speed
Recovery
speed
Recovery
granularity
mysqldump ALL WARM MEDUIM SLOWEST
MOST
FLEXIBLE
mysqldump INNODB HOT MEDUIM SLOWEST
MOST
FLEXIBLE
select into outfile ALL WARM SLOW SLOW
MOST
FLEXIBLE
mk-parallel-backup ALL WARM MEDUIM MEDUIM FLEXIBLE
ibbackup INNODB HOT FAST FAST FLEXIBLE
ibbackup ALL WARM FAST FAST FLEXIBLE
backup command
in mysqld
ALL HOT FAST FAST FLEXIBLE
filesystem (copy files) ALL COLD FASTEST FASTEST NOT FLEXIBLE
26. snapshot (using LVM,
ZFS, VMWare)
ALL
ALMOST
HOT
FAST FAST
LEAST
FLEXIBLE
mysqlhotcopy MyISAM
MOSTLY
COLD
FAST FAST FLEXIBLE
MySQLdump:
mysqldump is an effective tool to backup MySQL database. It creates a *.sql file with DROP table, CREATE
table and INSERT into sql-statements of the source database. To restore the database, execute the *.sql file on
destination database.
mysqldump
# backup all databases
mysqldump --user=root --password --all-databases > backup_<date>_all.sql
# backup a specific database
mysqldump --user=root --password <database_name>>
backup_<date>_<database_name>.sql
# backup multiple databases
mysqldump --user=root --password <database_name>,<database_name>>
backup_<date>.sql
# backup a table from a database
mysqldump --user=root --password <database_name><table_name>>
backup_<date>_<database_name>_<table_name>.sql
# backup some specific data
mysqldump --user=root --password <database_name><table_name> --where
"last_name='VALLE' order by first_name > backup_<date>.sql
# dumping from one database to another
mysqldump --databases <database_name> | mysql -h
<destination_host><database_name>
restore a
mysqldump
# all databases
mysql --user=root --password < backup.sql
# specific database
mysql --user=<user> --password <database_name><
backup_<dataabse_name>.sql
27. SELECT INTO OUTFILE
The SELECT INTO OUTFILE SQL statement is actually a variant of the SELECT query. It is used when you
want to direct query output to a text file. This file can then be opened by a spreadsheet application, or imported into
another database like Microsoft Access, Oracle, or any other software that supports delimitation.
Ex
SELECT id, data INTO @x, @y FROM test.t1 LIMIT 1;
LOAD DATA INFILE
The LOAD DATA INFILE statement reads rows from a text file into a table at a very high speed. The file name
must be given as a literal string.
Ex
LOAD DATA INFILE 'data.txt' INTO TABLE db2.my_table;
mysqlhotcopy
mysqlhotcopy is a Perl script that was originally written and contributed by Tim Bunce. It uses FLUSH
TABLES, LOCK TABLES, and cp or scp to make a database backup. It is a fast way to make a backup of the
database or single tables, but it can be run only on the same machine where the database directories are
located. mysqlhotcopy works only for backing up MyISAM and ARCHIVE tables.
Ex
#backup a database
mysqlhotcopy <database_name> /backups
# backup multiple databases
mysqlhotcopy <database_name> accounts /backups
#backup a database to to another server
mysqlhotcopy --method=scp <database_name> username@backup.server:/backup
# use pattern match to backup databases and tables
mysqlhotcopy <database_name>./^employees/ /backup
10 Ways to Automatically & Manually Backup MySQL Database