Streaming a file over SSH with scp
The problem scenario: you have a large SQL file weighing about 60GB on a remote server that you need to import into your local MySQL database. Normally, you would copy the file over using scp
and then import that file. Unfortunately, your local disk only has 65GB left. It's enough to save the file, but not enough to import. It'd take double to do that. Is it possible to stream the file into MySQL?
It is! Thanks to the UNIX design that everything is a file. Copy that data to stdout and then pipe it into mysql.
scp user@remote.server.com:big-dump-data.sql /dev/stdout | mysql my_db -uroot