Need Linux scripting help

[Vicken]

Hi all, I am using ubuntu linux and I am trying to backup an ftp site with a script. I have ncftpget, but the problem is I keep getting disconnected. I also want it to tar it automatically and delete the downloaded files. I need a script that reconnects and resumes where it left off. How is this done? Also, any recommendations on backing up mysql databases automatically?

 

[Crusty]

If you have access to cron, you can write a script that uses mysqldump to create a dump of the DB and then ftp it somewhere.

 

[rmrf]

you could use wget to retrieve the files, I'll let someone else comment on the tar portion.

wget -cr ftp://username:_password@server.domain.com/
(minus underscore before password)

c = continue (in case connection is broken)
r = recursively. to grep through directories and grab all the files.

if you are running a machine that will have people logged in, you should look at putting your info in a file, and invoking with wget -i. this is because if anyone ran a ps on your system, the username/password would show up in the process list.

that should be enough to get you started anyway.

edit: removed from command.

 

[drag]

Try looking at rsync.

It's a very advanced tool usefull for synchronizing files across insecure and slow internet links. It basicly does a 'running checksum' on files to compare any changes between them and then only transfers the changes between the machines to match the files. That way you don't waste bandwidth and it's done as quickly as possible. Works over ssh (that's configurable) and is secure, you can setup shared keys to automate it with no passwords. Much better then ssh.

You can use it to do complete backups or take snapshots. It's very scriptable and is only realy limited by your imagination.

For some examples:
http://aplawrence.com/Basics/rsync.html
http://www.mikerubel.org/computers/rsync_snapshots/

There is a interesting article about rdiff-backup, something that is designed to make backups easier but works in a similar fasion to rsync.
http://arstechnica.com/articles/columns/linux/linux-20060202.ars
On the 2nd page.

 

[rmrf]

Originally posted by: drag
Try looking at rsync.

It's a very advanced tool usefull for synchronizing files across insecure and slow internet links. It basicly does a 'running checksum' on files to compare any changes between them and then only transfers the changes between the machines to match the files. That way you don't waste bandwidth and it's done as quickly as possible. Works over ssh (that's configurable) and is secure, you can setup shared keys to automate it with no passwords. Much better then ssh.

You can use it to do complete backups or take snapshots. It's very scriptable and is only realy limited by your imagination.

For some examples:
http://aplawrence.com/Basics/rsync.html
http://www.mikerubel.org/computers/rsync_snapshots/

There is a interesting article about rdiff-backup, something that is designed to make backups easier but works in a similar fasion to rsync.
http://arstechnica.com/articles/columns/linux/linux-20060202.ars
On the 2nd page.

at one point, I was doing a ton of debian installs, so I wanted to rsync the repository and could never get it to work. it was my first thought, but it seemed a little more difficult to set up than using wget. of course, it is way more secure, so that could have something to do with it.

 

[drag]

It shouldn't be very difficult to use. It just tunnels over ssh.

As for Debian stuff... Try using apt-proxy next time. It's like a caching proxy just for Debian packages. You do one install, it downloads the packages and installs them normally. Next install you download from the local apt-proxy mirror rather then through the internet.