{"id":7832,"date":"2022-01-07T00:48:45","date_gmt":"2022-01-06T23:48:45","guid":{"rendered":"http:\/\/www.chipwreck.de\/blog\/?p=7832"},"modified":"2022-01-07T00:55:34","modified_gmt":"2022-01-06T23:55:34","slug":"backup-remote-servers-via-ftp-rsync","status":"publish","type":"post","link":"https:\/\/www.chipwreck.de\/blog\/2022\/01\/07\/backup-remote-servers-via-ftp-rsync\/","title":{"rendered":"Backup remote servers via ftp\/rsync"},"content":{"rendered":"\n<p>I maintain several websites and needed a way to regularly backup the site content to my local machine (where time machine provides the next backup layer). Some sites can only be accessed via (S)FTP, for some I have SSH access. So I created a small bash script to back up all servers . This script works with rsync, and <a href=\"http:\/\/lftp.tech\/\">lftp<\/a>.<\/p>\n\n\n\n<p>The script runs on OSX, but it should be easy to adapt it to any Unix platform by replacing the &#8220;osascript&#8221;-calls with something else.<\/p>\n\n\n\n<p><em><strong>Please note:<\/strong> This is not a backup solution which works out of the box, but an example script you can modify to fit your own needs. Bash knowledge is required.<\/em><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">How it works:<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li>Each backup job is defined by a bash function whose name starts with &#8220;<code>backup\u2026<\/code>&#8220;<\/li><li>To run a single backup job: &#8220;<kbd>cw-backup.sh backupSomesite<\/kbd>&#8220;<\/li><li>To run all backup jobs: &#8220;<kbd>cw-backup.sh all<\/kbd>&#8220;<br><\/li><li>The backups are saved in subfolders of the folder given by <kbd>$backupDir<\/kbd><\/li><\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Backup via (s)ftp:<\/h4>\n\n\n\n<p>The mirror function of lftp will be used. Here is an example method:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>backupMyblog()\n{\n\t_lftpBkup \\\n\tsrcHost='www.myblog.example' \\\n\tsrcDirs='\/downloads\/,\/data\/' \\\n\topts='--exclude=cache\/ --exclude-glob=*.zip' \\\n\ttargetDir='myblog'\n}\n\n<\/code><\/pre>\n\n\n\n<p>Set Hostname (srcHost), the folder name for this backup (targetDir) and the directories which should be backed up (srcDirs). Multiple directories can be separated with a comma.<\/p>\n\n\n\n<p><meta charset=\"utf-8\">Additionally you can add specific options (opts).<\/p>\n\n\n\n<p>The username and password for FTP-access are taken from the ~\/.netrc file (see in the man page or here <a href=\"https:\/\/stackoverflow.com\/a\/42787829\">https:\/\/stackoverflow.com\/a\/42787829<\/a>)<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Backup via rsync:<\/h4>\n\n\n\n<p>Here is an example:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>backupMyproject()\n{\n\t_rsyncBkup \\\n\tsrcHost='user@myproject.example' \\\n\tsrcDirs='\/var\/www\/myproject\/data' \\\n\tport='22' \\\n\topts='--exclude=**\/.cache --exclude=*.gz' \\\n\ttargetDir='myproject'\n}<\/code><\/pre>\n\n\n\n<p><meta charset=\"utf-8\">Set Hostname (srcHost), the rsync-Port (port), the folder name for this backup (targetDir) and the directories which should be backed up (srcDirs). <meta charset=\"utf-8\">Multiple directories can be separated with a comma.<\/p>\n\n\n\n<p>Additionally you can add specific options (opts).<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Configuration<\/h4>\n\n\n\n<p>In the &#8220;settings&#8221; part of the script you should set the location of the backup folder. You can also set the locations for lftp and rsync and default options for those commands.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Periodic execution<\/h4>\n\n\n\n<p>I use launchd on OSX to execute the backup periodically.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">The script<\/h4>\n\n\n\n<p>Find the source code here:<br><a href=\"https:\/\/bitbucket.org\/mariofischer\/cw-backup\/\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/bitbucket.org\/mariofischer\/cw-backup\/<\/a><\/p>\n\n\n\n<h5 class=\"wp-block-heading\">Settings<\/h5>\n\n\n\n<pre class=\"wp-block-code\" style=\"font-size:11px\"><code># backup root dir, must end with \/\nbackupDir=\"$HOME\/Projekte\/_example-backup\/\"\n\n# ltfp binary\nlftpBin=\"\/usr\/local\/bin\/lftp\"\n\n# lftp mirror options\nlftpMirrorOptions=\" --verbose=1 --skip-noaccess --parallel=5 --delete --only-newer \"\n\n# lftp command start\nlftpCmdStart=\"set ssl:verify-certificate false &amp;&amp; set cache:enable true &amp;&amp; set dns:cache-enable true &amp;&amp; set ftp:use-feat off \"\n\n# rsync binary\nrsyncBin=\"\/usr\/bin\/rsync\"\n\n# rsync options\nrsyncOptions=\" -a -v --partial \"\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>I maintain several websites and needed a way to regularly backup the site content to my local machine (where time machine provides the next backup layer). Some sites can only be accessed via (S)FTP, for some I have SSH access. So I created a small bash script to back up all servers . This script &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.chipwreck.de\/blog\/2022\/01\/07\/backup-remote-servers-via-ftp-rsync\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Backup remote servers via ftp\/rsync&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[8,4],"tags":[],"class_list":["post-7832","post","type-post","status-publish","format-standard","hentry","category-webdevelopment","category-osx"],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/paPEN-22k","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/posts\/7832","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/comments?post=7832"}],"version-history":[{"count":10,"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/posts\/7832\/revisions"}],"predecessor-version":[{"id":8288,"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/posts\/7832\/revisions\/8288"}],"wp:attachment":[{"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/media?parent=7832"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/categories?post=7832"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.chipwreck.de\/blog\/wp-json\/wp\/v2\/tags?post=7832"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}