applications – Comparing Time Machine backups with Beyond Compare?

applications – Comparing Time Machine backups with Beyond Compare? – Ask Different

Need help merging multiple full SQL Server Backups into single database

SQL server doesn’t have any built-in capacity to merge multiple backups into a single database, you’ll have to restore each backup with a different database name (like WellProductionInjection_2019 and WellProductionInjection_2020), then do the work yourself to combine them however you need.

Once the databases are restored, you can combine them using a variety of tools; the simplest one is called the “Import/Export Wizard”. You can use this to copy entire tables from one database to another (if all tables have different names), or to append the data from table in one database to the tables in the other (if you actually want to combine the data into the same table).

debian – How do production web applications make backups?

So, I’m almost finished building my first major production web app, and am wondering how to manage the backup protocol.

Cold backups via my hosting control panel seem ideal – but daily downtime sounds awful for UX.
Hot backups seem pointless as they cannot be trusted to not be corrupt.

My server runs Debian.

Is there a way to SSH in and clone the filesystem on my local machine, before encrypting with Veracrypt, before posting it to the moon, as an off-planet backup? (Lol.)

I suppose I don’t need to backup the entire fs on a daily basis, but definitely Mongodb. What’s the easiest way to automate a mongodb backup?

Thanks so much for reading 🙂

Incremental Remote Backups Using rsnapshot

There’s an old adage about backups:

There are two kinds of people: people who’ve never lost data, and people who’ll never lose data again.

If you’ve ever experienced data loss, you will instantly become passionate about backups.  To prevent bad experiences with your data, you want backups that are comprehensive, manageable, versioned, automated, and secure.  Let’s break that down:

  • comprehensive: They should include everything by default.  It’s certainly legitimate to exclude OS directories, temp files, etc. but you don’t want a system where you have to manually add directories as you add applications and data.  Inevitably, you’ll forget something and not know it until it’s too late.
  • manageable: If you have a 1TB server and take a full backup every day and retain them for a month, that’s 30TB in a month.  You need a system that allows for regular pruning.
  • versioned: If you have a system that simply copies everything from system A to system B once a night, that’s better than nothing, but on Monday you trash a file and don’t notice it until Thursday, you can’t recover.
  • automated: Because humans are lazy.
  • secure: It’s annoying to be hacked.  It’s heartbreaking to find the hacker also destroyed your backups.

In this tutorial, we’ll show you how to setup backups using rsnapshot.  Quoting rsnapshot.org:

rsnapshot is a filesystem snapshot utility based on rsync. rsnapshot makes it easy to make periodic snapshots of local machines, and remote machines over ssh. The code makes extensive use of hard links whenever possible, to greatly reduce the disk space required.

This means if you have 500MB of files, you want to retain 30 days’ backups, and your change rate is 10% over that period, you don’t need 30 * 500 = 15,000MB but rather only 550MB.  Beautifully, you still have point-in-time recovery (depending on your backup schedule) throughout that period.

In this tutorial, we’ll setup the following:

  • server1.lowend.party has a directory called /data with lots of valuable files.
  • We want to back it up to backup.lowend.party using a scheme of hourly/daily/weekly/monthly backups.  These are stored in /backups/server1.lowend.party
  • backup.lowend.party has other hosts it backs up as well.
  • We’re using passwordless ssh keys for authentication so we can run everything out of cron.

Before we start, there’s one more key concept.

I’ve long advocated pull backups.  In other words, the backup server comes along and backs up the client.  In this scenario, backup.lowend.party initiates the backups and contacts server1.lowend.party to get the data.  This is in contrast to push backups, where server1.lowend.party contacts backup.lowend.party and pushes the backups to it.

What’s the difference?  Imagine server1 is hacked.  If we’re using push backups, it would be trivial for the hacker to use the passwordless ssh keys to nuke the backups as well.  In a pull-based model, backup.lowend.party can authenticate to server1, but not vice-versa, so the hacker is out of luck.

On Debian, it’s as easy as

apt install rsnapshot

rsnapshot’s config lives in /etc/rsnapshot.conf.  I recommend making a backup of it before you start changing things:

mv /etc/rsnapshot.conf /etc/rsnapsnap.conf.default

There are different philosophies about how to setup rsnapshot configs.  I prefer to have a separate config file for each client (system being backed up).  If you only have one system to backup, this is not necessary.  You can backup multiple systems in one config file, but you lose some flexibility.  Experiment and decide which you like.  In my case, I do this:

cp /etc/rsnapshot.conf.default /etc/rsnapshot.conf.server1

Now modify as follows.  Important Note: rsnapshot.conf requires TABs between elements.  So “cmd /usr/bin/ssh” is “cmd<TAB>/usr/bin/ssh”.

Enable remote backups:

cmd_ssh /usr/bin/ssh

Add these backup intervals:

interval hourly 6
interval daily 7
interval weekly 4
interval monthly 3

I’m using a passwordless ssh key stored in /root/.ssh/backup.  I also use a different ssh port.  So make this change:

ssh_args -p 8989 -i ~/.ssh/backup
These two commands are for reporting (see below):

rsync_long_args --stats --delete --numeric-ids --relative --delete-excluded
verbose 4

Now I tell rsnapshot where to save backups:

snapshot_root /backups/server1.lowend.party/

Finally, I add the backup definition:

backup root@server1.lowend.party:/data/ .

This will keep files in /backups/server1.lowend.party/hourly.0, etc.

I want to exclude /data/cache on my backups:

exclude_file /etc/rsnapshot.server1.exclude

And in that file I put:

- /data/cache/*

OK, we’re ready to go.  Now because I’m not using the default /etc/rsnapshot.conf name, I need to use the -c parameter for all rsnapshot commands.  Let’s start by testing the config:

root@backup:/etc# rsnapshot -c /etc/rsnapshot.conf.server1 configtest
Syntax OK

Now we can run a simulation:

root@backup:/etc# rsnapshot -c /etc/rsnapshot.conf.server1 -t hourly
echo 9633 > /var/run/rsnapshot.pid
mkdir -m 0755 -p /backups/hourly.0/
/usr/bin/rsync -a --stats --delete --numeric-ids --relative --delete-excluded 
--exclude-from=/etc/rsnapshot.server1.exclude --rsh=/usr/bin/ssh -p 8989 
-i ~/.ssh/backup root@server1.lowend.party:/data/ 
/backups/hourly.0/server1.lowend.party/
touch /backups/hourly.0/

One more thing to do.  I like to use rsnapshot’s reporting tool, so let’s enable it:

cp /usr/share/doc/rsnapshot/examples/utils/rsnapreport.pl /usr/local/bin
chmod 755 /usr/local/bin/rsnapreport.pl 

We’re good to go!

On server1, I have 547MB in /data, and 30MB in /data/cache which will be excluded:

root@server1:~# du -sm /data
547 /data
root@server1:~# du -sm /data/cache
30 /data/cache

Let’s run our first rsnapshot backup:

root@backup:/backups/server1.lowend.party# rsnapshot -c /etc/rsnapshot.conf.server1 hourly
Setting locale to POSIX "C"
echo 10012 > /var/run/rsnapshot.pid
mkdir -m 0755 -p /backups/server1.lowend.party/hourly.0/
/usr/bin/rsync -av --stats --delete --numeric-ids --relative 
--delete-excluded --exclude-from=/etc/rsnapshot.server1.exclude 
--rsh=/usr/bin/ssh -p 8989 -i ~/.ssh/backup 
root@server1.lowend.party:/data/ 
/backups/server1.lowend.party/hourly.0/.
receiving incremental file list
data/
<snipped>
data/cache/

Number of files: 10,982 (reg: 10,980, dir: 2)
Number of created files: 10,982 (reg: 10,980, dir: 2)
Number of deleted files: 0
Number of regular files transferred: 10,980
Total file size: 518,702,282 bytes
Total transferred file size: 518,702,282 bytes
Literal data: 518,702,282 bytes
Matched data: 0 bytes
File list size: 611,123
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 208,691
Total bytes received: 519,874,481

sent 208,691 bytes received 519,874,481 bytes 80,012,795.69 bytes/sec
total size is 518,702,282 speedup is 1.00
touch /backups/server1.lowend.party/hourly.0/
rm -f /var/run/rsnapshot.pid
/usr/bin/logger -p user.info -t rsnapshot(10012) /usr/bin/rsnapshot -c 
/etc/rsnapshot.conf.server1 hourly: completed successfully

Now I can also run that using the rsnapshotreport.pl script we setup.  If I do, the output will look like this (the TOTAL MB is a little different because I ran these at different times):

# rsnapshot -c /etc/rsnapshot.conf.server1 hourly | /usr/local/bin/rsnapshotreport.pl
SOURCE TOTAL FILES FILES TRANS TOTAL MB MB TRANS LIST GEN TIME FILE XFER TIME
--------------------------------------------------------------------------------------------------------------------
server1.lowend.party:/data/ 11982 1 564.81 46.10 0.001 seconds 0.000 seconds

Now if I continue running hourly backups, I see new directories being created in /backups/server1.lowend.party:

drwxr-xr-x 3 root root 4096 Jul 12 16:03 hourly.0
drwxr-xr-x 3 root root 4096 Jul 12 16:01 hourly.1
drwxr-xr-x 3 root root 4096 Jul 12 15:58 hourly.2

Interestingly, hourly.0 is 500-odd MB, will the rest are only 1MB.  Why?  Because hourly.1, hourly.2, etc. are simply hard links back to hourly.0.  This is a huge space savings.

If I nuke some files on server1’s /data and run another couple backups, you’ll see this:

root@backup:/backups/server1.lowend.party# du -sm *
526 hourly.0
39 hourly.1
1 hourly.2
1 hourly.3

rsnapshot is retaining data in hourly.1 because it’s needed to reconstruct the backups for that hour.

Setting up automated backups is as easy as putting jobs in cron.  For example:

MAILTO=you@somewhere.com
0 * * * * root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 hourly 2>&1 | /usr/local/bin/rsnapreport.pl
0 3 * * * root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 daily 2>&1 | /usr/local/bin/rsnapreport.pl 
0 3 * * 1 root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 weekly 2>&1 | /usr/local/bin/rsnapreport.pl
30 2 1 * * root /usr/bin/rsnapshot -c /etc/rsnapshot.conf.server1 monthly 2>&1 | /usr/local/bin/rsnapreport.pl

Now

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

Has the Time Machine option to “”Delete all backups of” disappeared in Big Sur?

I have a folder that gets filled with large files that I want Time Machine to remember while retaining the option of regularly forgetting these large files. For this and up until Catalina I would regularly enter Time machine, go to this folder and use the command Delete all backups of... (see for instance the screen capture of @Inder ‘s answer in this thread). As of my recent upgrade to Big Sur I no longer see this contextual menu option.

Is it gone from Macos for good? Is there an alternative that does not involve under-the-hood sudo-type terminal surgery?

Thanks!

Hostisto | SSD | 99.9% Uptime | Free Domain | Daily Backups | Forum Promotion


Swefty is a privately owned web hosting company that was founded in 2012. That means we have been going strong for over 9 years! With no interruptions! What I mean by this, unlike many other small hosting companies, we have been providing our services truly for 9 years strong. We have not gone offline and came back later or been acquired. We are still the same people, running the same company, providing the same awesome personalized support we have been doing for years.

So what’s so great about our company?
Well our mission! “Provide excellent hosting, fair pricing, and personalized support”. Swefty has now grown into a sustaining business which continues to provide one of the best hosting experiences out there. We believe we are a leading example of what a web hosting provider should be today.

We are NOT a reseller. We have our own servers. We have our own unique custom design. Be comfortable knowing that we spared no expense in making sure our clients have the best!

What Are Our Features?
Swefty offers a very large wide range of features for every single one of our lovely clients!

Industry Leading cPanel Control Panel
It’s no secret. People prefer the cPanel interface over any other control panel out there. That’s why all of our clients get access to cPanel to manage their websites effortlessly, and easily.

99.9% Uptime Guarantee
We provide a guarantee of 99.9% uptime or your money back! See our SLA Agreement for more info:
https://swefty.com/sla-agreement.php

FREE Domain
We are currently offering free domains with our gold, platinum, and diamond hosting packages. No strings attached. Unlike other providers, we do not force you to purchase on an annual plan. You can get your free domain regardless if you purchase the monthly or annual plan.

FREE SSL Certificates
We believe in helping to secure the internet, we give out free SSL’s to ALL of our clients. Get that extra Google SEO boost at no extra cost! Regardless of what your site is for, if you collect any form of user information (Not just payments!) an SSL is highly encouraged. Swefty was ahead of the curve and was offering FREE Positive SSL Certificates before almost all other hosts out there. We took security seriously from the beginning. Now with Let’s Encrypt, securing the web has never been easier.

FREE Daily Backups
Although, we always encourage our clients to take their own backups, we also take daily backups of both their files, and databases! At NO extra cost! Rest easy in knowing your website is safe.

Instant Installs
It has become pretty standard to offer instant installs for hosting clients. So naturally we do to. With 400+ scripts to instantly install on your server ever so easily!

Current Promotions
Due to the ever changing coupons, you can always see what current promotions we have going on by visiting this page:
https://www.hostisto.com/promotions.php

About our servers:
We use the powerful industry standard cPanel to power your website.
We use the LiteSpeed Web server which is up to 75% faster than standard Apache
We use Cloudlinux to provide the best security and ensure other clients don’t take your site down.
Our servers are using SSD as a foundation for their performance. SSD massively outperforms traditional spinning hard-disk drives and provide up to 20 times faster speeds.

What are you waiting for? Come visit us and check out our plans!
swefty.com
You have nothing to loose with our 30-Day (90 Days if purchased yearly) Money Back Guarantee no questions asked!

FAQ:
Swefty believes in open discussion, and transparency! We try to provide as much detail and answer as many questions as we can. If something isn’t in the FAQ Below, feel free to ask!

Are you a reseller?
In all sense of the question you are most likely referring to, we are not a reseller. All of our servers are leased directly from a top tier NOC and we manage our own servers. We are however, not an ICANN accredited domain registrar. We mainly specialize in website hosting, and our domains are sold from NetEarthOne who are an accredited domain registrar who provides the tools necessary for companies like us to provide addon domains to our clients. Currently, we sell .com domains for $9.99 USD.

Are there any hidden costs, or setup fees?
Absolutely not! Feel free too, in fact we encourage everyone to read our TOS and AUP! What you see is what you pay.

Have any more questions? Feel free to check out some of these links:
– Why don’t I just go with the cheapest hosting?
– Can I order even though I do not live in the US?
– Do you offer a money back guarantee?
– Do you offer unlimited bandwidth/space/transfer?

checkout.png

7 – It’s not allowed to store backups inside the Drupal root directory

I must be missing something because I can’t determine what I’m doing wrong. I’ve got a D7 site, and we’re running Drush 8.1.10

I’m trying to run through some updates but I keep running into this error:

It's not allowed to store backups inside the Drupal root directory.

I don’t have the backups directory inside the drupal root directory – or at least so far as I know, it’s not. I’ve set the directory to be one level above the webroot and declared it in my settings.php as follows:

/** declaring the private files directory 
*/
$settings['file_private_path'] = '/var/www/vhosts/drush-backups';

In my file system I’ve also got my private files path set to the same to the same dir just trying to figure out where this issue is coming from.

Can someone help me figure out what I’m missing?

50% OFF >> 6/year – linux SSD – Unlimited Bandwidth – Free Domains & SSL – Daily Backups | Proxies-free

Hostpoco.com specializes in making reliable hosting affordable. Offering affordable web hosting at a great price is our passion and always will be. We provide blazing-fast SSD web hosting, with our servers maintaining more than 99.9% uptime. We care about your website and data, so we provide free daily backups and our servers are protected with a Web Application Firewall (WAF) for maximum security.

Whether it be shared web hosting or reseller web hosting, you can rest assured you are receiving the best possible price and service with Hostpoco.com!

Payment methods: PayPal, Razorpay, Stripe.

Promotion Code: “HP50” for 50% discount on any plan

Linux Shared Hosting Features:
– Latest cPanel Control Panel
– 24/7 Technical Support
– Money-Back Guarantee
– 99.9% Uptime Guarantee
– Softaculous Auto Installer
– Server Level Protection
– Unlimited Bandwidth
– Free Let’s Encrypt SSL
– Unlimited Sub-Domains
– Unlimited MySQL Databases
– Unlimited Email Accounts
– Unlimited FTP Accounts
– FREE Online Sitebuilder
– FREE Auto SSL
– CloudLinux
– Cloudflare
– Unlimited Parked Domains
– FREE DDOS Protection
– FREE Backup Restore
– FREE PhpMyAdmin
– FREE Virus Scanner

Detailed Plan Info: https://www.hostpoco.com/one-dollar-cheap-ssd-hosting.php

For more web hosting plans, please visit our website: https://www.hostpoco.com/
Should you have any questions, email us: sales@hostpoco.com

 

50% OFF >> 6/year – Linux SSD – Unlimited Bandwidth – Free Domains & SSL – Daily Backups | Forum Promotion

DreamProxies - Cheapest USA Elite Private Proxies 100 Private Proxies 200 Private Proxies 400 Private Proxies 1000 Private Proxies 2000 Private Proxies ExtraProxies.com - Buy Cheap Private Proxies Buy 50 Private Proxies Buy 100 Private Proxies Buy 200 Private Proxies Buy 500 Private Proxies Buy 1000 Private Proxies Buy 2000 Private Proxies ProxiesLive Proxies-free.com New Proxy Lists Every Day Proxies123