TechBask

Learn | Explore | Share | Challenge Yourself | Enjoy | Make This World Beautiful Place To Live For Everyone|

Archive for the category “Mongodb”

Automate Backup of MongoDB.

I work with MongoDB  in day today basis and one of the challenge is taking backup of different DB’s deployed. Taking backup is important for recovery in case of  mishap. Mostly there are enormous write operations and database size keeps growing so the chances of failure. Initially we started to do this manually which was okay till we had limited traffic but as  registration and engagement increased so we started looking for automating this process. Cron job scheduler in Linux and Amazon S3 came to the rescue. 

Pre-requisites:

  • MongoDB 2.33 or higher deployed on Linux Environment.
  • Amazon S3 account to store all backup data.

 

Tools Required(or their alternatives):

  • S3cmd – S3cmd is a free command line tool and client for uploading, retrieving and managing data in Amazon S3.
  • Linux Cron utility
  • Mongodump utility provided by MongoDB
  • Tar archiving utility– GNU version.
  • Command Line Interpretor(CLI).

 

Installation and Configuration of S3cmd:

  1. Run to install in CLI or Shell
    sudo su
    sudo yum –enablerepo epel install s3cmd
  2. configure
    s3cmd –configure
  3. Display all buckets in your Amazon S3 account
    s3cmd ls

 

Writing a Shell Script to Backup MongoDB:

  1. Force file syncronization and lock writes on MongoDB using fsync
    mongo admin –eval “printjson(db.fsyncLock())”
  2. Declare some variables
    MONGODUMP_PATH=”path-to-mongodb-utility”
    MONGO_HOST=”server-ip”
    MONGO_PORT=”port-here”
    USER_NAME=”username-used-for-mongodb-authentication”
    PASSWORD=”password-used-for-mongodb-authentication”
    MONGO_DATABASE=”DB-Name”
    TIMESTAMP=`date +%F-%H%M` // use as it is. S3_BUCKET_NAME=”name-of-s3-bucket”
    S3_BUCKET_PATH=”mongodb-backups”
  3. Take mongoDB dump using mongodump utility
    $MONGODUMP_PATH -h $MONGO_HOST:$MONGO_PORT -d $MONGO_DATABASE -u $USER_NAME -p $PASSWORD
  4. Unlock database writes
    mongo admin –eval “printjson(db.fsyncUnlock())”
  5. Add timestamp to backup folder name
    mv dump mongodb-$HOSTNAME-$TIMESTAMP
  6. Compress the folder
    tar cf mongodb-$HOSTNAME-$TIMESTAMP.tar mongodb-$HOSTNAME-$TIMESTAMP
  7. Upload to S3
    /usr/local/bin/s3cmd put mongodb-$HOSTNAME-$TIMESTAMP.tar s3://$S3_BUCKET_NAME/$S3_BUCKET_PATH/mongodb-$HOSTNAME-$TIMESTAMP.tar
  8. Delete local dump folder
    /bin/rm -r mongodb-$HOSTNAME-* // to delete all use *

Find full bash script here

 

Run this script at a particular time using crontab

Post Navigation