Download CPanel Backups

A recent improvement in CPanel security means that you can no longer download your site’s backups using wget.

CPanel now disables “Basic Authentication” and instead uses cookies and a “cpsession” value embedded in the admin URLs. Whilst this is said to reduce the incidence of certain types of attacks it also creates major headaches for any admins who need to automatically retrieve their site or database backups.

There are many CPanel backup scripts floating around but all the examples I found still used “Basic Authentication” so were rendered useless by these new authentication requirements.

CPanel offers an undocumented feature in their XML-API to create a Full Site Backup however in my instance there were problems with this.

Unlike regular backups the Full Site Backup cannot be restored from within the CPanel admin (though it can be used by webhosts to transfer an entire site).

Also the Full Site Backup does not give you a download link, instead it begins generating an archive in your site’s space (thus potentially sending you over quote), and then once finished it fires you off an email. Alternatively it can FTP/SCP it to a remote server, but this requires you to have a server available as well as making it more difficult for scheduling multiple backups if your backup server has limited bandwidth.

A Solution

Below is a script I whipped up to download CPanel backups using curl. It logs into the admin site just like a web browser then uses the authentication cookies and the “cpsession” to download the site backups. It only requires the curl application and runs in bash so doesn’t require PHP or any other languages to run.

If you find it useful then please leave me a comment.

#!/bin/bash
 
##
## Download CPanel backups using Cookie authentication
##
## Backups: Home dir, Aliases, Filters and specified Databases
##
## This script is provided as is with no implied warranty.
## Use at your own risk.
## Copyright (c) 2012, McFang.com
 
##########################################################
# Create a configuration file with the following
# block of variables and call this script giving
# the configuration file as the first argument.
##########################################################
 
## CPanel domain name
#DOMAIN=example.com
 
## actual shared host url, should match the SSL certificate
#SITE=webhost.example.com
 
## CPanel user & Password
#USER=user
#PASS=pass
 
## Databases to backup
#DBarray=( db1 db2 db3 )
 
## Dir for saving backups. No trailing slash.
#DIR=/backups/$USER/`eval date +%Y%m%d`
 
##########################################################
 
CONFIG=$1
if [ "$CONFIG" == "" ]; then
    echo "Usage: `basename $0` <config-file>"
    exit 1;
fi
 
if [ -f $CONFIG ]; then
    . $CONFIG
else
    echo "Config file not found"
    exit 2;
fi
 
mkdir -p $DIR
if [ "$?" -gt "0" ]; then
    echo "FAILED creating backup folder: $DIR";
    exit 3;
fi
 
PWD=`pwd`
cd $DIR
 
# Submit POST login and save cookies to file
REDIRECT=`curl -f -o /dev/null  -w "%{redirect_url}" -c cookies.txt -d "user=${USER}&pass=${PASS}" -ssl https://${SITE}:2083/login/`
#echo "Redirect to ${REDIRECT}"
# eg. https://example.com:2083/cpsess11603609/frontend/x3/index.html?post_login=42460004784831
if [ "$?" -gt "0" ]; then
    echo "FAILED logging into $DOMAIN";
    exit 3;
else
    echo "Logged into $DOMAIN, downloading backups to $DIR";
fi
 
 
CPSESS=`echo ${REDIRECT} | sed 's/.*\(cpsess[0-9]*\).*/\1/'`
#echo "CPsession ${CPSESS}"
 
# Download aliases
curl -f -O -b cookies.txt -ssl https://${SITE}:2083/${CPSESS}/getaliasbackup/aliases-${DOMAIN}.gz
if [ "$?" -gt "0" ]; then
    echo "FAILED downloading aliases";
fi
 
# Download filters
curl -f -O -b cookies.txt -ssl https://${SITE}:2083/${CPSESS}/getfilterbackup/filter_info.${USER}.yaml.gz
if [ "$?" -gt "0" ]; then
    echo "FAILED downloading filters";
fi
 
 
# Download Databases
for element in $(seq 0 $((${#DBarray[@]} - 1)))
do
    #echo -n "${DBarray[$element]}"
    curl -f -O -b cookies.txt -ssl https://${SITE}:2083/${CPSESS}/getsqlbackup/${USER}_${DBarray[$element]}.sql.gz
    if [ "$?" -gt "0" ]; then
        echo "FAILED downloading database: ${DBarray[$element]}";
    fi
done
 
# Download backup
curl -f -O -b cookies.txt -ssl https://${SITE}:2083/${CPSESS}/getbakcup/backup-${DOMAIN}.tar.gz
if [ "$?" -gt "0" ]; then
    echo "FAILED downloading backup";
fi
 
 
# Cleanup
rm -f cookies.txt
 
cd $PWD

Tags: ,

Sites that are exposing user information

Like many other tech-savvy users I provide a unique email address to each business as this makes it blatantly obvious that a leak has occurred: spam starts arriving at the same account.

Please note that this list is only based on my email account receiving spam directed at an alias given to that business, and whilst I don’t have any proof to claim that their entire database of members has been compromised it is not far fetched to believe that if one account has been exposed then others have also been stolen.

The list:

 

Tags: ,

MKV videos slow as hell

I noticed that the initial opening of mkv files was becoming increasingly slow and that it appeared to depend on the number of mkv files in the same folder.

The culprit? A setting in Haali Media Splitter.

To prevent the scanning of other mkv files when playback begins use this setting in Haali:

Options -> Input -> Try to open linked files -> NO

Tags: , , ,

Windows Registry permissions reset

Windows Registry permissions can be mangled by third party applications such as Anti-Virus or Firewall applications attempting to give you more “security”. The downside of this is it can cause havoc when you try to install other software or drivers.

Thankfully it is easy, albeit time-consuming, to reset the registry permissions with just a few commands:

cd /d "%ProgramFiles%\Windows Resource Kits\Tools"
subinacl /subkeyreg HKEY_LOCAL_MACHINE /grant=administrators=f /grant=system=f
subinacl /subkeyreg HKEY_CURRENT_USER /grant=administrators=f /grant=system=f
subinacl /subkeyreg HKEY_CLASSES_ROOT /grant=administrators=f /grant=system=f
subinacl /subdirectories %SystemDrive%\ /grant=administrators=f /grant=system=f

Tags: , ,

Enable Japanese tags in Rockbox

Note that these instructions enable the display of Japanese in your Database and Now Playing screens when you have files with Japanese ID3 tags. It will not change the language of your device and your menus will remain in their default language.

Set the Default Codepage to Japanese

Settings > General Settings > Display > Default Codepage > Japanese

Clear the Database to reload all tags from files

Database > Initialize now

Reboot after Initialize finishes in order to incorporate the database changes.

And finally, remember to set a font with Japanese character support:

Settings > Theme Settings > Font > *Sazanami-Mincho

*or any Rockbox fonts that support Japanese.

Tags: , ,

Migrate Gmail to Gmail

If you created a new Gmail account and wished you could migrate all your mail from an old Gmail account across, or if you yearned to make a seamless switch from Gmail to Google Apps, then this will provide a solution for you.

The Solution: imapsync

This method uses imapsync and preserves labels and mail, including Sent items. imapsync is available in many linux distributions.

You must have enabled IMAP access on both your source and destination Gmail accounts. If you have enabled the IMAP-Controls feature in Gmail labs you must ensure all labels have been tagged “Show in IMAP”

This script will sync all labels and mail from user1 to user2. The process is done in two steps, first all labels are created, then the mail is copied. Mail from Trash and Spam is excluded.

It requires you have your passwords in $HOME/.imap/passfile1 and passfile2 for authentication.

#!/bin/bash
 
user1='username1@gmail.com'
user2='username2@gmail.com'
 
# escape the @ symbols
user1e=${user1//@/\\@}
user2e=${user2//@/\\@}
 
# Create folders/labels first
imapsync --host1 imap.gmail.com \
--port1 993 --user1 "$user1" \
--passfile1 $HOME/.imap/passfile1 --ssl1 \
--host2 imap.gmail.com \
--port2 993 --user2 "$user2" \
--passfile2 $HOME/.imap/passfile2 --ssl2 \
--syncinternaldates --split1 100 --split2 100 \
--authmech1 LOGIN --authmech2 LOGIN \
--exclude "\[Gmail\].*" \
--justfolders --nofoldersizes
 
# Sync mail, only copy new items
imapsync --host1 imap.gmail.com \
--port1 993 --user1 "$user1" \
--passfile1 $HOME/.imap/passfile1 --ssl1 \
--host2 imap.gmail.com \
--port2 993 --user2 "$user2" \
--passfile2 $HOME/.imap/passfile2 --ssl2 \
--syncinternaldates --split1 100 --split2 100 \
--authmech1 LOGIN --authmech2 LOGIN \
--useheader "Message-ID" --skipsize --allowsizemismatch \
--exclude "\[Gmail\]\/Spam|\[Gmail\]\/Trash" \
--regexmess "s/Delivered-To: $user1e/Delivered-To: $user2e/g"

References

Gmail to Google Apps Email Migration

Migrate email from Gmail to Google Apps

Other Options

Gmail-Backup (not tested)

Python migration script (not tested)

Tags: , , , ,

SSH Proxy

Did you know that OpenSSH includes function to act as a SOCKS4 and SOCKS5 host? (I didn’t)

It is as simple as creating a “Dynamic forwarded port” in PuTTY or using the -D switch with the SSH client.

Tunnelling just became a whole lot easier.

References

HOWTO: Secure Firefox and IM with PuTTY

Tags: , ,

Fun with Google Suggest

ThisĀ short article compares some results of how language and grammar effect the results from Google Suggest and may suggest the “intelligence” of the questioners.

You is trollin boxxy

Tags: ,

Comparison of Twitter clients

Echofon

(formally known as Twitterfox)

Pros Cons
  • Easy to favourite, reply, retweet
  • Shortcut key for open and paste in current page’s url
  • Automatically shrinks urls
    Uses browser’s dictionary
  • Easy to open selected tweet’s owner’s page in browser
  • Compact user interface
  • Preview destination of short urls before clicking
  • Only loads subset of recent tweets, not all since last runtime
  • Seems to randomly switch between using bit.ly and tinyurl.com
  • Ugly theme since renaming to “Echofon”
  • No marking of posts as read

Tweetdeck

Pros Cons
  • Preview destination of shortened links before opening in browser
  • Can mark posts ‘read’
  • Fixed height for all tweets wastes precious screen space.
  • NO colourcoding of text (eg. links, usernames, etc)
  • Marking post ‘read’ only marks it in current tab
  • No smooth scrolling, scrolls item by item

Seesmic

Pros Cons
  • Different Tweet types are colourcoded (eg. messages, replies, ReTweets)
  • No checklist of current followed accounts, makes it tedious to create a list of friends – have to locate an existing tweet and then click ‘add to userlist’. (compared to how Tweetdeck does it)
  • Adding a user to a list will not bring over their existing tweets (unless you restart the program)
  • No marking of posts as read
  • No smooth scrolling, scrolls item by item

Peoplebrowsr

Pros Cons
  • Has web and desktop versions (thus far appear identical, and settings carry-over between them)
  • Can switch between Lite, Advanced, and Business modes.
  • Can show people’s @replies that are not directed at you
  • Marking a post as ‘read’ markes it in ALL tabs (and changes its colour)
  • Smooth scrolling
  • Preview the destination of short urls
  • Desktop App (Adobe Air) is buggy and slow

Tags: , , , , , ,

Deleting in Gmail IMAP

Getting “Delete” to work as expected in Gmail IMAP was surprisingly complicated.

The default action when a client deletes a message is to archive it (ie: move it to “All Mail”). I didn’t like this because then if I accessed the account from POP the deleted mail would still be downloaded.

Recommended IMAP settings tells you all the things NOT to do, but doesn’t give any useful advice on how to setup clients.

Eventually after trying many different settings I realised the simplest method was to tell the client to MOVE the message to the Gmail Trash folder when it is deleted. To do this in Thunderbird2 required manually editing the prefs.js file to specify a custom IMAP trash folder:

user_pref("mail.server.server#.trash_folder_name", "[Gmail]/Trash");

Note that the case of the trash folder name IS important.

Tags: , , ,