Category Archives: admin

Drop MySQL Tables by Partial Name

This is an excerpt from Adminbuntu, a site for Ubuntu Server administrators:

http://www.adminbuntu.com/drop_mysql_tables_by_partial_name

IMPORTANT! First back up your database!

This procedure will allow you to drop many tables at once where each table name to be dropped starts with the same string.

There are two steps in the procedure:

  • Create a MySQL statement file containing all the DROP commands called drop_commands.sql
  • Run the drop_commands.sql file

1. Create drop_commands.sql File

This creates a MySQL statement file that will drop all tables that begin with a specified string.

  • Replace STRING1 with the string to match
  • Replace USERNAMEHERE with the MySQL user to use
  • Replace PASSWORDHERE with the correct password
  • Replace DATABASENAMEHERE with the name of the database
mysql --user=USERNAMEHERE --password=PASSWORDHERE -e "SELECT table_name FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE 'STRING1%' AND TABLE_SCHEMA='DATABASENAMEHERE' " | grep -v table_name | xargs -L 1 echo "DROP TABLE " | sed "s/\$/;/" | sed -e '1 i SET FOREIGN_KEY_CHECKS = 0;'| sed -e '$s@$@\nSSET FOREIGN_KEY_CHECKS = 1;@' >drop_commands.sql

2. Execute the drop_commands.sql Command File

Examine drop_commands.sql to make sure it is doing what you want.

less drop_commands.sql

Run the drop_commands.sql text file through the mysql interpreter to drop all the selected tables.

mysql --user=USERNAMEHERE --password=PASSWORDHERE DATABASENAMEHERE < drop_commands.sql

Drop All Tables in a MySQL Database

This is an excerpt from Adminbuntu, a site for Ubuntu Server administrators:

http://www.adminbuntu.com/drop_all_tables_in_a_database

IMPORTANT: This is a dangerous command. Back up the database first!

Drop all tables in a database, without dropping the database itself.

This creates a MySQL statement file that will drop all tables that begin with a certain string.

  • Replace USERNAMEHERE with the MySQL user to use
  • Replace PASSWORDHERE with the correct password
  • Replace DATABASENAMEHERE with the name of the database
mysql --user=USERNAMEHERE --password=USERNAMEHERE -BNe "SHOW TABLES" DATABASENAMEHERE | tr '\n' ',' | sed -e 's/,$//' | awk '{print "SET FOREIGN_KEY_CHECKS = 0;DROP TABLE IF EXISTS " $1 ";SET FOREIGN_KEY_CHECKS = 1;"}' | mysql --user=USERNAMEHERE --password=USERNAMEHERE DATABASENAMEHERE

Upgrading Ubuntu Server to lastest LTS version

From Adminbuntu, Everything for the Ubuntu Server Administrator

This is an excerpt from Adminbuntu, a site for Ubuntu Server administrators:

http://www.adminbuntu.com/upgrading_ubuntu_server_to_lastest_lts_version

If possible, don’t use SSH when upgrading a server. On Linode, you can use their Lish terminal, available from the virtual server’s console page.

This was tested while upgrading a 10.04 LTS Ubuntu Server to 12.04 LTS. The test server was a production web server with a large number of packages installed and configuration changes.

Back up the Server First

If your virtual hosting provider offers image backups, this is a good option. The important thing is knowing for certain that you can restore/recreate the server in case the upgraded server is not left in a bootable, usable condition.

Install the Upgrade Manager

sudo aptitude -y install update-manager-core

Double-check Configuration File

Run this command to check whether “/etc/update-manager/release-upgrades” has the line “Prompt=lts”.

[[ `grep Prompt=lts /etc/update-manager/release-upgrades` = 'Prompt=lts' ]] && echo '"/etc/update-manager/release-upgrades" is Ok' || echo 'Edit /etc/update-manager/release-upgrades and add line "Prompt=lts"'

If the line is not present edit “/etc/update-manager/release-upgrades” with:

sudo vi /etc/update-manager/release-upgrades

…and add the line:

Prompt=lts

Run the Upgrade Manger

sudo do-release-upgrade

Follow the on-screen instructions.

When this was tested on a production server:

  • The upgrade went smoothly.
  • When prompted for a new MySQL root password (sever times during the upgrade) Enter was pressed without entering a new password. The existing MySQL password was retained by the server with no issues.
  • When the upgrade manager encountered a configuration file with custom changes, the existing, modified configuration file was retained (not replaced with the distribution default configuration file). This worked well. The only change needed after upgrading was adding a new line to phpMyAdmin’s configuration file that was needed for the new phpMyAdmin version.

Google PageSpeed Apache Module on Ubuntu Server

From Adminbuntu, Everything for the Ubuntu Server Administrator

This is an excerpt from Adminbuntu, a site for Ubuntu Server administrators:

http://www.adminbuntu.com/google_pagespeed_module_on_apache

PageSpeed speeds up your website and reduces page load time. The PageSpeed Apache module automatically applies web performance best practices to pages and assets like CSS, JavaScript and images, without requiring site modification.

PageSpeed Project Page: https://developers.google.com/speed/pagespeed/module

Installation

Determine Whether 32-bit or 64-bit Ubuntu Server is Installed

if [[ `uname -a` == *_64* ]] ; then echo '64-bit' ; else echo '32-bit' ; fi

Download the PageSpeed Apache Module

  • Using your web browser, navigate to https://developers.google.com/speed/pagespeed/module/download.
  • Look on the right side of the page, under Latest Stable Version and right-click on either the 32-bit or 64-bit .deb package (based on whether you are running 32-bit or 64-bit Ubuntu Server. Select Copy Link Address to get the download link in your clipboard.
    In your server’s terminal use wget to download the module.

This this example, the link for the 32-bit .deb file is https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_i386.deb:

wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_i386.deb

Install the PageSpeed Apache Module

sudo dpkg -i mod-pagespeed-*.deb
sudo apt-get -f install
sudo service apache2 restart

Test Installation

You can verify that the PageSpeed module is installed and enabled with:

[[ -a '/etc/apache2/mods-enabled/pagespeed.conf' ]] ; then echo 'pagespeed is enabled' ; else echo 'pagespeed is not enabled' ; fi

Flushing PageSpeed Server-Side Cache

When developing web pages with PageSpeed enabled, it is sometimes necessary to flush the server’s PageSpeed cache to get the system to reload CSS or JavaScript files that have been updated before the cache lifetime expires.

To do this, touch the file cache.flush:

sudo touch /var/cache/mod_pagespeed/cache.flush

http://www.adminbuntu.com/

Install Dropbox on Ubuntu Server

Dropbox is so useful! Wouldn’t it be great to have that same convenience and function for your user account on your server, just like you have on your workstation?

This has been tested on Ubuntu Lucid and Jaunty. This procedure will create a system with:

  • Install a separate Dropbox client (daemon) for individual users
  • Each user has a separate Dropbox account
  • All the daemons will be managed together with normal daemon controls

This allows individuals to have their own Dropbox accounts, each with a separate process syncing their individual ~/Dropbox directory. With one command an admin can start or stop all the daemons at once.

Install Prerequisites

Later, we’ll need to read a sqlite3 database record, so install sqlite3.

sudo aptitude -y install sqlite3

Install Dropbox client for an individual user

This step is repeated for each user that wants a Dropbox client. Start by setting up your own account, then repeat for each user. This is run with a user’s own account. The changes made all take place in their home directory.

First, determine whether you have 32-bit or 64-bit Ubuntu Server. You must install the correct version, either 32 or 64 bit or it will not work. The following command will tell you which is installed:

uname -a | grep '_64' >/dev/null && echo 'A 64-bit OS is installed'; uname -a | grep '_64' >/dev/null || echo 'A 32-bit OS is installed'

Run the correct installation, based on whether a 32-bit or 64-bit OS is installed.

32-bit installation:

cd ~
wget -O dropbox.tar.gz http://www.dropbox.com/download/?plat=lnx.x86
tar -zxof dropbox.tar.gz

or

64-bit installation:

cd ~
wget -O dropbox.tar.gz http://www.dropbox.com/download/?plat=lnx.x86_64
tar -zxof dropbox.tar.gz

Link user’s Dropbox client to their Dropbox account:

wget http://dl.dropbox.com/u/6995/dbmakefakelib.py
python dbmakefakelib.py

The above will run for a little while without printing anything, then print “dropboxd ran for 15 seconds without quitting – success?”. When it does so, press control-c twice. Yes, it is unusual. What this does is populate a sqlite3 database with an ID from the Dropbox server. Next, we’ll extract that code and use it to link your Dropbox user account with this CLI Dropbox client instance.

On the server, via SSH

Get the URL with:

echo https://www.dropbox.com/cli_link?host_id=`echo '.dump config' | sqlite3 ~/.dropbox/dropbox.db | grep host_id | cut -d \' -f 4 | python -c 'print raw_input().decode("base64")' | grep '^V' | cut -b 2-`

On your local machine, in a web browser

Copy the URL that the above printed and paste it into a web browser. When you do so, Dropbox will register your client on the server with your Dropbox account.

At this point, Dropbox will not be quite working yet. The next steps will take care of that.

Create Dropbox daemon control

The next task is to create a system to start and stop the dropbox daemon for each user on the system that has Dropbox installed for his/her user account. The following daemon init script was lifted from: http://wiki.dropbox.com/TipsAndTricks/TextBasedLinuxInstall.

Of course, this is for the use of the system admin. This creates a normal daemon init start/stop script and installs it so the Dropbox daemons are started when the system boots. The admin can also use this to control the Dropbox daemons manually.

sudo vi /etc/init.d/dropbox

Paste in the following code. Then, modify line 3, replacing “user1 user2″ with your username. For future reference, additional user’s Dropbox daemons can be controlled with this one script – add additional username separated with spaces.

# dropbox service
# separate usernames in the following line with spaces.
DROPBOX_USERS="user1 user2"
 
DAEMON=.dropbox-dist/dropbox
 
start() {
    echo "Starting dropbox..."
    for dbuser in $DROPBOX_USERS; do
        HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
        if [ -x $HOMEDIR/$DAEMON ]; then
            HOME="$HOMEDIR" start-stop-daemon -b -o -c $dbuser -S -u $dbuser -x $HOMEDIR/$DAEMON
        fi
    done
}
 
stop() {
    echo "Stopping dropbox..."
    for dbuser in $DROPBOX_USERS; do
        HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
        if [ -x $HOMEDIR/$DAEMON ]; then
            start-stop-daemon -o -c $dbuser -K -u $dbuser -x $HOMEDIR/$DAEMON
        fi
    done
}
 
status() {
    for dbuser in $DROPBOX_USERS; do
        dbpid=`pgrep -u $dbuser dropbox`
        if [ -z $dbpid ] ; then
            echo "dropboxd for USER $dbuser: not running."
        else
            echo "dropboxd for USER $dbuser: running (pid $dbpid)"
        fi
    done
}

case "$1" in
  start)
    start
    ;;
 
  stop)
    stop
    ;;
 
  restart|reload|force-reload)
    stop
    start
    ;;
 
  status)
    status
    ;;
 
  *)
    echo "Usage: /etc/init.d/dropbox {start|stop|reload|force-reload|restart|status}"
    exit 1
 
esac
 
exit 0

Make the init script executable and restart the daemon:

sudo chmod +x /etc/init.d/dropbox
sudo /etc/init.d/dropbox restart

When you restart the daemon, it will be running correctly linked to your Dropbox account. It will create a Dropbox directory in your home directory and will start to populate it with files you have on Dropbox.

Have the daemon(s) run automatically at boot time:

sudo update-rc.d dropbox defaults

The above correctly copies links as needed so the daemon(s) start when the server boots.

Managing the Daemons

A separate daemon will be run for each user that has the Dropbox client installed – with only one command. This makes it easy for individual users to have separate Dropbox accounts, each syncing to ~/Dropbox for their user account. Here are the commands to manage these daemons:

Start Dropbox services for all users:

sudo /etc/init.d/dropbox start

Stop Dropbox services for all users:

sudo /etc/init.d/dropbox stop

Restart Dropbox services for all users:

sudo /etc/init.d/dropbox restart

Get service status for each user Dropbox service:

sudo /etc/init.d/dropbox status

Script to create a Kohana instance

This is a copy of the script I use to create an instance of a Kohana project. It works on Ubuntu server (tested with Lucid). Adapt it to your own needs, of course.

#! /bin/bash
 
CODE="upc"
OWNER="myusername"
GROUP="mygroupname"
KOHANA_URL="http://dev.kohanaframework.org/attachments/download/1355/kohana-v2.3.4.zip"
KOHANA_ZIP="kohana-v2.3.4.zip"
SITESROOT="/usr/local/var/www"
URL="www.mysite.com"
# database.php parameters
DBUSER="mydbuser"
DBPWD="mydbpasswd"
DBHOST="localhost"
DBDATABASE="mydbname"
 
#
# make sure this is run as root
#
if [[ $UID -ne 0 ]]; then
    echo "Not running as root"
    exit
fi
 
if [ ! -d $SITESROOT ]; then
	echo "$SITESROOT directory does not exist"
	exit
fi
 
#
# change to the siteroot directory
#
cd $SITESROOT
 
if [ -d $URL ]; then
	echo "$SITESROOT/$URL directory already exists"
	exit
fi 
 
mkdir $URL
chown $OWNER:$GROUP $URL
cd $URL
 
#
# create kohana directory
#
echo creating kohana directory
mkdir kohana
chown $OWNER:$GROUP kohana
chmod g+ws kohana
wget "$KOHANA_URL"
unzip "$KOHANA_ZIP"
rm "$KOHANA_ZIP"
chown -R $OWNER:$GROUP kohana
chmod -R g+w kohana
chown $OWNER:www-data kohana/application/logs
rm kohana/example.htaccess
rm kohana/install.php
rm kohana/Kohana\ License.html
rm kohana/kohana.png
 
#
# create public directory
#
echo creating public directory
mkdir public
chown $OWNER:$GROUP public
chmod g+ws public
mv kohana/index.php public/
sed -i "s\\kohana_application =.*$\\kohana_application = '../kohana/application';\\" public/index.php
sed -i "s\\kohana_modules =.*$\\kohana_modules = '../kohana/modules';\\" public/index.php
sed -i "s\\kohana_system =.*$\\kohana_system = '../kohana/system';\\" public/index.php
echo "# turn on URL rewriting
RewriteEngine On
RewriteBase /
" > public/.htaccess
 
#
# create Apache logs directory
#
echo creating logs directory
mkdir logs
chown $OWNER:www-data logs
chmod g+ws logs
touch logs/error.log
touch logs/combined.log
 
#
# create utilties directory
#
echo creating utilties directory
mkdir utilities
chown $OWNER:$GROUP utilities
chmod g+ws utilities
 
#
# modify kohana/application/config/config.php
#
echo modifying kohana/application/config/config.php
sed -i "s\\'/kohana/'\\'$URL/'\\" kohana/application/config/config.php
 
#
# create kohana/application/config/database.php
#
echo creating kohana/application/config/database.php
cp kohana/system/config/database.php kohana/application/config/database.php
chown $OWNER:$GROUP kohana/application/config/database.php
chmod g+w kohana/application/config/database.php
sed -i "s\\'user'.*$\\'user'     => '$DBUSER',\\" kohana/application/config/database.php
sed -i "s\\'pass'.*$\\'pass'     => '$DBPWD',\\" kohana/application/config/database.php
sed -i "s\\'host'.*$\\'host'     => '$DBHOST',\\" kohana/application/config/database.php
sed -i "s\\'database'.*$\\'database' => '$DBDATABASE',\\" kohana/application/config/database.php
 
#
# create kohana/application/config/routes.php
#
echo creating kohana/application/config/routes.php
cp kohana/system/config/routes.php kohana/application/config/routes.php
chown $OWNER:$GROUP kohana/application/config/routes.php
chmod g+w kohana/application/config/routes.php
sed -i "s\\'welcome'\\'/index'\\" kohana/application/config/routes.php
 
#
# copy kohana/application/config/profiler.php
#
echo creating kohana/application/config/profiler.php
cp kohana/system/config/profiler.php kohana/application/config/profiler.php
chown $OWNER:$GROUP kohana/application/config/profiler.php
chmod g+w kohana/application/config/profiler.php
 
#
# create Apache virtual site
#
echo creating Apache virtual site file in /etc/apache2/sites-available
echo "
	ServerName $URL
	DocumentRoot $SITESROOT/$URL/public
	DirectoryIndex index.php
	LogLevel warn
	ErrorLog $SITESROOT/$URL/logs/error.log
	CustomLog $SITESROOT/$URL/logs/combined.log combined

" > /etc/apache2/sites-available/$URL
 
#
# finish up
#
echo "enable site with: sudo a2ensite $URL"
echo "restart Apache with: sudo /etc/init.d/apache2 restart"

Creating a Perl Daemon in Ubuntu

If you have a process that need to run in the background, creating a daemon is the key. Some examples of what your can do:

  • Wait for video files to be dropped via FTP and then process them
  • Check for a recurring problem and report it to admin
  • Monitor system load and report to the admin at a certain threshold

In this post we’ll take a look at how to create a Perl daemon script and how to get it to run as a daemon whenever the system is started. For the Perl script, a template will be created and for control of the daemon process, standard Debian facilities will be used.

What will be covered:

  • Create the start/stop script in /etc/init.d
  • Create the Perl daemon script in /usr/bin/
  • Create the log file in /var/log/
  • Get the daemon to start automatically when the system boots
  • How to manage your daemon

The start/stop script starts and stops the daemon. It is also used by the system to start your daemon when the system starts.

The Perl daemon script contains your custom Perl code to run in the background. It executes your code every x number of seconds.

Create the Start/stop Script

Happily, Debian and therefore Ubuntu, supplies a template that uses the Debian start-stop-daemon command to start and stop daemons. It is only necessary to copy this template to a new file of the correct name and modify it for the purpose.

These scripts reside in the /etc/init.d/ directory. The script you create will have the exact same filename as the Perl daemon script you will create and the name cannot have a file extension – just the bare name.

Substitute your daemon’s name for “mydaemon” in the examples.

sudo cp /etc/init.d/skeleton /etc/init.d/mydaemon
sudo chmod +x /etc/init.d/mydaemon
sudo vi /etc/init.d/mydaemon

Make changes to your start/stop script. Locate the header in the script:

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="Description of the service"
NAME=daemonexecutablename
DAEMON=/usr/bin/$NAME
DAEMON_ARGS="--options args"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
  • Change the content of DESC to a meaningful description of your daemon.
  • Change the content for NAME to the exact name of your daemon.
  • Create the Perl Daemon Script

    Use the Perl Daemon Script Template located at the end of this post. In the following, the script template is copied to /usr/bin/where executables specific to this system are located. Actually, you would copy the template to your home directory, make the changes and then copy the daemon script to /usr/bin/.

    sudo cp daemon_template.pl /usr/bin/mydaemon
    sudo vi /usr/bin/mydaemon
    sudo chmod +x /usr/bin/mydaemon
    

    The script template is commented with the changes that are needed. In particular, change the value of $daemonName to the exact name of your new daemon. Ten, add your custom code where # do something appears.

    Prerequisites

    The script uses File::Pid and POSIX from CPAN, so you’ll need to install that module:

    sudo aptitude install cpan
    sudo cpan POSIX
    sudo cpan File::Pid
    

    Create the Log File

    We’ll create the log file so it has the correct ownership and permissions. The log fie has the daemon name appended with “.log”. It is located in the /var/log/ directory.

    sudo touch /var/log/mydaemon.log
    sudo chmod 640 /var/log/mydaemon.log
    sudo chown root:adm /var/log/mydaemon.log
    

    The permissions and ownership change allow adm group members to read the log, per convention. Add yourself to adm group to view logs.

    Run your Daemon

    As you have created a standard start/stop script for your daemon, you can start it the standard way:

    sudo /etc/init.d/mydaemon start
    

    Stop your Daemon

    Similarly, your daemon is easy to stop and restart:

    sudo /etc/init.d/mydaemon stop
    
    sudo /etc/init.d/mydaemon restart
    

    Automatically Start your Daemon

    This command tells your system to start your daemon automatically when the system starts:

    update-rc.d mydaemon defaults 99
    

    To keep your daemon from starting, if needed:

    update-rc.d -f mydaemon remove
    

    Managing Daemons

    List running daemons

    Well-behaved daemons generally create a PID file in /var/run/. List that directory and your have a fair list of running daemons.

    sudo ls /var/run/
    

    Get PID for a particular daemon

    If you know the name of a daemon and want to see if it is running and get its PID, use pgrep.

    pgrep mydaemon
    

    Also useful is the ps command, which list processes.

    ps aux
    

    The aux switch limits output to processes not associated with a terminal.

    Show programs running as daemons

    This line attempts to show programs that are running as daemons.

    which `ps aux | cut -c 66- | cut -d\  -f 1` | sort | uniq
    

    The Perl Daemon Script Template

    #!/usr/bin/perl -w
    #
    # mydaemon.pl by Andrew Ault, www.andrewault.net
    #
    # Free software. Use this as you wish.
    #
    # Throughout this template "mydaemon" is used where the name of your daemon should
    # be, replace occurrences of "mydaemon" with the name of your daemon.
    #
    # This name will also be the exact name to give this file (WITHOUT a ".pl" extension).
    #
    # It is also the exact name to give the start-stop script that will go into the
    # /etc/init.d/ directory.
    #
    # It is also the name of the log file in the /var/log/ directory WITH a ".log"
    # file extension.
    #
    # Replace "# do something" with your super useful code.
    #
    # Use "# logEntry("log something");" to log whatever your need to see in the log.
    #
    use strict;
    use warnings;
    use POSIX;
    use File::Pid;
     
    # make "mydaemon.log" file in /var/log/ with "chown root:adm mydaemon"
     
    # TODO: change "mydaemon" to the exact name of your daemon.
    my $daemonName    = "mydaemon";
    #
    my $dieNow        = 0;                                     # used for "infinte loop" construct - allows daemon mode to gracefully exit
    my $sleepMainLoop = 120;                                    # number of seconds to wait between "do something" execution after queue is clear
    my $logging       = 1;                                     # 1= logging is on
    my $logFilePath   = "/var/log/";                           # log file path
    my $logFile       = $logFilePath . $daemonName . ".log";
    my $pidFilePath   = "/var/run/";                           # PID file path
    my $pidFile       = $pidFilePath . $daemonName . ".pid";
     
    # daemonize
    use POSIX qw(setsid);
    chdir '/';
    umask 0;
    open STDIN,  '/dev/null'   or die "Can't read /dev/null: $!";
    open STDOUT, '>>/dev/null' or die "Can't write to /dev/null: $!";
    open STDERR, '>>/dev/null' or die "Can't write to /dev/null: $!";
    defined( my $pid = fork ) or die "Can't fork: $!";
    exit if $pid;
     
    # dissociate this process from the controlling terminal that started it and stop being part
    # of whatever process group this process was a part of.
    POSIX::setsid() or die "Can't start a new session.";
     
    # callback signal handler for signals.
    $SIG{INT} = $SIG{TERM} = $SIG{HUP} = \&signalHandler;
    $SIG{PIPE} = 'ignore';
     
    # create pid file in /var/run/
    my $pidfile = File::Pid->new( { file => $pidFile, } );
     
    $pidfile->write or die "Can't write PID file, /dev/null: $!";
     
    # turn on logging
    if ($logging) {
    	open LOG, ">>$logFile";
    	select((select(LOG), $|=1)[0]); # make the log file "hot" - turn off buffering
    }
     
    # "infinite" loop where some useful process happens
    until ($dieNow) {
    	sleep($sleepMainLoop);
     
    	# TODO: put your custom code here!
    	# do something
     
    	# logEntry("log something"); # use this to log whatever you need to
    }
     
    # add a line to the log file
    sub logEntry {
    	my ($logText) = @_;
    	my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime(time);
    	my $dateTime = sprintf "%4d-%02d-%02d %02d:%02d:%02d", $year + 1900, $mon + 1, $mday, $hour, $min, $sec;
    	if ($logging) {
    		print LOG "$dateTime $logText\n";
    	}
    }
     
    # catch signals and end the program if one is caught.
    sub signalHandler {
    	$dieNow = 1;    # this will cause the "infinite loop" to exit
    }
     
    # do this stuff when exit() is called.
    END {
    	if ($logging) { close LOG }
    	$pidfile->remove if defined $pidfile;
    }
    

    UPDATE:

    Looks like there’s a page in French about this post: http://www.duncane.net/2011/02/25/perl-daemon/. Merci Duncane!

    Securing an Ubuntu Server

    For more current information, see: http://www.adminbuntu.com/security

    Security is relative. Will these steps make your server “secure”? It will be more secure than it was before. And more secure than most servers. Your server will not be “low hanging fruit”. Security is an on-going process. It includes settings, practices and procedures. Make it your business to regularly read about security and to understand the concepts and our system. Paranoia is useful with regard to server security.

    I’ve tested what is presented here in Ubuntu Server 10.04 (Lucid) and 10.10 (Maverick). If you want to harden your new Ubuntu server, this is a good start.

    Ubuntu server is well designed, regularly updated and relatively secure. The Ubuntu Security Team manifests an onging effort to keep Ubuntu secure. Regular security updates are available and easy to implement.

    • No open ports
    • Role-based administration
    • No X server
    • Security updates
    • Kernel and compiler hardening

    In this post, we are going to meet the security challenge in with multi-pronged effort that will include: system analysis, changing settings for additional hardening against attack, installing a firewall maintenance system, scanning for rootkits, and offering a regular maintenance regimen.

    • Change settings for increased security
    • Implement UFW, the uncomplicated firewall
    • Use denyhosts to automatically blacklist attackers
    • Scan the system for vulnerabilities with Tiger
    • Detect attempted intrusions with psad
    • Install nmap and scan the system for open ports
    • Check the system for rootkits with chkrootkit
    • Monitor logs

    Change settings for increased security

    see also: https://help.ubuntu.com/community/StricterDefaults

    Secure shared memory

    /dev/shm can be used in an attack against a running service, such as httpd. Modify /etc/fstab to make it more secure.

    sudo vi /etc/fstab
    

    Add this line:

    tmpfs     /dev/shm     tmpfs     defaults,noexec,nosuid     0     0
    

    Disable root SSH login

    The root account is disabled by default in Ubuntu. If you installed Ubuntu on Slicehost or Linode, root is enabled. In any case, it is a good idea to disable root SSH access. Edit /etc/ssh/sshd_config and set PermitRootLogin to no.

    sudo vi /etc/ssh/sshd_config
    

    Change PermitRootLogin to no:

    PermitRootLogin no
    

    Of course, if you access your server via SSH, you should make sure you have sudo working for your user before disabling SSH root access.

    Only allow admin users to use su

    This helps prevent privilege escalation.

    By default, Ubuntu does not have an admin group. Create an admin group:

    sudo groupadd admin
    

    Add yourself to the admin group:

    sudo usermod -a -G admin andrew
    

    Restrict access to /bin/su to admin group members:

    sudo dpkg-statoverride --update --add root admin 4750 /bin/su
    

    Check permissions for /bin/su with:

    ls -lh /bin/su
    

    …and see the following:

    -rwsr-x--- 1 root admin 31K 2010-01-26 17:09 /bin/su
    

    Do not permit source routing of incoming packets

    see also: http://www.cromwell-intl.com/security/security-stack-hardening.html

    sudo sysctl -w net.ipv4.conf.all.accept_source_route=0
    sudo sysctl ­-w net.ipv4.conf.default.accept_source_route=0
    

    Don’t allow system users to access an FTP server

    This is only needed is ftpd is installed and running. Only if you’ve installed ftpd. However, it is Ok to do this anyway and it will remove a FAIL from the tiger report.

    SFTP is probably better than FTP, if it is usable for your files transfer needs.

    see ftpusers manual: http://manpages.ubuntu.com/manpages/lucid/man5/ftpusers.5.html

    Edit /etc/ftpusers:

    sudo vi /etc/ftpusers
    

    Add system users to deny use of ftpd:

    backup
    bin
    daemon
    games
    gnats
    irc
    libuuid
    list
    lp
    mail
    man
    mysql
    news
    ntp
    postfix
    proxy
    sshd
    sync
    sys
    syslog
    uucp
    www-data
    

    UFW: basic firewall

    previous post: Ubuntu UFW Uncomplicated Firewall Examples

    community documentation: https://help.ubuntu.com/community/UFW

    server guide: https://help.ubuntu.com/10.04/serverguide/C/firewall.html

    ufw manual: http://manpages.ubuntu.com/manpages/lucid/en/man8/ufw.8.html

    project wiki: https://wiki.ubuntu.com/UncomplicatedFirewall

    nice article: http://savvyadmin.com/ubuntus-ufw/

    UFW (Uncomplicated Firewall) provides an easy to understand interface to control iptables (iptables conteol Netfilter, which is built into the kernel). Will just a few commands, your server can control access. Checking status is also easy.

    UFW (uncomplicated firewall) is a simple interface used to configure iptables.

    Install and enable Uncomplicated Firewall:

    sudo aptitude install -y ufw
    sudo ufw enable
    

    Display available UFW commands:

    sudo ufw show
    

    Display UFW configuration:

    sudo ufw status
    

    Allow SSH and HTTP access to the Apache server:

    sudo ufw allow ssh
    sudo ufw allow http
    

    In the above example, ports for OpenSSH and Apache were opened by service name (“ssh” and “http”). You can use a port number instead of the service name (like “80” instead of “http”).

    See services running and which names to use:

    The practice here is to open only ports that you use – ports that use a service that have a service running. To see a list of services that you have running for which you might want to open ports for:

    sudo ufw app list
    

    To see a list of services that UFW uses (like in the “sudo ufw allow ssh” example, above):

    less /etc/services
    

    Denyhosts: avoid SSH attacks

    project: http://denyhosts.sourceforge.net/

    Looking at /var/log/auth.log on servers that I manage shows a steady streams of attacks on SSH. I am countering these attacks in a number of ways, starting with denyhosts.

    Denyhosts periodically scans /var/log/auth.log for repeated failures to access the system via SSH. It then adds these offenders to /etc/hosts.deny. See the project page for details.

    sudo aptitude -y install denyhosts
    

    That does it – the rest is automatic. You can see the IP addresses added to /etc/hosts.deny with:

    sudo less /etc/hosts.deny
    

    Tiger: security system scanner

    project: http://www.nongnu.org/tiger/

    Tiger creates an automated security audit by analyzing files and settings on the system and creating a report listing what has been analyzed and listing warning, alerts and failures.

    The tiger command creates a report of potential security problems in /var/log/tiger. The use the tigexp command to look up the resulting codes generated for a detailed explanation and what to do to make the system more secure. The problems tiger considers most serious are marked with FAIL.

    It has been a while since Tiger has been updated. It still produces a useful report.

    Install tiger:

    sudo aptitude -y install tiger
    

    Run tiger to create a report of security issues.

    sudo tiger
    

    Use less to view the most recent tiger report:

    sudo -i
    less /var/log/tiger/`ls -t1 /var/log/tiger | head -1`
    exit
    

    Use tigexp to list explanations for FAIL codes:

    tigexp dev002f
    

    Google is also helpful, naturally.

    Ignore these:

    --FAIL-- [dev002f] /dev/fuse has world permissions
    --FAIL-- [logf005f] Log file /var/log/btmp permission should be 660
    

    Changing permissions for these could cause problems.

    Detect attempted intrusions with psad

    project: http://www.cipherdyne.org/psad/

    Psad is a collection of lightweight daemons that log attempted intrusions, in particular monitoring iptables.

    Installation:

    sudo aptitude -y install psad
    

    The daemons will run automatically.

    To check current status:

    sudo psad -S
    

    You can modify psad settings to e-mail the admin in the event of intrusion detection.

    Nmap: port scanning

    project: http://nmap.org/

    This allows you to see which ports are open, verifying that UFW/iptables is working correctly.

    Installing nmap:

    sudo aptitude install -y nmap
    

    Port scanning:

    nmap -v -sT localhost
    

    SYN Scanning:

    sudo nmap -v -sS localhost
    

    scan type explanations: http://nmap.org/book/man-port-scanning-techniques.html

    Chkrootkit: check for rootkit presence

    project: http://www.chkrootkit.org/

    Chkrootkit scans the system for evidence that a rootkit has been installed.

    This is a confidence test to be used to test whether your system has been compromised. In a perfect world you would not need this…but in this world, it is good to run periodically.

    Installing chkrootkit:

    sudo aptitude install -y chkrootkit
    

    Running chkrootkit:

    sudo chkrootkit
    

    LogWatch

    Ubuntu community documentation: https://help.ubuntu.com/community/Logwatch

    The most detailed and informative logs in the world are useless if no one looks at them. Logwatch winnows the deluge to a succinct report…which you will look at. Even so, familiarize yourself with your system’s logs and review them on a regular basis. A daily logwatch habit would be a good start.

    Installation:

    sudo aptitude -y install logwatch
    

    Usage:

    sudo logwatch | less
    

    Ongoing maintenance

    Your server is now more secure. Once a week, perform on-going maintenance.

    Updating software:

    sudo aptitude update
    sudo aptitude safe-upgrade
    

    The safe-upgrade action is preferred by me because it does not upgrade packages that rely on dependencies that have not been upgraded to required levels.

    see: http://wiki.debian.org/Aptitude

    Or, you could set-up automatic security updates, if you cannot do the weekly maintenance. This is not a perfect solution because an administrator is not monitoring what is being updated and testing after updates. see: https://help.ubuntu.com/10.04/serverguide/C/automatic-updates.html

    Check for attempted instrusions:

    sudo psad -S
    

    UPDATED: Analyze system with tiger. Because the tiger reports in /var/log/tiger/are owned by root, run these commands one at a time. (This solves a problem some people were having with permissions.)

    sudo -i
    tiger
    grep FAIL /var/log/tiger/`ls -t1 /var/log/tiger | head -1`
    exit
    

    In the above, FAILs are pulled from the newest report file with grep. The ls clause in backticks gives grep the newest file in the directory. The sudo -i command allows you to run multiple commands as root, ending with exit.

    Use tigexp to list explanations for FAIL codes:

    tigexp dev002f
    

    Scan ports with nmap:

    sudo nmap -v -sS localhost
    

    Check for rootkits

    sudo chkrootkit
    

    Look at logs:

    sudo logwatch | less
    

    Keep up with trends

    visit: http://www.linuxsecurity.com/

    Elsewhere

    http://www.itsecurity.com/features/ubuntu-secure-install-resource/

    http://www.cyberciti.biz/tips/linux-security.html

    List of installed packages on Ubuntu

    This lists packages added after Ubuntu was installed.

    When I upgrade from one version of Ubuntu to another, I do a fresh install. I have a procedure on my personal wiki with all of the steps to move in. Part of that is restoring data from backup, Part is restoring or recreating configuration files.

    But what about software packages? There is a baseline of packages that I use all the time, but I also add packages ad hoc as required. It is convenient to get a list of what is installed.

    Here’s a one-liner to list installed packages on Ubuntu:

    cat /var/log/installer/initial-status.gz | gzip -d | grep '^Package:' | awk '{ print $2}' > tmp.txt && aptitude search -F %p '~i!~M' | awk '{ print $1}' | grep -v -F -f tmp.txt && rm tmp.txt
    

    The list that is returned is pretty good. You can save the output to a text file and edit that file to get the list of packages you want to install on the new system.

    Here’s how to output to packages.txt:

    cat /var/log/installer/initial-status.gz | gzip -d | grep '^Package:' | awk '{ print $2}' > tmp.txt && aptitude search -F %p '~i!~M' | awk '{ print $1}' | grep -v -F -f tmp.txt > packages.txt && rm tmp.txt
    

    As you can see, it greps out a list of packages included in the original install, grabs the second field with awk and stuffs the result in a temporary file. Then a list of installed packages is generated, again using awk to parse out the package name field. Then grep is used to compare thee two lists and output just the packages installed after initial installation.

    Here is what I have installed:

    apache2
    apache2-utils
    apache2.2-common
    audacity
    bluefish
    build-essential
    figlet
    flashplugin-installer
    gimp
    gnome-do
    google-chrome-beta
    inkscape
    keepassx
    libapache2-mod-php5
    mailutils
    mysql-server
    ninvaders
    nvidia-current
    padre
    php5
    php5-cli
    php5-common
    php5-curl
    php5-dev
    php5-gd
    php5-imagick
    php5-mcrypt
    php5-memcache
    php5-mysql
    php5-pspell
    php5-snmp
    php5-sqlite
    php5-xmlrpc
    php5-xsl
    phpmyadmin
    skype
    vim
    vlc
    winff
    xtightvncviewer
    

    So, this is a list of 40 packages installed on my machine, out of a total of 1560 packages returned by “dpkg –list” (thanks, Mark!). Just those I installed, not everything including all of baseline Ubuntu.