Monthly Archives: May 2010

Creating a Perl Daemon in Ubuntu

If you have a process that need to run in the background, creating a daemon is the key. Some examples of what your can do:

  • Wait for video files to be dropped via FTP and then process them
  • Check for a recurring problem and report it to admin
  • Monitor system load and report to the admin at a certain threshold

In this post we’ll take a look at how to create a Perl daemon script and how to get it to run as a daemon whenever the system is started. For the Perl script, a template will be created and for control of the daemon process, standard Debian facilities will be used.

What will be covered:

  • Create the start/stop script in /etc/init.d
  • Create the Perl daemon script in /usr/bin/
  • Create the log file in /var/log/
  • Get the daemon to start automatically when the system boots
  • How to manage your daemon

The start/stop script starts and stops the daemon. It is also used by the system to start your daemon when the system starts.

The Perl daemon script contains your custom Perl code to run in the background. It executes your code every x number of seconds.

Create the Start/stop Script

Happily, Debian and therefore Ubuntu, supplies a template that uses the Debian start-stop-daemon command to start and stop daemons. It is only necessary to copy this template to a new file of the correct name and modify it for the purpose.

These scripts reside in the /etc/init.d/ directory. The script you create will have the exact same filename as the Perl daemon script you will create and the name cannot have a file extension – just the bare name.

Substitute your daemon’s name for “mydaemon” in the examples.

sudo cp /etc/init.d/skeleton /etc/init.d/mydaemon
sudo chmod +x /etc/init.d/mydaemon
sudo vi /etc/init.d/mydaemon

Make changes to your start/stop script. Locate the header in the script:

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="Description of the service"
NAME=daemonexecutablename
DAEMON=/usr/bin/$NAME
DAEMON_ARGS="--options args"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
  • Change the content of DESC to a meaningful description of your daemon.
  • Change the content for NAME to the exact name of your daemon.
  • Create the Perl Daemon Script

    Use the Perl Daemon Script Template located at the end of this post. In the following, the script template is copied to /usr/bin/where executables specific to this system are located. Actually, you would copy the template to your home directory, make the changes and then copy the daemon script to /usr/bin/.

    sudo cp daemon_template.pl /usr/bin/mydaemon
    sudo vi /usr/bin/mydaemon
    sudo chmod +x /usr/bin/mydaemon
    

    The script template is commented with the changes that are needed. In particular, change the value of $daemonName to the exact name of your new daemon. Ten, add your custom code where # do something appears.

    Prerequisites

    The script uses File::Pid and POSIX from CPAN, so you’ll need to install that module:

    sudo aptitude install cpan
    sudo cpan POSIX
    sudo cpan File::Pid
    

    Create the Log File

    We’ll create the log file so it has the correct ownership and permissions. The log fie has the daemon name appended with “.log”. It is located in the /var/log/ directory.

    sudo touch /var/log/mydaemon.log
    sudo chmod 640 /var/log/mydaemon.log
    sudo chown root:adm /var/log/mydaemon.log
    

    The permissions and ownership change allow adm group members to read the log, per convention. Add yourself to adm group to view logs.

    Run your Daemon

    As you have created a standard start/stop script for your daemon, you can start it the standard way:

    sudo /etc/init.d/mydaemon start
    

    Stop your Daemon

    Similarly, your daemon is easy to stop and restart:

    sudo /etc/init.d/mydaemon stop
    
    sudo /etc/init.d/mydaemon restart
    

    Automatically Start your Daemon

    This command tells your system to start your daemon automatically when the system starts:

    update-rc.d mydaemon defaults 99
    

    To keep your daemon from starting, if needed:

    update-rc.d -f mydaemon remove
    

    Managing Daemons

    List running daemons

    Well-behaved daemons generally create a PID file in /var/run/. List that directory and your have a fair list of running daemons.

    sudo ls /var/run/
    

    Get PID for a particular daemon

    If you know the name of a daemon and want to see if it is running and get its PID, use pgrep.

    pgrep mydaemon
    

    Also useful is the ps command, which list processes.

    ps aux
    

    The aux switch limits output to processes not associated with a terminal.

    Show programs running as daemons

    This line attempts to show programs that are running as daemons.

    which `ps aux | cut -c 66- | cut -d\  -f 1` | sort | uniq
    

    The Perl Daemon Script Template

    #!/usr/bin/perl -w
    #
    # mydaemon.pl by Andrew Ault, www.andrewault.net
    #
    # Free software. Use this as you wish.
    #
    # Throughout this template "mydaemon" is used where the name of your daemon should
    # be, replace occurrences of "mydaemon" with the name of your daemon.
    #
    # This name will also be the exact name to give this file (WITHOUT a ".pl" extension).
    #
    # It is also the exact name to give the start-stop script that will go into the
    # /etc/init.d/ directory.
    #
    # It is also the name of the log file in the /var/log/ directory WITH a ".log"
    # file extension.
    #
    # Replace "# do something" with your super useful code.
    #
    # Use "# logEntry("log something");" to log whatever your need to see in the log.
    #
    use strict;
    use warnings;
    use POSIX;
    use File::Pid;
     
    # make "mydaemon.log" file in /var/log/ with "chown root:adm mydaemon"
     
    # TODO: change "mydaemon" to the exact name of your daemon.
    my $daemonName    = "mydaemon";
    #
    my $dieNow        = 0;                                     # used for "infinte loop" construct - allows daemon mode to gracefully exit
    my $sleepMainLoop = 120;                                    # number of seconds to wait between "do something" execution after queue is clear
    my $logging       = 1;                                     # 1= logging is on
    my $logFilePath   = "/var/log/";                           # log file path
    my $logFile       = $logFilePath . $daemonName . ".log";
    my $pidFilePath   = "/var/run/";                           # PID file path
    my $pidFile       = $pidFilePath . $daemonName . ".pid";
     
    # daemonize
    use POSIX qw(setsid);
    chdir '/';
    umask 0;
    open STDIN,  '/dev/null'   or die "Can't read /dev/null: $!";
    open STDOUT, '>>/dev/null' or die "Can't write to /dev/null: $!";
    open STDERR, '>>/dev/null' or die "Can't write to /dev/null: $!";
    defined( my $pid = fork ) or die "Can't fork: $!";
    exit if $pid;
     
    # dissociate this process from the controlling terminal that started it and stop being part
    # of whatever process group this process was a part of.
    POSIX::setsid() or die "Can't start a new session.";
     
    # callback signal handler for signals.
    $SIG{INT} = $SIG{TERM} = $SIG{HUP} = \&signalHandler;
    $SIG{PIPE} = 'ignore';
     
    # create pid file in /var/run/
    my $pidfile = File::Pid->new( { file => $pidFile, } );
     
    $pidfile->write or die "Can't write PID file, /dev/null: $!";
     
    # turn on logging
    if ($logging) {
    	open LOG, ">>$logFile";
    	select((select(LOG), $|=1)[0]); # make the log file "hot" - turn off buffering
    }
     
    # "infinite" loop where some useful process happens
    until ($dieNow) {
    	sleep($sleepMainLoop);
     
    	# TODO: put your custom code here!
    	# do something
     
    	# logEntry("log something"); # use this to log whatever you need to
    }
     
    # add a line to the log file
    sub logEntry {
    	my ($logText) = @_;
    	my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime(time);
    	my $dateTime = sprintf "%4d-%02d-%02d %02d:%02d:%02d", $year + 1900, $mon + 1, $mday, $hour, $min, $sec;
    	if ($logging) {
    		print LOG "$dateTime $logText\n";
    	}
    }
     
    # catch signals and end the program if one is caught.
    sub signalHandler {
    	$dieNow = 1;    # this will cause the "infinite loop" to exit
    }
     
    # do this stuff when exit() is called.
    END {
    	if ($logging) { close LOG }
    	$pidfile->remove if defined $pidfile;
    }
    

    UPDATE:

    Looks like there’s a page in French about this post: http://www.duncane.net/2011/02/25/perl-daemon/. Merci Duncane!

    Passive Mode (PASV) FTP client on an Ubuntu server

    If you need to communicate from your Ubuntu server to an FTP server that requires passive mode, there is a problem: your firewall likely blocks communication. Using an FTP client manually, you can probably connect with the server, but not list or transfer files!

    The reasons for this are straightforward, your system is operating exactly as it is configured to. The explanation requires a little understanding of FTP and firewalls.

    Most IP protocols use one port on the local machine and port on the server being connected to. FTP happens to use two ports instead of one. When negotiating a connection, the two computers negotiate which port to send data to. This brings us to an important difference between the two modes:

    • In active mode FTP, the client sends the server a PORT command, which tells the server client which port to use for data. The client connects to the server.
    • In passive more, the client sends the server a PASV command that asks for a server port to use for data. The server connects with the client.

    The tricky bits concern this second port that is negotiated. This port is not a fixed number, it is a dynamically allocated port above 1023. The port number is encoded in a packet as two numbers that need to be multiplied together to get the port number. The firewalls involved need to be smart enough to recognize the FTP negotiation and extract this data from the data, open that specified port and keep it open during the FTP session.

    In active mode, this tricky bit is handled by the server, but in passive mode, it is handled by the client’s firewall! Ah ha! So, you need to configure your firewall to be smart about address translation and FTP connections.

    Configuring the firewall

    You will need to activate a couple of kernel modules for iptables. These will turn on NAT (network address translation) for FTP and FTP connection tracking. As iptables/Netfilter is part of the kernel, we need to use modprobe to add these to the current session and also make changes to /etc/modules so the modules will load next time the server is rebooted.

    First, use modprobe to use these two modules now:

    sudo modprobe ip_nat_ftp
    sudo modprobe ip_conntrack_ftp
    

    Then, modify /etc/modules so the modules will load on next reboot:

    sudo vi /etc/modules
    

    Add these lines:

    ip_nat_ftp
    ip_conntrack_ftp
    

    With these two modules, you should now be able to use passive mode from an FTP client on your Ubuntu server.

    Securing an Ubuntu Server

    For more current information, see: http://www.adminbuntu.com/security

    Security is relative. Will these steps make your server “secure”? It will be more secure than it was before. And more secure than most servers. Your server will not be “low hanging fruit”. Security is an on-going process. It includes settings, practices and procedures. Make it your business to regularly read about security and to understand the concepts and our system. Paranoia is useful with regard to server security.

    I’ve tested what is presented here in Ubuntu Server 10.04 (Lucid) and 10.10 (Maverick). If you want to harden your new Ubuntu server, this is a good start.

    Ubuntu server is well designed, regularly updated and relatively secure. The Ubuntu Security Team manifests an onging effort to keep Ubuntu secure. Regular security updates are available and easy to implement.

    • No open ports
    • Role-based administration
    • No X server
    • Security updates
    • Kernel and compiler hardening

    In this post, we are going to meet the security challenge in with multi-pronged effort that will include: system analysis, changing settings for additional hardening against attack, installing a firewall maintenance system, scanning for rootkits, and offering a regular maintenance regimen.

    • Change settings for increased security
    • Implement UFW, the uncomplicated firewall
    • Use denyhosts to automatically blacklist attackers
    • Scan the system for vulnerabilities with Tiger
    • Detect attempted intrusions with psad
    • Install nmap and scan the system for open ports
    • Check the system for rootkits with chkrootkit
    • Monitor logs

    Change settings for increased security

    see also: https://help.ubuntu.com/community/StricterDefaults

    Secure shared memory

    /dev/shm can be used in an attack against a running service, such as httpd. Modify /etc/fstab to make it more secure.

    sudo vi /etc/fstab
    

    Add this line:

    tmpfs     /dev/shm     tmpfs     defaults,noexec,nosuid     0     0
    

    Disable root SSH login

    The root account is disabled by default in Ubuntu. If you installed Ubuntu on Slicehost or Linode, root is enabled. In any case, it is a good idea to disable root SSH access. Edit /etc/ssh/sshd_config and set PermitRootLogin to no.

    sudo vi /etc/ssh/sshd_config
    

    Change PermitRootLogin to no:

    PermitRootLogin no
    

    Of course, if you access your server via SSH, you should make sure you have sudo working for your user before disabling SSH root access.

    Only allow admin users to use su

    This helps prevent privilege escalation.

    By default, Ubuntu does not have an admin group. Create an admin group:

    sudo groupadd admin
    

    Add yourself to the admin group:

    sudo usermod -a -G admin andrew
    

    Restrict access to /bin/su to admin group members:

    sudo dpkg-statoverride --update --add root admin 4750 /bin/su
    

    Check permissions for /bin/su with:

    ls -lh /bin/su
    

    …and see the following:

    -rwsr-x--- 1 root admin 31K 2010-01-26 17:09 /bin/su
    

    Do not permit source routing of incoming packets

    see also: http://www.cromwell-intl.com/security/security-stack-hardening.html

    sudo sysctl -w net.ipv4.conf.all.accept_source_route=0
    sudo sysctl ­-w net.ipv4.conf.default.accept_source_route=0
    

    Don’t allow system users to access an FTP server

    This is only needed is ftpd is installed and running. Only if you’ve installed ftpd. However, it is Ok to do this anyway and it will remove a FAIL from the tiger report.

    SFTP is probably better than FTP, if it is usable for your files transfer needs.

    see ftpusers manual: http://manpages.ubuntu.com/manpages/lucid/man5/ftpusers.5.html

    Edit /etc/ftpusers:

    sudo vi /etc/ftpusers
    

    Add system users to deny use of ftpd:

    backup
    bin
    daemon
    games
    gnats
    irc
    libuuid
    list
    lp
    mail
    man
    mysql
    news
    ntp
    postfix
    proxy
    sshd
    sync
    sys
    syslog
    uucp
    www-data
    

    UFW: basic firewall

    previous post: Ubuntu UFW Uncomplicated Firewall Examples

    community documentation: https://help.ubuntu.com/community/UFW

    server guide: https://help.ubuntu.com/10.04/serverguide/C/firewall.html

    ufw manual: http://manpages.ubuntu.com/manpages/lucid/en/man8/ufw.8.html

    project wiki: https://wiki.ubuntu.com/UncomplicatedFirewall

    nice article: http://savvyadmin.com/ubuntus-ufw/

    UFW (Uncomplicated Firewall) provides an easy to understand interface to control iptables (iptables conteol Netfilter, which is built into the kernel). Will just a few commands, your server can control access. Checking status is also easy.

    UFW (uncomplicated firewall) is a simple interface used to configure iptables.

    Install and enable Uncomplicated Firewall:

    sudo aptitude install -y ufw
    sudo ufw enable
    

    Display available UFW commands:

    sudo ufw show
    

    Display UFW configuration:

    sudo ufw status
    

    Allow SSH and HTTP access to the Apache server:

    sudo ufw allow ssh
    sudo ufw allow http
    

    In the above example, ports for OpenSSH and Apache were opened by service name (“ssh” and “http”). You can use a port number instead of the service name (like “80” instead of “http”).

    See services running and which names to use:

    The practice here is to open only ports that you use – ports that use a service that have a service running. To see a list of services that you have running for which you might want to open ports for:

    sudo ufw app list
    

    To see a list of services that UFW uses (like in the “sudo ufw allow ssh” example, above):

    less /etc/services
    

    Denyhosts: avoid SSH attacks

    project: http://denyhosts.sourceforge.net/

    Looking at /var/log/auth.log on servers that I manage shows a steady streams of attacks on SSH. I am countering these attacks in a number of ways, starting with denyhosts.

    Denyhosts periodically scans /var/log/auth.log for repeated failures to access the system via SSH. It then adds these offenders to /etc/hosts.deny. See the project page for details.

    sudo aptitude -y install denyhosts
    

    That does it – the rest is automatic. You can see the IP addresses added to /etc/hosts.deny with:

    sudo less /etc/hosts.deny
    

    Tiger: security system scanner

    project: http://www.nongnu.org/tiger/

    Tiger creates an automated security audit by analyzing files and settings on the system and creating a report listing what has been analyzed and listing warning, alerts and failures.

    The tiger command creates a report of potential security problems in /var/log/tiger. The use the tigexp command to look up the resulting codes generated for a detailed explanation and what to do to make the system more secure. The problems tiger considers most serious are marked with FAIL.

    It has been a while since Tiger has been updated. It still produces a useful report.

    Install tiger:

    sudo aptitude -y install tiger
    

    Run tiger to create a report of security issues.

    sudo tiger
    

    Use less to view the most recent tiger report:

    sudo -i
    less /var/log/tiger/`ls -t1 /var/log/tiger | head -1`
    exit
    

    Use tigexp to list explanations for FAIL codes:

    tigexp dev002f
    

    Google is also helpful, naturally.

    Ignore these:

    --FAIL-- [dev002f] /dev/fuse has world permissions
    --FAIL-- [logf005f] Log file /var/log/btmp permission should be 660
    

    Changing permissions for these could cause problems.

    Detect attempted intrusions with psad

    project: http://www.cipherdyne.org/psad/

    Psad is a collection of lightweight daemons that log attempted intrusions, in particular monitoring iptables.

    Installation:

    sudo aptitude -y install psad
    

    The daemons will run automatically.

    To check current status:

    sudo psad -S
    

    You can modify psad settings to e-mail the admin in the event of intrusion detection.

    Nmap: port scanning

    project: http://nmap.org/

    This allows you to see which ports are open, verifying that UFW/iptables is working correctly.

    Installing nmap:

    sudo aptitude install -y nmap
    

    Port scanning:

    nmap -v -sT localhost
    

    SYN Scanning:

    sudo nmap -v -sS localhost
    

    scan type explanations: http://nmap.org/book/man-port-scanning-techniques.html

    Chkrootkit: check for rootkit presence

    project: http://www.chkrootkit.org/

    Chkrootkit scans the system for evidence that a rootkit has been installed.

    This is a confidence test to be used to test whether your system has been compromised. In a perfect world you would not need this…but in this world, it is good to run periodically.

    Installing chkrootkit:

    sudo aptitude install -y chkrootkit
    

    Running chkrootkit:

    sudo chkrootkit
    

    LogWatch

    Ubuntu community documentation: https://help.ubuntu.com/community/Logwatch

    The most detailed and informative logs in the world are useless if no one looks at them. Logwatch winnows the deluge to a succinct report…which you will look at. Even so, familiarize yourself with your system’s logs and review them on a regular basis. A daily logwatch habit would be a good start.

    Installation:

    sudo aptitude -y install logwatch
    

    Usage:

    sudo logwatch | less
    

    Ongoing maintenance

    Your server is now more secure. Once a week, perform on-going maintenance.

    Updating software:

    sudo aptitude update
    sudo aptitude safe-upgrade
    

    The safe-upgrade action is preferred by me because it does not upgrade packages that rely on dependencies that have not been upgraded to required levels.

    see: http://wiki.debian.org/Aptitude

    Or, you could set-up automatic security updates, if you cannot do the weekly maintenance. This is not a perfect solution because an administrator is not monitoring what is being updated and testing after updates. see: https://help.ubuntu.com/10.04/serverguide/C/automatic-updates.html

    Check for attempted instrusions:

    sudo psad -S
    

    UPDATED: Analyze system with tiger. Because the tiger reports in /var/log/tiger/are owned by root, run these commands one at a time. (This solves a problem some people were having with permissions.)

    sudo -i
    tiger
    grep FAIL /var/log/tiger/`ls -t1 /var/log/tiger | head -1`
    exit
    

    In the above, FAILs are pulled from the newest report file with grep. The ls clause in backticks gives grep the newest file in the directory. The sudo -i command allows you to run multiple commands as root, ending with exit.

    Use tigexp to list explanations for FAIL codes:

    tigexp dev002f
    

    Scan ports with nmap:

    sudo nmap -v -sS localhost
    

    Check for rootkits

    sudo chkrootkit
    

    Look at logs:

    sudo logwatch | less
    

    Keep up with trends

    visit: http://www.linuxsecurity.com/

    Elsewhere

    http://www.itsecurity.com/features/ubuntu-secure-install-resource/

    http://www.cyberciti.biz/tips/linux-security.html

    Ubuntu command line: see PDF of a man page

    So, you use the command line. And, you’d like to look at a command’s manual page.

    Wouldn’t it be handy to open the page into another window, nicely formatted, all typeset and neat? That is exactly what this little script will do.

    I keep my personal scripts and executables in ~/bin (the bin directory inside my home directory). My ~.profile file in Ubuntu already had that in the path, if ~/bin exists.

    If needed, create ~/bin:

    mkdir ~/bin
    

    Create a shell script called ~/bin/gman with these lines:

    #!/bin/sh -e
    man -t $1 | ps2pdf - > "/tmp/$1.man.pdf"
    gnome-open "/tmp/$1.man.pdf"
    

    Make the shell script executable with:

    chmod +x ~/bin/gman
    

    If needed, exit your shell and open it back up again. This would be to ensure that ~/bin is detected and added to your PATH.

    Then, to use it, just type gman and the command you are interested in, ie. grep.

    gman grep
    

    This will show you a beautiful, formatted document:

    List of installed packages on Ubuntu

    This lists packages added after Ubuntu was installed.

    When I upgrade from one version of Ubuntu to another, I do a fresh install. I have a procedure on my personal wiki with all of the steps to move in. Part of that is restoring data from backup, Part is restoring or recreating configuration files.

    But what about software packages? There is a baseline of packages that I use all the time, but I also add packages ad hoc as required. It is convenient to get a list of what is installed.

    Here’s a one-liner to list installed packages on Ubuntu:

    cat /var/log/installer/initial-status.gz | gzip -d | grep '^Package:' | awk '{ print $2}' > tmp.txt && aptitude search -F %p '~i!~M' | awk '{ print $1}' | grep -v -F -f tmp.txt && rm tmp.txt
    

    The list that is returned is pretty good. You can save the output to a text file and edit that file to get the list of packages you want to install on the new system.

    Here’s how to output to packages.txt:

    cat /var/log/installer/initial-status.gz | gzip -d | grep '^Package:' | awk '{ print $2}' > tmp.txt && aptitude search -F %p '~i!~M' | awk '{ print $1}' | grep -v -F -f tmp.txt > packages.txt && rm tmp.txt
    

    As you can see, it greps out a list of packages included in the original install, grabs the second field with awk and stuffs the result in a temporary file. Then a list of installed packages is generated, again using awk to parse out the package name field. Then grep is used to compare thee two lists and output just the packages installed after initial installation.

    Here is what I have installed:

    apache2
    apache2-utils
    apache2.2-common
    audacity
    bluefish
    build-essential
    figlet
    flashplugin-installer
    gimp
    gnome-do
    google-chrome-beta
    inkscape
    keepassx
    libapache2-mod-php5
    mailutils
    mysql-server
    ninvaders
    nvidia-current
    padre
    php5
    php5-cli
    php5-common
    php5-curl
    php5-dev
    php5-gd
    php5-imagick
    php5-mcrypt
    php5-memcache
    php5-mysql
    php5-pspell
    php5-snmp
    php5-sqlite
    php5-xmlrpc
    php5-xsl
    phpmyadmin
    skype
    vim
    vlc
    winff
    xtightvncviewer
    

    So, this is a list of 40 packages installed on my machine, out of a total of 1560 packages returned by “dpkg –list” (thanks, Mark!). Just those I installed, not everything including all of baseline Ubuntu.

    Interactive, Full Screen Aptitude

    Debian’s aptitude command is, according to their documentation, the preferred program for package management. I use Ubuntu for my work machine and servers and have found aptitude to be easy to understand and use, as well as reliable.

    see also: Aptitude vs Apt-get Comparison

    Unlike apt-get, aptitude offers a full-screen (or full-window more likely) mode that can be used to interactively manage packages on a Debian or Ubuntu system. As is often the case, a combination of command line operation and an interactive utility prove to be very attractive and useful.

    You might use command line mode most often, to install, remove and upgrade packages, and sometimes use the interactive mode to check the status of your system and mark packages to manage how particular packages are dealt with.

    To start in interactive mode, simply enter aptitide, with no subcommands:

    aptitude
    


    note: You do not need to sudo aptitude because, in interactive mode, it will prompt you and run sudo itself, if needed. Oh, the convenience!

    Right away, you will see the interactive mode user interface:

    Drop-down menus

    The interactive interface includes drop-down menus at the top of the display. These are accessed with control-r or F10. Many of the of the commands available have a shortcut command key available, listed on the right side of the menu choice.

    A full list of menu commands can be found here: http://algebraicthunk.net/~dburrows/projects/aptitude/doc/en/ch02s01s02.html

    Package categories

    Packages are shown in groups, organized by the following categories:

    category explanation
    New packages Packages that have been added since the list of new packages has been cleared (with the f command).
    Installed packages Packages that are installed on the computer.
    Not installed packages Packages that are not installed on the computer.
    Obsolete and locally created packages Packages that are installed on the computer, but not available from an apt source.
    Virtual packages Pseudonymous for other packages for compatibility or convenience.
    Tasks Groups of packages that offer an easy means to install groups of packages for some purpose.

    While the user interface is pretty well designed, it can be tricky to use because a variety of single letter codes are displayed and commands are entered via arcane single letter commands.

    Navigation command keys

    key action
    F10 or control-t Show the drop-down menu
    ? Show help
    Arrow keys Navigation
    Enter key Select
    +/- keys Mark a package to be installed, updated or removed
    g key* Go forward: preview/confirm actions
    q key* Go back: quit

    * The q command quits the current operation and goes back to the previous mode. When at the top level, the q command quits the program. In general, the g goes forward and the q goes back.

    Example workflow

    • press u to update local list of available packages
    • press U to mark upgradable packages
    • press g to review pending actions (modify if desired*)
    • press g (again) to start the process

    Press g twice? Yes, the first time results in a list of packages that will be processed and the second time completes the action. This gives you a chance to review the changes before proceeding.

    If you’ve erred, you can select ‘Cancel pending actions’ in the ‘Actions’ menu.

    *When reviewing pending actions:

    • a explicitly accepts an action (press it again to un-accept).
    • r rejects a pending action.
    • g again goes ahead with pending actions.

    Common actions

    Use control-t for the drop-down menu to see all available actions. All of these commands are found in the Actions drop-down menu.

    shortcut key item in Action menu action
    g Install/remove packages Show preview, or if preview visiable, perform actions
    u Update package list Update local package list from Internet sources
    U Mark Upgradable Flag upgradable packages
    f Forget new packages Clear “new” packages list
    Q Quit Quit aptitude

    Common package commands

    These commands are found in the Package drop-down menu.

    shortcut key item in Action menu action
    + Install Marks package for installation
    - Remove Marks package for removal
    i Cycle Package Information Changes information displayed for selected package

    Current state flags

    These flags appear in the first column of a table of packages when you’ve drilled down in one of the categories.

    flag meaning
    i Package is installed and all its dependencies are satisfied.
    c Package was removed, but its configuration files are still present.
    p Package and all its configuration files were removed, or the package was never installed.
    v Package is virtual.
    B Package has broken dependencies.
    u Package has been unpacked but not configured.
    C Half-configured: the package’s configuration was interrupted.
    H Half-installed: the package’s installation was interrupted.

    Position of the Current State and Action flags in the Package lists:

    Action flags

    These flags appear right after the current state flag in column one of displayed packages when you’ve given a command to make an alteration. One or more action flags will appear.

    flag meaning
    i Package will be installed.
    u Package will be upgraded.
    d Package will be deleted: it will be removed, but its configuration files will remain on the system.
    p Package will be purged: it and its configuration files will be removed.
    h Package will be held back: it will be kept at its current version, even if a newer version becomes available, until the hold is cancelled.
    F An upgrade of the package has been forbidden.
    r Package will be reinstalled.
    B Package is broken: some of its dependencies will not be satisfied. aptitude will not allow you to install, remove, or upgrade anything while you have broken packages.

    Sources

    http://wiki.debian.org/Aptitude

    http://algebraicthunk.net/~dburrows/projects/aptitude/doc/en/

    Aptitude vs Apt-get Comparison

    see also: Interactive, Full Screen Aptitude

    One of the many attractive features of Ubuntu and Debian Linux is the package management system. Coming from other operating systems and other distributions makes the discovery of the Advanced Packaging Tool, APT, a source of pleasure and delight. Here is a system that solves dependency hell, makes keeping soft up to date easy and facilitates simple installation and removal of software packages.

    The mainstay of this system has been apt-get, an extremely useful and versatile program that has been the heart of the Debian APT system. A great and useful program, but not perfect, the newer program aptitude is the result of an effort to improve on apt-get. In addition to a cleaner command line interface, aptitude offers a fullscreen character-based UI and more complete tracking of what has been installed and interdependencies.

    aptitude is a newer and improved replacement for apt-get

    These two programs provides higher level capabilities compared to dpkg, the Debian low-level package management utility. They offer an interface to package repositories and provide relief from dependency hell.

    feature apt-get command aptitude command
    fullscreen interface N/A aptitude
    install package apt-get install ‘pkgname’ aptitude install ‘pkgname’
    remove package apt-get remove ‘pkgname’ aptitude remove ‘pkgname’
    purge package (removes package
    and installation files)
    apt-get –purge remove ‘pkgname’ aptitude purge ‘pkgname’
    upgrade installed packages apt-get upgrade aptitude upgrade
    upgrade installed packages
    even if other packages
    must be removed
    apt-get dist-upgrade aptitude dist-upgrade
    show package details (apt-cache show ‘pkgname’) aptitude show ‘pkgname’
    search for packages (apt-file ‘searchpattern’) aptitude search ‘searchpattern’
    delete installation files apt-get clean aptitude clean
    delete obsolete installation files apt-get autoclean aptitude autoclean
    update local cache of
    available packages
    apt-get update aptitude update
    Show package details apt-get show ‘pkgname’ aptitude show ‘pkgname’
    Retain the current version
    of a package going forward
    N/A aptitude hold pkgname
    Clear the hold on a
    package from
    ‘aptitude hold pkgname’ command
    N/A aptitude unhold pkgname
    List reverse dependencies apt-cache rdepends packagename aptitude -D packagename
    super cow powers apt-get moo aptitude -v[v[v[v[v]]]] moo

    As you can see, in addition to a more complete dependency tracking solution, aptitude also provides a (well designed) full-screen interface and cleaned-up command line syntax. Also, contrary to reports elsewhere, aptitude offers the ever important super cow functionality, with sarcasm added (Easter egg).

    When installing a package, aptitude will show which other packages, though not required, are recommended or suggested, so you can decide whether or not to also install those.

    Because aptitude more completely track dependencies, if you use it exclusively your will not need to use deborphan or debfoster.

    Apt-get does not remove packages it installed as dependencies when the package you specified to be installed is removed. Aptitude does remove unneeded dependencies. This basic difference can cause problems when switching from apt-get to aptitude.

    Don’t use both apt-get and aptitude interchangeably

    Use one or the other. I’ve switched completely from using apt-get to using aptitude. If you do use apt-get, afterward run aptitude and fix any problems detected by first pressing g, which will show broken dependencies and packages that aptitude would remove. If you want to retain the packages that would be removed, arrow down to the header of that category and press the + key. Pressing g again will update your system with your indicated changes.

    There is a difference in how aptitude keeps track of installed packages and their dependencies compared to apt-get. Aptitude is more precise is tracking these dependencies. Consequently, when you switch from using apt-get to aptitude, the first time you use aptitude you can run into problems because it can remove needed packages because it does not know that they are needed. This can be resolved through aptitudes’s full-screen interface or by entering the following command:

    sudo aptitude keep-all
    

    This will cause aptitude to retain all current packages going forward.

    Aptitude’s full-screen interface

    As displayed in the comparison table, aptitude offers a character-based full-screen interactive user interface if aptitude is entered as a command with no options.

    The interface is very well designed and implemented, logical and easy to use after a few minutes of examination and trial. It has a drop-down menu system, which I found to work well using a mouse. The help system can be access via the drop-down menu or by pressing the ‘?’ key. A plethora of options are presented, most of which will make sense from what you’ve seen in the command line operation of the program. Pressing ‘q’ exits help…indeed pressing ‘q’ is the way to exit other modes in the program and exiting the program itself.

    Other thoughts

    I’ve found aptitude to be so well done, that I use it for all my package management needs. I use Synaptic to help search for software sometimes, whether for my local machine or a server. Then, I install it with aptitude…in any case Synaptic would be pretty useless over SSH to a server.

    Some pages I found helpful: http://pthree.org/2007/08/12/aptitude-vs-apt-get/ and http://www.garfieldtech.com/blog/your-debian-aptitude.

    Mix in some CLI fun on your server

    This post is directed at Ubuntu and Debian server admins. As all work and no play makes Jack a dull boy, it is imperative that you immediately make your server more fun. If you do not get a little smile when you log into your server via SSH, then something is terribly wrong! Avoid dullness by all means.

    Here I will show how to add and use figlet, fortune, cowsay and xmlstarlet to have big banners, random quotes’n’quips, talking cows and word of the day appear when using SSH to access your server.

    FIGlet is a program for making large letters out of ordinary text.

    FIGlet project: http://www.figlet.org/

    Fortune is a simple program that displays a random message from a database of quotations.

    Cowsay is a filter that takes text and displays a cow saying it.

    Cowsay project page: http://www.nog.net/~tony/warez/cowsay.shtml

    Cowsay article: http://linuxgazette.net/issue67/orr.html

    XMLStarlet is a set of command line utilities (tools) which can be used to transform, query, validate, and edit XML documents and files using simple set of shell commands in similar way it is done for plain text files using UNIX grep, sed, awk, diff, patch, join, etc commands.

    XMLSartlet project page: http://xmlstar.sourceforge.net/overview.php

    Adding Universe and Multiverse Repository in Ubuntu

    These packages are in the Universe and Multiverse repositories. If you need to add these repositories, just un-remark the pertinent lines in /etc/apt/sources.list and run aptitude update:

    First, make a backup of the original /etc/apt/sources.list file.

    sudo cp /etc/apt/sources.list /etc/apt/sources.list.original
    

    Edit /etc/apt/sources.list:

    sudo vi /etc/apt/sources.list
    

    Un-remark the universe and multiverse lines (remove the leading # character) so the lines look something like this:

    deb http://us.archive.ubuntu.com/ubuntu/ lucid universe
    deb-src http://us.archive.ubuntu.com/ubuntu/ lucid universe
    deb http://us.archive.ubuntu.com/ubuntu/ lucid-updates universe
    deb-src http://us.archive.ubuntu.com/ubuntu/ lucid-updates universe
    
    deb http://us.archive.ubuntu.com/ubuntu/ lucid multiverse
    deb-src http://us.archive.ubuntu.com/ubuntu/ lucid multiverse
    deb http://us.archive.ubuntu.com/ubuntu/ lucid-updates multiverse
    deb-src http://us.archive.ubuntu.com/ubuntu/ lucid-updates multiverse
    

    Finish by retrieving the updated package lists to your system with:

    sudo aptitude update
    

    By the way, to search for packages in the Ubuntu packages repositories, visit:

    
    http://packages.ubuntu.com/
    
    

    Figlet

    Figlet creates character graphic block letter banners. Thus:

    media2
    

    becomes…

                           ___      ___ 
       ____ ___  ___  ____/ (_)___ |__ \
      / __ `__ \/ _ \/ __  / / __ `/_/ /
     / / / / / /  __/ /_/ / / /_/ / __/ 
    /_/ /_/ /_/\___/\__,_/_/\__,_/____/ 
    

    Install Figlet

    On an Unbuntu/Debian system, install figlet like this:

    sudo aptitude install figlet
    

    The following will create the media2 banner, as above:

    figlet -f slant media2
    

    Modifying Message of the Day

    Once you are satisfied with the output of figlet, you can have your character graphics banner appear whenever a user uses SSH to access the server, just modify the /etc/motd.tail file with:

    sudo figlet -f slant media2 >>/etc/motd.tail
    

    Bingo, whenever a user logs into the media2 server then will see your nifty banner! A little dullness have been bannished.

    Installing xmlstarlet, cowsay and fortune

    These are the commands we’ll be using to offer some fresh content on every login.

    sudo aptitude install xmlstarlet cowsay fortune
    

    Putting it all together

    Modify the /etc/bash.bashrc file. This affects all users that use bash as their default shell.

    fortune -a | cowsay -f $(ls /usr/share/cowsay/cows/ | shuf | head -n1)
    
    echo -n "word of the day: "
    /usr/bin/xmlstarlet sel --net -t -m "/rss/channel/item/description" -v "." "http://dictionary.reference.com/wordoftheday/wotd.rss"
    

    The fortune command pops out a random quip, piped into cowsay, which is configured here with -f to use a random character graphic image. Then, word of the day is sourced from an RSS feed with xmlstarlet.

    With everything, here is an example of what will appear when your SSH into your server:

    Linux ijuki 2.6.32-x86_64-somewhere #1 SMP Sat Dec 5 16:55:26 UTC 2009 x86_64
    
                           ___      ___ 
       ____ ___  ___  ____/ (_)___ |__ \
      / __ `__ \/ _ \/ __  / / __ `/_/ /
     / / / / / /  __/ /_/ / / /_/ / __/ 
    /_/ /_/ /_/\___/\__,_/_/\__,_/____/ 
    
    Last login: Sun May  2 18:39:14 2010 from 00.000.000.000
     ________________________________________
    / Everything that you know is wrong, but \
    \ you can be straightened out.           /
     ----------------------------------------
           \    ____
            \  /    \
              | ^__^ |
              | (oo) |______
              | (__) |      )\/\
               \____/|----w |
                    ||     ||
    
    	         Moofasa
    word of the day: sesquipedalianism: given to using long words.