Perl script to get video details

This is a command line Perl script I made a while back to simply show details about a specified video. It is tested on Ubuntu.

You can modify it for your specific requirements or grab some code for another use.

#!/usr/bin/perl -w
use strict;
use warnings;
 
use IPC::Open3;
 
# example
my $filename  = $ARGV[0];
my %videoInfo = videoInfo($filename);
print "duration: " . $videoInfo{'duration'} . "\n";
print "durationsecs: " . $videoInfo{'durationsecs'} . "\n";
print "bitrate: " . $videoInfo{'bitrate'} . "\n";
print "vcodec: " . $videoInfo{'vcodec'} . "\n";
print "vformat: " . $videoInfo{'vformat'} . "\n";
print "acodec: " . $videoInfo{'acodec'} . "\n";
print "asamplerate: " . $videoInfo{'asamplerate'} . "\n";
print "achannels: " . $videoInfo{'achannels'} . "\n";
 
#
# returns media information in a hash
sub videoInfo {
	# ffmpeg command
	my $ffmpeg = '/usr/local/bin/ffmpeg';
 
	my %finfo = (
				  'duration'     => "00:00:00.00",
				  'durationsecs' => "0",
				  'bitrate'      => "0",
				  'vcodec'       => "",
				  'vformat'      => "",
				  'acodec'       => "",
				  'asamplerate'   => "0",
				  'achannels'       => "0", 
	);
 
	my $file = shift;
 
	# escaping characters
	$file =~ s/(\W)/\\$1/g;
 
	open3( "/dev/null", \*ERPH, "$ffmpeg -i $file" ) or die "can't run $ffmpeg\n";
	my @res = ;
 
	# parse ffmpeg output
	foreach (@res) {
        print;
 
		# duration
		if (m!Duration: ([0-9][0-9]:[0-9][0-9]:[0-9][0-9].[0-9][0-9])!) {
			$finfo{'duration'} = $1;
		}
 
		# bitrate
		if (m!bitrate: (\d*) kb/s!) {
			$finfo{'bitrate'} = $1;
		}
 
		# vcodec and vformat
		if (/Video: (\w*), (\w*),/) {
			$finfo{'vcodec'}  = $1;
			$finfo{'vformat'} = $2;
		}
 
        # Stream #0.1(und): Audio: aac, 48000 Hz, 1 channels, s16, 64 kb/s
 
		# acodec, samplerate, stereo and audiorate
		if (m!Audio: (\w*), (\d*) Hz, (\d*)!) {
			$finfo{'acodec'}     = $1;
			$finfo{'asamplerate'} = $2;
			$finfo{'achannels'}     = $3;
		}
	}
 
	my $tenths  = substr( $finfo{'duration'}, 9, 2 );
	my $seconds = substr( $finfo{'duration'}, 6, 2 );
	my $minutes = substr( $finfo{'duration'}, 3, 2 );
	my $hours   = substr( $finfo{'duration'}, 0, 2 );
	$finfo{'durationsecs'} = ( $tenths * .01 ) + $seconds + ( $minutes * 60 ) + ( $hours * 360 );
 
	return %finfo;
}

Create a Drupal site on an Ubuntu Server

This will help you quickly install a Drupal site on an Ubuntu server.

This is my script for this purpose. Use it as you will. This procedure assumes that you have already installed Apache, MySQL, phpMyAdmin and PHP. It also assumes that you are familiar with everything. This is probably not a beginner’s guide, but an aid to someone more experienced that wants to install Drupal quickly.

Create and run a shell script to create your site

Use this template to create a shell script that will create your site. Create a text file from the following template called “make_dru.sh” (in your home directory is fine). Modify lines 7 to 17 for your installation. Then use “chmod +x make_dru.sh” and execute the script with “sudo ./make_dru.sh”.

#!/bin/bash
#
# make drupal project
#

# 
OWNER="andrew" 			# change to your username
GROUP="staff"			# change to your username or the group that will work on the site
DRUPAL_URL="http://ftp.drupal.org/files/projects/drupal-6.17.tar.gz" # update to the current version, if there is one
SITESROOT="/usr/local/var/www"  # change to the directory where you put website directories
URL="www.sitename.com"		# change to your site's address

# database.php parameters
DBUSER="sitename"		# change to something meaningful
DBPWD="passwordgoeshere"		# change to a long password
DBHOST="localhost"		# leave alone
DBDATABASE=$DBUSER		# leave alone
 
#
# make sure this is run as root
#
if [[ $UID -ne 0 ]]; then
    echo "Not running as root"
    exit
fi
 
if [ ! -d $SITESROOT ]; then
	echo "$SITESROOT directory does not exist"
	exit
fi
 
#
# check for existence of the specified user owner
#
/bin/egrep  -i "^$OWNER" /etc/passwd > /dev/null
if [ ! $? -eq 0 ]; then
   echo "User $OWNER, which is specified as the owner, does not exist in /etc/passwd"
   exit
fi
 
#
# check for existence of the specified group
#
/bin/egrep -i "^$GROUP" /etc/group > /dev/null
if [ ! $? -eq 0 ]; then
   echo "$GROUP, which is specified as the group to use, does not exist in /etc/group"
   exit
fi
 
#
# change to the siteroot directory
#
cd $SITESROOT
 
if [ -d $URL ]; then
	echo "$SITESROOT/$URL directory already exists"
	exit
fi 
 
mkdir $URL
chown $OWNER:$GROUP $URL
cd $URL
 
#
# get Drupal
#
echo 'get Drupal'
wget -O drupal.tgz $DRUPAL_URL
tar -xzvf drupal.tgz
find . -maxdepth 1 -type d -name 'drupal*' -exec mv {} public \;
sudo chown -R $OWNER:$GROUP public
find . -type d -exec chmod g+ws {} \;
 
#
# temporarily allow write to settings
#
cp public/sites/default/default.settings.php public/sites/default/settings.php
chmod a+w public/sites/default/settings.php
chmod a+w public/sites/default
 
#
# logs
#
echo "set up logs"
mkdir logs
sudo chown $OWNER:www-data logs
chmod g+ws logs
touch logs/error.log
touch logs/combined.log
 
#
# create Apache virtual site
#
echo creating Apache virtual site file in /etc/apache2/sites-available
echo "
	ServerName $URL
	DocumentRoot $SITESROOT/$URL/public
	DirectoryIndex index.php
	LogLevel warn
	ErrorLog $SITESROOT/$URL/logs/error.log
	CustomLog $SITESROOT/$URL/logs/combined.log combined

" > /etc/apache2/sites-available/$URL
 
#
# finish up
#
echo -e "Drupal website was created in: $SITESROOT/$URL\n"
echo -e "To finish, do the following:\n"
echo "1. Enable site with: sudo a2ensite $URL"
echo "2. Restart Apache with: sudo /etc/init.d/apache2 restart"
echo "3. Make a crontab entry ('sudo crontab -e') like: 0 * * * * wget -O - -q -t 1 http://$URL/cron.php"
echo -e "4. Create a MySQL database with:\n"
echo -e "\t     user: $DBUSER"
echo -e "\t      pwd: $DBPWD"
echo -e "\t     host: $DBHOST"
echo -e "\t database: $DBDATABASE"
echo "5. Open the site in your browser and finish the set-up."
echo "6. Remove write privileges with: sudo chmod og-w public/sites/default/settings.php"
echo "   ..and: sudo chmod og-w public/sites/default"

Create the database and user with phpMyAdmin

  • click the “Privileges” tab and add a new user
  • in the “User name” field, enter the sitename that you entered for DBUSER in the script
  • in the Host field, select “Local”
  • enter or generate a password – make it a long one, like 20 characters
  • in the Database for User list, select “Create database with same name and grant all privileges”
  • click Go

Enable the new Drupal website

This will tell Apache to enable the virtual site that was made in the script and then restart Apache.

sudo a2ensite www.sitename.com
sudo /etc/init.d/apache2 restart

Configure the site

Open site in browser and configure it

Remove temporary write privileges

During site creation, write privileges were needed, after site set-up, remove them. In a shell, while in the sites directory (where the public directory is run these commands:

sudo chmod og-w public/sites/default/settings.php
sudo chmod og-w public/sites/default

Linux command line language translator

I wanted a command language translator that can be used in bash shell scripts. There are a couple of options available, but none that were versatile enough. A little research resulted in finding that Google Translate offered what I wanted and that there was a JSON interface, which I could use with Perl’s JSON module.

I added a few niceties such as help, multiple source options and a listing of available languages.

Entering:

tranny -t de ‘I am a citizen of Berlin’

gets this result:

Ich bin ein Bürger von Berlin

and entering:

tranny ‘Ich bin ein Bürger von Berlin’

results:

I am a citizen of Berlin

I created a Google Code project for tranny at: http://code.google.com/p/tranny/

If you need a scriptable translator, give it a try. If you run into trouble or would like to suggest changes, leave comments here or at the project.

#!/usr/bin/perl
#
# what:      tranny, a language translator
# project:   https://code.google.com/p/tranny/
# copyright: Copyright 2010, Andrew Ault
# license:   This content is released under the  http://code.google.com/p/tranny/wiki/license MIT License.
#
# Uses the JSON module from CPAN. To install: "sudo cpan JSON"
#

use strict;
use warnings;
use POSIX;
use Getopt::Std;
use JSON;
use LWP;

require 'sys/ioctl.ph';
die "no TIOCGWINSZ " unless defined &TIOCGWINSZ;

my $original;
my $winsize;
my $has_tty = 1;
my ( $screen_rows, $screen_cols, $screen_xpixels, $screen_ypixels );

my %languages = (
				  'afrikaans'      => 'af',
				  'albanian'       => 'sq',
				  'amharic'        => 'am',
				  'arabic'         => 'ar',
				  'armenian'       => 'hy',
				  'azerbaijani'    => 'az',
				  'basque'         => 'eu',
				  'belarusian'     => 'be',
				  'bengali'        => 'bn',
				  'bihari'         => 'bh',
				  'breton'         => 'br',
				  'bulgarian'      => 'bg',
				  'burmese'        => 'my',
				  'catalan'        => 'ca',
				  'cherokee'       => 'chr',
				  'chinese'        => 'zh',
				  'chinese simp'   => 'zh-cn',
				  'chinese trad'   => 'zh-tw',
				  'corsican'       => 'co',
				  'croatian'       => 'hr',
				  'czech'          => 'cs',
				  'danish'         => 'da',
				  'dhivehi'        => 'dv',
				  'dutch'          => 'nl',
				  'english'        => 'en',
				  'esperanto'      => 'eo',
				  'estonian'       => 'et',
				  'faroese'        => 'fo',
				  'filipino'       => 'tl',
				  'finnish'        => 'fi',
				  'french'         => 'fr',
				  'frisian'        => 'fy',
				  'galician'       => 'gl',
				  'georgian'       => 'ka',
				  'german'         => 'de',
				  'greek'          => 'el',
				  'gujarati'       => 'gu',
				  'haitian creole' => 'ht',
				  'hebrew'         => 'iw',
				  'hindi'          => 'hi',
				  'hungarian'      => 'hu',
				  'icelandic'      => 'is',
				  'indonesian'     => 'id',
				  'inuktitut'      => 'iu',
				  'irish'          => 'ga',
				  'italian'        => 'it',
				  'japanese'       => 'ja',
				  'javanese'       => 'jw',
				  'kannada'        => 'kn',
				  'kazakh'         => 'kk',
				  'khmer'          => 'km',
				  'korean'         => 'ko',
				  'kurdish'        => 'ku',
				  'kyrgyz'         => 'ky',
				  'lao'            => 'lo',
				  'latin'          => 'la',
				  'latvian'        => 'lv',
				  'lithuanian'     => 'lt',
				  'luxembourgish'  => 'lb',
				  'macedonian'     => 'mk',
				  'malay'          => 'ms',
				  'malayalam'      => 'ml',
				  'maltese'        => 'mt',
				  'maori'          => 'mi',
				  'marathi'        => 'mr',
				  'mongolian'      => 'mn',
				  'nepali'         => 'ne',
				  'norwegian'      => 'no',
				  'occitan'        => 'oc',
				  'oriya'          => 'or',
				  'pashto'         => 'ps',
				  'persian'        => 'fa',
				  'polish'         => 'pl',
				  'portuguese'     => 'pt',
				  'punjabi'        => 'pa',
				  'quechua'        => 'qu',
				  'romanian'       => 'ro',
				  'russian'        => 'ru',
				  'sanskrit'       => 'sa',
				  'scots_gaelic'   => 'gd',
				  'serbian'        => 'sr',
				  'sindhi'         => 'sd',
				  'sinhalese'      => 'si',
				  'slovak'         => 'sk',
				  'slovenian'      => 'sl',
				  'spanish'        => 'es',
				  'sundanese'      => 'su',
				  'swahili'        => 'sw',
				  'swedish'        => 'sv',
				  'syriac'         => 'syr',
				  'tajik'          => 'tg',
				  'tamil'          => 'ta',
				  'tatar'          => 'tt',
				  'telugu'         => 'te',
				  'thai'           => 'th',
				  'tibetan'        => 'bo',
				  'tonga'          => 'to',
				  'turkish'        => 'tr',
				  'ukrainian'      => 'uk',
				  'urdu'           => 'ur',
				  'uzbek'          => 'uz',
				  'uighur'         => 'ug',
				  'vietnamese'     => 'vi',
				  'welsh'          => 'cy',
				  'yiddish'        => 'yi',
				  'yoruba'         => 'yo',
);

# get window size for country listing
open( TTY, "+;
	close FILE;
# text is from STDIN
} else {
	# slurp STDIN
	local $/ = undef;
	$original = ;
}

my $ua = LWP::UserAgent->new;
$ua->agent("PGDict/1.0");
my $request =
  HTTP::Request->new( GET => "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=$from|$to&q=$original" );
my $response = $ua->request($request);

if ( $response->is_success ) {
	my $perl_res = from_json( $response->content );
	if ( $perl_res->{'responseStatus'} eq '200' ) {
		print $perl_res->{'responseData'}->{'translatedText'} . "\n";
	} else {
		warn "error " . $perl_res->{'responseDetails'} . "\n";
	}
} else {
	print $response->status_line . "\n";
}

sub usage {
	print "usage: ";
	print "\ttranny -f language_code -t language_code [original text]\n\n";
	print "-f language_code (optional)\n\n";
	print "-t language_code (optional)\n\n";
	print "-o original_file (optional)\n\n";
	print "-h this help\n\n";
	print "-l language list\n\n";
	print "Tranny uses Google Translate and requires an Internet connection to work.\n";
	print "Text is translated from STDIN, from the command line or a file with -o.\n\n";
	print "By default,the 'from' language is automatically detected and translated to English (en).\n\n";
	if ( defined $opts{l} && $opts{l} == 1 ) { list_languages() }
	exit;
}

sub list_languages {
	my $num_columns = ceil( $screen_cols / 23 );
	my $num_rows = ceil ( keys(%languages) /$num_columns );
	my $row = 0;
	my $col = 0;
	my @formatted_languages = ( );
	foreach my $key ( sort ( keys(%languages) ) ) {
		$row++;
		$formatted_languages[$col][$row] =  sprintf( "%-14s %-6s", $key, $languages{$key} );
		if ( $row == $num_rows ){
			$row = 0;
			$col++;
		}
	}
	for ($row = 0; $row <= $num_rows; $row++) {
		for ($col = 0; $col <= $num_columns; $col++) {
			if ( defined $formatted_languages[$col][$row] ){
				print $formatted_languages[$col][$row];
			}
		}
		print "\n";
	}
}

Bash for the word lover

It’s a WYSIWYG world. After all, we’re nearly in the future, which I define as 2019, the year Rick Deckard chases down replicants in the Blade Runner. Still no flying cars, which is disappointing. Even so, we have Steve Jobs, so the future coming, right?

GUI everything isn’t all that it could be. For many, many tasks, it is more expeditious to open a terminal and get a bash prompt. CLI. Character. Text. It could be green on black, or it would be a rainbow on white, but it is not different from a Televideo terminal, or a Teletype for that matter. As good as the Bourne Again Shell is, it is not graphical or fancy.

What it is is efficient. For a sharp mind and one given to efficiency, the terminal is power. Want to replace frick with frack in 800 HTML files?

find ~/web/project3 -name '*.php' | xargs perl -pi -e 's/frick/frack/g'

Bam. Done.

This is why I have 3 terminals open right now. One is connected to a server somewhere in Texas. I just fixed some text on a site with a command much like the one above.

But you already knew all that. You Googled and found this page, so you are already 1337 or whatnot. How about some word power on the command line?

Install some packages

This will install the packages we will use on an Ubuntu or Debian system. For other distributions, you will need to use your distributions package system.

To install on Ubuntu or Debian, just install the needed APT packages:

sudo aptitude -y install wordnet wamerican-large curl wget an

Definitions

This grabs a definition for a word from dict.org. For “unusual” for example:

curl --stderr /dev/null dict://dict.org/d:unusual | sed '/^[.,0-9].*$/d'

Which returns:

Unusual \Un*u"su*al\, a.
Not usual; uncommon; rare; as, an unusual season; a person of
unusual grace or erudition. -- {Un*u"su*al*ly}, adv. --
{Un*u"su*al*ness}, n.
[1913 Webster]

As you can see you are using curl to request a definition for “unusual”, then using sed to filter the results, to exclude extra stuff you don’t want. You could just enter “curl dict://dict.org/d:unusual” for the raw deal. Good on ya.

You can turn this into a script:

#! /bin/bash
# display definition of a word
#
curl --stderr /dev/null dict://dict.org/d:$1 | sed '/^[.,0-9].*$/d'

Save that in a file called “def” and run “chmod +x def” to make it executable. Then “def unusual” will return the same definition. You just created your own tool. You rock.

Wordnet

How about more power? Princeton has a project called Wordnet, which organizes nouns, verbs, adjectives and adverbs into set of “cognitive synonyms” and provides tools to use this data. With Wordnet, synonyms, antonyms and other lexical relations can be found for a given word.

To show a definition, (still using “unusual” as an example):

wn unusual -over

Here’s the output:

Overview of adj unusual

The adj unusual has 3 senses (first 3 from tagged texts)

1. (24) unusual -- (not usual or common or ordinary; "a scene of unusual beauty"; "a man of unusual ability"; "cruel and unusual punishment"; "an unusual meteorite")
2. (1) strange, unusual -- (being definitely out of the ordinary and unexpected; slightly odd or even a bit weird; "a strange exaltation that was indefinable"; "a strange fantastical mind"; "what a strange sense of humor she has")
3. (1) unusual -- (not commonly encountered; "two-career families are no longer unusual")

This uses the “-over” option. Some other options are:

-synsa adjective synonyms
-synsn noun synonyms
-synsr adverb synonyms
-antsa adjective antonymns
-antsn noun antonymns
-antsr adverb antonymns

Wordnet is extensive and there are many more options, run “man wn” for more.

Crossword help

This is simply a use of grep to pattern match words in a word list file.

Use a regular expression to find a word. In quotes, start your pattern with a “^” character and end with a “$” character. Use a period “.” for each unknown character.

grep '^.a...f.c.n...$' /usr/share/dict/words

Magnificent!

Rhyming

This uses the rhyme project, which provides a rhyming dictionary for the command line.

To get, build and install rhyme on your system:

sudo aptitude -y install build-essential libgdbm-dev libreadline-dev
cd ~
DIR="src" && [ -d "$DIR" ] || mkdir "$DIR"
cd src
wget http://softlayer.dl.sourceforge.net/project/rhyme/rhyme/0.9/rhyme-0.9.tar.gz
tar -xzf rhyme-0.9.tar.gz
cd rhyme-0.9
make
sudo make install

Holy smokes, you just built software! There is no stopping you. To find a rhyme, using “house” as an example:

rhyme house

House rhymes! Lots of them.

Anagrams

Anagrams are pretty much pure word fun. It is fun to see what an anagram of your name is.

Print single-word anagrams of “andrew”:

an -l 1 andrew

Just call me the wander warden.

Install Dropbox on Ubuntu Server

Dropbox is so useful! Wouldn’t it be great to have that same convenience and function for your user account on your server, just like you have on your workstation?

This has been tested on Ubuntu Lucid and Jaunty. This procedure will create a system with:

  • Install a separate Dropbox client (daemon) for individual users
  • Each user has a separate Dropbox account
  • All the daemons will be managed together with normal daemon controls

This allows individuals to have their own Dropbox accounts, each with a separate process syncing their individual ~/Dropbox directory. With one command an admin can start or stop all the daemons at once.

Install Prerequisites

Later, we’ll need to read a sqlite3 database record, so install sqlite3.

sudo aptitude -y install sqlite3

Install Dropbox client for an individual user

This step is repeated for each user that wants a Dropbox client. Start by setting up your own account, then repeat for each user. This is run with a user’s own account. The changes made all take place in their home directory.

First, determine whether you have 32-bit or 64-bit Ubuntu Server. You must install the correct version, either 32 or 64 bit or it will not work. The following command will tell you which is installed:

uname -a | grep '_64' >/dev/null && echo 'A 64-bit OS is installed'; uname -a | grep '_64' >/dev/null || echo 'A 32-bit OS is installed'

Run the correct installation, based on whether a 32-bit or 64-bit OS is installed.

32-bit installation:

cd ~
wget -O dropbox.tar.gz http://www.dropbox.com/download/?plat=lnx.x86
tar -zxof dropbox.tar.gz

or

64-bit installation:

cd ~
wget -O dropbox.tar.gz http://www.dropbox.com/download/?plat=lnx.x86_64
tar -zxof dropbox.tar.gz

Link user’s Dropbox client to their Dropbox account:

wget http://dl.dropbox.com/u/6995/dbmakefakelib.py
python dbmakefakelib.py

The above will run for a little while without printing anything, then print “dropboxd ran for 15 seconds without quitting – success?”. When it does so, press control-c twice. Yes, it is unusual. What this does is populate a sqlite3 database with an ID from the Dropbox server. Next, we’ll extract that code and use it to link your Dropbox user account with this CLI Dropbox client instance.

On the server, via SSH

Get the URL with:

echo https://www.dropbox.com/cli_link?host_id=`echo '.dump config' | sqlite3 ~/.dropbox/dropbox.db | grep host_id | cut -d \' -f 4 | python -c 'print raw_input().decode("base64")' | grep '^V' | cut -b 2-`

On your local machine, in a web browser

Copy the URL that the above printed and paste it into a web browser. When you do so, Dropbox will register your client on the server with your Dropbox account.

At this point, Dropbox will not be quite working yet. The next steps will take care of that.

Create Dropbox daemon control

The next task is to create a system to start and stop the dropbox daemon for each user on the system that has Dropbox installed for his/her user account. The following daemon init script was lifted from: http://wiki.dropbox.com/TipsAndTricks/TextBasedLinuxInstall.

Of course, this is for the use of the system admin. This creates a normal daemon init start/stop script and installs it so the Dropbox daemons are started when the system boots. The admin can also use this to control the Dropbox daemons manually.

sudo vi /etc/init.d/dropbox

Paste in the following code. Then, modify line 3, replacing “user1 user2″ with your username. For future reference, additional user’s Dropbox daemons can be controlled with this one script – add additional username separated with spaces.

# dropbox service
# separate usernames in the following line with spaces.
DROPBOX_USERS="user1 user2"
 
DAEMON=.dropbox-dist/dropbox
 
start() {
    echo "Starting dropbox..."
    for dbuser in $DROPBOX_USERS; do
        HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
        if [ -x $HOMEDIR/$DAEMON ]; then
            HOME="$HOMEDIR" start-stop-daemon -b -o -c $dbuser -S -u $dbuser -x $HOMEDIR/$DAEMON
        fi
    done
}
 
stop() {
    echo "Stopping dropbox..."
    for dbuser in $DROPBOX_USERS; do
        HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
        if [ -x $HOMEDIR/$DAEMON ]; then
            start-stop-daemon -o -c $dbuser -K -u $dbuser -x $HOMEDIR/$DAEMON
        fi
    done
}
 
status() {
    for dbuser in $DROPBOX_USERS; do
        dbpid=`pgrep -u $dbuser dropbox`
        if [ -z $dbpid ] ; then
            echo "dropboxd for USER $dbuser: not running."
        else
            echo "dropboxd for USER $dbuser: running (pid $dbpid)"
        fi
    done
}

case "$1" in
  start)
    start
    ;;
 
  stop)
    stop
    ;;
 
  restart|reload|force-reload)
    stop
    start
    ;;
 
  status)
    status
    ;;
 
  *)
    echo "Usage: /etc/init.d/dropbox {start|stop|reload|force-reload|restart|status}"
    exit 1
 
esac
 
exit 0

Make the init script executable and restart the daemon:

sudo chmod +x /etc/init.d/dropbox
sudo /etc/init.d/dropbox restart

When you restart the daemon, it will be running correctly linked to your Dropbox account. It will create a Dropbox directory in your home directory and will start to populate it with files you have on Dropbox.

Have the daemon(s) run automatically at boot time:

sudo update-rc.d dropbox defaults

The above correctly copies links as needed so the daemon(s) start when the server boots.

Managing the Daemons

A separate daemon will be run for each user that has the Dropbox client installed – with only one command. This makes it easy for individual users to have separate Dropbox accounts, each syncing to ~/Dropbox for their user account. Here are the commands to manage these daemons:

Start Dropbox services for all users:

sudo /etc/init.d/dropbox start

Stop Dropbox services for all users:

sudo /etc/init.d/dropbox stop

Restart Dropbox services for all users:

sudo /etc/init.d/dropbox restart

Get service status for each user Dropbox service:

sudo /etc/init.d/dropbox status

Script to create a Kohana instance

This is a copy of the script I use to create an instance of a Kohana project. It works on Ubuntu server (tested with Lucid). Adapt it to your own needs, of course.

#! /bin/bash
 
CODE="upc"
OWNER="myusername"
GROUP="mygroupname"
KOHANA_URL="http://dev.kohanaframework.org/attachments/download/1355/kohana-v2.3.4.zip"
KOHANA_ZIP="kohana-v2.3.4.zip"
SITESROOT="/usr/local/var/www"
URL="www.mysite.com"
# database.php parameters
DBUSER="mydbuser"
DBPWD="mydbpasswd"
DBHOST="localhost"
DBDATABASE="mydbname"
 
#
# make sure this is run as root
#
if [[ $UID -ne 0 ]]; then
    echo "Not running as root"
    exit
fi
 
if [ ! -d $SITESROOT ]; then
	echo "$SITESROOT directory does not exist"
	exit
fi
 
#
# change to the siteroot directory
#
cd $SITESROOT
 
if [ -d $URL ]; then
	echo "$SITESROOT/$URL directory already exists"
	exit
fi 
 
mkdir $URL
chown $OWNER:$GROUP $URL
cd $URL
 
#
# create kohana directory
#
echo creating kohana directory
mkdir kohana
chown $OWNER:$GROUP kohana
chmod g+ws kohana
wget "$KOHANA_URL"
unzip "$KOHANA_ZIP"
rm "$KOHANA_ZIP"
chown -R $OWNER:$GROUP kohana
chmod -R g+w kohana
chown $OWNER:www-data kohana/application/logs
rm kohana/example.htaccess
rm kohana/install.php
rm kohana/Kohana\ License.html
rm kohana/kohana.png
 
#
# create public directory
#
echo creating public directory
mkdir public
chown $OWNER:$GROUP public
chmod g+ws public
mv kohana/index.php public/
sed -i "s\\kohana_application =.*$\\kohana_application = '../kohana/application';\\" public/index.php
sed -i "s\\kohana_modules =.*$\\kohana_modules = '../kohana/modules';\\" public/index.php
sed -i "s\\kohana_system =.*$\\kohana_system = '../kohana/system';\\" public/index.php
echo "# turn on URL rewriting
RewriteEngine On
RewriteBase /
" > public/.htaccess
 
#
# create Apache logs directory
#
echo creating logs directory
mkdir logs
chown $OWNER:www-data logs
chmod g+ws logs
touch logs/error.log
touch logs/combined.log
 
#
# create utilties directory
#
echo creating utilties directory
mkdir utilities
chown $OWNER:$GROUP utilities
chmod g+ws utilities
 
#
# modify kohana/application/config/config.php
#
echo modifying kohana/application/config/config.php
sed -i "s\\'/kohana/'\\'$URL/'\\" kohana/application/config/config.php
 
#
# create kohana/application/config/database.php
#
echo creating kohana/application/config/database.php
cp kohana/system/config/database.php kohana/application/config/database.php
chown $OWNER:$GROUP kohana/application/config/database.php
chmod g+w kohana/application/config/database.php
sed -i "s\\'user'.*$\\'user'     => '$DBUSER',\\" kohana/application/config/database.php
sed -i "s\\'pass'.*$\\'pass'     => '$DBPWD',\\" kohana/application/config/database.php
sed -i "s\\'host'.*$\\'host'     => '$DBHOST',\\" kohana/application/config/database.php
sed -i "s\\'database'.*$\\'database' => '$DBDATABASE',\\" kohana/application/config/database.php
 
#
# create kohana/application/config/routes.php
#
echo creating kohana/application/config/routes.php
cp kohana/system/config/routes.php kohana/application/config/routes.php
chown $OWNER:$GROUP kohana/application/config/routes.php
chmod g+w kohana/application/config/routes.php
sed -i "s\\'welcome'\\'/index'\\" kohana/application/config/routes.php
 
#
# copy kohana/application/config/profiler.php
#
echo creating kohana/application/config/profiler.php
cp kohana/system/config/profiler.php kohana/application/config/profiler.php
chown $OWNER:$GROUP kohana/application/config/profiler.php
chmod g+w kohana/application/config/profiler.php
 
#
# create Apache virtual site
#
echo creating Apache virtual site file in /etc/apache2/sites-available
echo "
	ServerName $URL
	DocumentRoot $SITESROOT/$URL/public
	DirectoryIndex index.php
	LogLevel warn
	ErrorLog $SITESROOT/$URL/logs/error.log
	CustomLog $SITESROOT/$URL/logs/combined.log combined

" > /etc/apache2/sites-available/$URL
 
#
# finish up
#
echo "enable site with: sudo a2ensite $URL"
echo "restart Apache with: sudo /etc/init.d/apache2 restart"

Creating a Perl Daemon in Ubuntu

If you have a process that need to run in the background, creating a daemon is the key. Some examples of what your can do:

  • Wait for video files to be dropped via FTP and then process them
  • Check for a recurring problem and report it to admin
  • Monitor system load and report to the admin at a certain threshold

In this post we’ll take a look at how to create a Perl daemon script and how to get it to run as a daemon whenever the system is started. For the Perl script, a template will be created and for control of the daemon process, standard Debian facilities will be used.

What will be covered:

  • Create the start/stop script in /etc/init.d
  • Create the Perl daemon script in /usr/bin/
  • Create the log file in /var/log/
  • Get the daemon to start automatically when the system boots
  • How to manage your daemon

The start/stop script starts and stops the daemon. It is also used by the system to start your daemon when the system starts.

The Perl daemon script contains your custom Perl code to run in the background. It executes your code every x number of seconds.

Create the Start/stop Script

Happily, Debian and therefore Ubuntu, supplies a template that uses the Debian start-stop-daemon command to start and stop daemons. It is only necessary to copy this template to a new file of the correct name and modify it for the purpose.

These scripts reside in the /etc/init.d/ directory. The script you create will have the exact same filename as the Perl daemon script you will create and the name cannot have a file extension – just the bare name.

Substitute your daemon’s name for “mydaemon” in the examples.

sudo cp /etc/init.d/skeleton /etc/init.d/mydaemon
sudo chmod +x /etc/init.d/mydaemon
sudo vi /etc/init.d/mydaemon

Make changes to your start/stop script. Locate the header in the script:

PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="Description of the service"
NAME=daemonexecutablename
DAEMON=/usr/bin/$NAME
DAEMON_ARGS="--options args"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
  • Change the content of DESC to a meaningful description of your daemon.
  • Change the content for NAME to the exact name of your daemon.
  • Create the Perl Daemon Script

    Use the Perl Daemon Script Template located at the end of this post. In the following, the script template is copied to /usr/bin/where executables specific to this system are located. Actually, you would copy the template to your home directory, make the changes and then copy the daemon script to /usr/bin/.

    sudo cp daemon_template.pl /usr/bin/mydaemon
    sudo vi /usr/bin/mydaemon
    sudo chmod +x /usr/bin/mydaemon
    

    The script template is commented with the changes that are needed. In particular, change the value of $daemonName to the exact name of your new daemon. Ten, add your custom code where # do something appears.

    Prerequisites

    The script uses File::Pid and POSIX from CPAN, so you’ll need to install that module:

    sudo aptitude install cpan
    sudo cpan POSIX
    sudo cpan File::Pid
    

    Create the Log File

    We’ll create the log file so it has the correct ownership and permissions. The log fie has the daemon name appended with “.log”. It is located in the /var/log/ directory.

    sudo touch /var/log/mydaemon.log
    sudo chmod 640 /var/log/mydaemon.log
    sudo chown root:adm /var/log/mydaemon.log
    

    The permissions and ownership change allow adm group members to read the log, per convention. Add yourself to adm group to view logs.

    Run your Daemon

    As you have created a standard start/stop script for your daemon, you can start it the standard way:

    sudo /etc/init.d/mydaemon start
    

    Stop your Daemon

    Similarly, your daemon is easy to stop and restart:

    sudo /etc/init.d/mydaemon stop
    
    sudo /etc/init.d/mydaemon restart
    

    Automatically Start your Daemon

    This command tells your system to start your daemon automatically when the system starts:

    update-rc.d mydaemon defaults 99
    

    To keep your daemon from starting, if needed:

    update-rc.d -f mydaemon remove
    

    Managing Daemons

    List running daemons

    Well-behaved daemons generally create a PID file in /var/run/. List that directory and your have a fair list of running daemons.

    sudo ls /var/run/
    

    Get PID for a particular daemon

    If you know the name of a daemon and want to see if it is running and get its PID, use pgrep.

    pgrep mydaemon
    

    Also useful is the ps command, which list processes.

    ps aux
    

    The aux switch limits output to processes not associated with a terminal.

    Show programs running as daemons

    This line attempts to show programs that are running as daemons.

    which `ps aux | cut -c 66- | cut -d\  -f 1` | sort | uniq
    

    The Perl Daemon Script Template

    #!/usr/bin/perl -w
    #
    # mydaemon.pl by Andrew Ault, www.andrewault.net
    #
    # Free software. Use this as you wish.
    #
    # Throughout this template "mydaemon" is used where the name of your daemon should
    # be, replace occurrences of "mydaemon" with the name of your daemon.
    #
    # This name will also be the exact name to give this file (WITHOUT a ".pl" extension).
    #
    # It is also the exact name to give the start-stop script that will go into the
    # /etc/init.d/ directory.
    #
    # It is also the name of the log file in the /var/log/ directory WITH a ".log"
    # file extension.
    #
    # Replace "# do something" with your super useful code.
    #
    # Use "# logEntry("log something");" to log whatever your need to see in the log.
    #
    use strict;
    use warnings;
    use POSIX;
    use File::Pid;
     
    # make "mydaemon.log" file in /var/log/ with "chown root:adm mydaemon"
     
    # TODO: change "mydaemon" to the exact name of your daemon.
    my $daemonName    = "mydaemon";
    #
    my $dieNow        = 0;                                     # used for "infinte loop" construct - allows daemon mode to gracefully exit
    my $sleepMainLoop = 120;                                    # number of seconds to wait between "do something" execution after queue is clear
    my $logging       = 1;                                     # 1= logging is on
    my $logFilePath   = "/var/log/";                           # log file path
    my $logFile       = $logFilePath . $daemonName . ".log";
    my $pidFilePath   = "/var/run/";                           # PID file path
    my $pidFile       = $pidFilePath . $daemonName . ".pid";
     
    # daemonize
    use POSIX qw(setsid);
    chdir '/';
    umask 0;
    open STDIN,  '/dev/null'   or die "Can't read /dev/null: $!";
    open STDOUT, '>>/dev/null' or die "Can't write to /dev/null: $!";
    open STDERR, '>>/dev/null' or die "Can't write to /dev/null: $!";
    defined( my $pid = fork ) or die "Can't fork: $!";
    exit if $pid;
     
    # dissociate this process from the controlling terminal that started it and stop being part
    # of whatever process group this process was a part of.
    POSIX::setsid() or die "Can't start a new session.";
     
    # callback signal handler for signals.
    $SIG{INT} = $SIG{TERM} = $SIG{HUP} = \&signalHandler;
    $SIG{PIPE} = 'ignore';
     
    # create pid file in /var/run/
    my $pidfile = File::Pid->new( { file => $pidFile, } );
     
    $pidfile->write or die "Can't write PID file, /dev/null: $!";
     
    # turn on logging
    if ($logging) {
    	open LOG, ">>$logFile";
    	select((select(LOG), $|=1)[0]); # make the log file "hot" - turn off buffering
    }
     
    # "infinite" loop where some useful process happens
    until ($dieNow) {
    	sleep($sleepMainLoop);
     
    	# TODO: put your custom code here!
    	# do something
     
    	# logEntry("log something"); # use this to log whatever you need to
    }
     
    # add a line to the log file
    sub logEntry {
    	my ($logText) = @_;
    	my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime(time);
    	my $dateTime = sprintf "%4d-%02d-%02d %02d:%02d:%02d", $year + 1900, $mon + 1, $mday, $hour, $min, $sec;
    	if ($logging) {
    		print LOG "$dateTime $logText\n";
    	}
    }
     
    # catch signals and end the program if one is caught.
    sub signalHandler {
    	$dieNow = 1;    # this will cause the "infinite loop" to exit
    }
     
    # do this stuff when exit() is called.
    END {
    	if ($logging) { close LOG }
    	$pidfile->remove if defined $pidfile;
    }
    

    UPDATE:

    Looks like there’s a page in French about this post: http://www.duncane.net/2011/02/25/perl-daemon/. Merci Duncane!

    Passive Mode (PASV) FTP client on an Ubuntu server

    If you need to communicate from your Ubuntu server to an FTP server that requires passive mode, there is a problem: your firewall likely blocks communication. Using an FTP client manually, you can probably connect with the server, but not list or transfer files!

    The reasons for this are straightforward, your system is operating exactly as it is configured to. The explanation requires a little understanding of FTP and firewalls.

    Most IP protocols use one port on the local machine and port on the server being connected to. FTP happens to use two ports instead of one. When negotiating a connection, the two computers negotiate which port to send data to. This brings us to an important difference between the two modes:

    • In active mode FTP, the client sends the server a PORT command, which tells the server client which port to use for data. The client connects to the server.
    • In passive more, the client sends the server a PASV command that asks for a server port to use for data. The server connects with the client.

    The tricky bits concern this second port that is negotiated. This port is not a fixed number, it is a dynamically allocated port above 1023. The port number is encoded in a packet as two numbers that need to be multiplied together to get the port number. The firewalls involved need to be smart enough to recognize the FTP negotiation and extract this data from the data, open that specified port and keep it open during the FTP session.

    In active mode, this tricky bit is handled by the server, but in passive mode, it is handled by the client’s firewall! Ah ha! So, you need to configure your firewall to be smart about address translation and FTP connections.

    Configuring the firewall

    You will need to activate a couple of kernel modules for iptables. These will turn on NAT (network address translation) for FTP and FTP connection tracking. As iptables/Netfilter is part of the kernel, we need to use modprobe to add these to the current session and also make changes to /etc/modules so the modules will load next time the server is rebooted.

    First, use modprobe to use these two modules now:

    sudo modprobe ip_nat_ftp
    sudo modprobe ip_conntrack_ftp
    

    Then, modify /etc/modules so the modules will load on next reboot:

    sudo vi /etc/modules
    

    Add these lines:

    ip_nat_ftp
    ip_conntrack_ftp
    

    With these two modules, you should now be able to use passive mode from an FTP client on your Ubuntu server.