|
|
by Andrew Ault, on February 14th, 2012
Do you have a server that you can access with OpenSSH? Do you want to be able to browse the web, even non-SSL, unencrypted pages, without others on the network being able to see what you’re looking at or even hijacking your sessions? Given the existence of Firesheep, it is really easy for even unsophisticated users to hijack a web browsing session.
The method I’m presenting is easy and effective. OpenSSH makes this a snap. Your web browsing packets will be routed via an encrypted connection to your server.
Create a SOCKS Proxy Connection on localhost
First, in a terminal, open a SOCKS connection to your server with OpenSSH. Just add “-D 9999″ to your normal SSH command. This will create a SOCKS proxy on localhost at port 9999.
ssh -D 9999 username@myserver.com
Depending on your configuration, you may need to enter your server account password. Whatever your normal authentication is for SSH. This will even open a normal SSH session, you will get a shell prompt on the server like normal. If you do not want a shell prompt, use “-ND” instead of “-D” and the

You now have a proxy on your local computer using SOCKS on port 9999. Now we just need to use it.
Install Switchy! in your Chrome browser
This can be easily found in the Chrome Web Store.
Use your Shiny New SOCKS Proxy
Open the Switchy! Options dialog. Type the a name for this proxy in Profile Name. On the SOCKS Host line, enter “localhost” in the first blank and “9999″ for the Port. Click the Save button. You are using a SOCKS proxy running on your localhost.
Then, select the proxy by clicking the Switchy! icon in Chrome and selecting the proxy name you just entered.
You are now using a secure connection to browse the web. Note that someone on the network where your server is hosted can still snoop your traffic, but not in the Starbucks where your are sitting.
by Andrew Ault, on January 28th, 2012
When you need to give users access to files that are not in the public directory, where you cannot simply use an anchor tag with an “href”, you need to do a bit of work. For example, if you’ve created an authentication system where only authenticated users can download or view a file, this can be necessary.
$filename = 'something';
$file_path = '/somepath/to/the/file/';
$file_fullpath = $file_path . $filename;
//
//
if (!file_exists($file_fullpath)) {
header("HTTP/1.0 404 Not Found");
return;
}
//
// get the mime type
$finfo = finfo_open(FILEINFO_MIME_TYPE);
$mimeType = finfo_file($finfo, $file_fullpath);
//
// calc values for header
$size = filesize($file_fullpath);
$time = date('r', filemtime($file_fullpath));
$fm = @fopen($file_fullpath, 'rb');
if (!$fm) {
header('HTTP/1.0 505 Internal server error');
return;
}
$begin = 0;
$end = $size;
if (isset($_SERVER['HTTP_RANGE'])) {
if (preg_match('/bytes=\h*(\d+)-(\d*)[\D.*]?/i', $_SERVER['HTTP_RANGE'], $matches)) {
$begin = intval($matches[0]);
if (!empty($matches[1])) $end = intval($matches[1]);
}
}
//
// create http header
if ($begin > 0 || $end < $size) {
header('HTTP/1.0 206 Partial Content');
}else {
header('HTTP/1.0 200 OK');
}
header('HTTP/1.0 200 OK');
header("Content-Type: $mimeType");
header('Cache-Control: public, must-revalidate, max-age=0');
header('Pragma: no-cache');
header('Accept-Ranges: bytes');
header('Content-Length:' . ($end - $begin));
header("Content-Range: bytes $begin-$end/$size");
header("Content-Disposition: inline; filename=$filename");
header("Content-Transfer-Encoding: binary\n");
header("Last-Modified: $time");
header('Connection: close');
// output the file
$cur = $begin;
fseek($fm, $begin, 0);
while (!feof($fm) && $cur < $end && (connection_status() == 0)) {
print fread($fm, min(1024 * 16, $end - $cur));
$cur+= 1024 * 16;
}
There you have it. Your user can click the link for the file, which then runs this code. In my case, this is in a Kohana controller.
The files are not stored in a public directory, so unauthenticated users cannot access them.
If a file is created on the fly, like a PDF of an invoice, this will also work.
by Andrew Ault, on June 24th, 2011
I misspelled a word when I exported a large group of photos from Aperture. I did not want to repeat the operation, so I renamed the files at a bash prompt in Terminal:
for FILE in * ; do NEWNAME=`echo $FILE | sed 's/oldstring/newstring/g' ` ; mv "$FILE" "$NEWNAME" ; done
Just change ‘oldstring’ and ‘newstring’ to the values needed.
by Andrew Ault, on July 9th, 2010
This is a command line Perl script I made a while back to simply show details about a specified video. It is tested on Ubuntu.
You can modify it for your specific requirements or grab some code for another use.
#!/usr/bin/perl -w
use strict;
use warnings;
use IPC::Open3;
# example
my $filename = $ARGV[0];
my %videoInfo = videoInfo($filename);
print "duration: " . $videoInfo{'duration'} . "\n";
print "durationsecs: " . $videoInfo{'durationsecs'} . "\n";
print "bitrate: " . $videoInfo{'bitrate'} . "\n";
print "vcodec: " . $videoInfo{'vcodec'} . "\n";
print "vformat: " . $videoInfo{'vformat'} . "\n";
print "acodec: " . $videoInfo{'acodec'} . "\n";
print "asamplerate: " . $videoInfo{'asamplerate'} . "\n";
print "achannels: " . $videoInfo{'achannels'} . "\n";
#
# returns media information in a hash
sub videoInfo {
# ffmpeg command
my $ffmpeg = '/usr/local/bin/ffmpeg';
my %finfo = (
'duration' => "00:00:00.00",
'durationsecs' => "0",
'bitrate' => "0",
'vcodec' => "",
'vformat' => "",
'acodec' => "",
'asamplerate' => "0",
'achannels' => "0",
);
my $file = shift;
# escaping characters
$file =~ s/(\W)/\\$1/g;
open3( "</dev/null", ">/dev/null", \*ERPH, "$ffmpeg -i $file" ) or die "can't run $ffmpeg\n";
my @res = <ERPH>;
# parse ffmpeg output
foreach (@res) {
print;
# duration
if (m!Duration: ([0-9][0-9]:[0-9][0-9]:[0-9][0-9].[0-9][0-9])!) {
$finfo{'duration'} = $1;
}
# bitrate
if (m!bitrate: (\d*) kb/s!) {
$finfo{'bitrate'} = $1;
}
# vcodec and vformat
if (/Video: (\w*), (\w*),/) {
$finfo{'vcodec'} = $1;
$finfo{'vformat'} = $2;
}
# Stream #0.1(und): Audio: aac, 48000 Hz, 1 channels, s16, 64 kb/s
# acodec, samplerate, stereo and audiorate
if (m!Audio: (\w*), (\d*) Hz, (\d*)!) {
$finfo{'acodec'} = $1;
$finfo{'asamplerate'} = $2;
$finfo{'achannels'} = $3;
}
}
my $tenths = substr( $finfo{'duration'}, 9, 2 );
my $seconds = substr( $finfo{'duration'}, 6, 2 );
my $minutes = substr( $finfo{'duration'}, 3, 2 );
my $hours = substr( $finfo{'duration'}, 0, 2 );
$finfo{'durationsecs'} = ( $tenths * .01 ) + $seconds + ( $minutes * 60 ) + ( $hours * 360 );
return %finfo;
}
by Andrew Ault, on July 6th, 2010
This will help you quickly install a Drupal site on an Ubuntu server.
This is my script for this purpose. Use it as you will. This procedure assumes that you have already installed Apache, MySQL, phpMyAdmin and PHP. It also assumes that you are familiar with everything. This is probably not a beginner’s guide, but an aid to someone more experienced that wants to install Drupal quickly.
Create and run a shell script to create your site
Use this template to create a shell script that will create your site. Create a text file from the following template called “make_dru.sh” (in your home directory is fine). Modify lines 7 to 17 for your installation. Then use “chmod +x make_dru.sh” and execute the script with “sudo ./make_dru.sh”.
#!/bin/bash
#
# make drupal project
#
#
OWNER="andrew" # change to your username
GROUP="staff" # change to your username or the group that will work on the site
DRUPAL_URL="http://ftp.drupal.org/files/projects/drupal-6.17.tar.gz" # update to the current version, if there is one
SITESROOT="/usr/local/var/www" # change to the directory where you put website directories
URL="www.sitename.com" # change to your site's address
# database.php parameters
DBUSER="sitename" # change to something meaningful
DBPWD="passwordgoeshere" # change to a long password
DBHOST="localhost" # leave alone
DBDATABASE=$DBUSER # leave alone
#
# make sure this is run as root
#
if [[ $UID -ne 0 ]]; then
echo "Not running as root"
exit
fi
if [ ! -d $SITESROOT ]; then
echo "$SITESROOT directory does not exist"
exit
fi
#
# check for existence of the specified user owner
#
/bin/egrep -i "^$OWNER" /etc/passwd > /dev/null
if [ ! $? -eq 0 ]; then
echo "User $OWNER, which is specified as the owner, does not exist in /etc/passwd"
exit
fi
#
# check for existence of the specified group
#
/bin/egrep -i "^$GROUP" /etc/group > /dev/null
if [ ! $? -eq 0 ]; then
echo "$GROUP, which is specified as the group to use, does not exist in /etc/group"
exit
fi
#
# change to the siteroot directory
#
cd $SITESROOT
if [ -d $URL ]; then
echo "$SITESROOT/$URL directory already exists"
exit
fi
mkdir $URL
chown $OWNER:$GROUP $URL
cd $URL
#
# get Drupal
#
echo 'get Drupal'
wget -O drupal.tgz $DRUPAL_URL
tar -xzvf drupal.tgz
find . -maxdepth 1 -type d -name 'drupal*' -exec mv {} public \;
sudo chown -R $OWNER:$GROUP public
find . -type d -exec chmod g+ws {} \;
#
# temporarily allow write to settings
#
cp public/sites/default/default.settings.php public/sites/default/settings.php
chmod a+w public/sites/default/settings.php
chmod a+w public/sites/default
#
# logs
#
echo "set up logs"
mkdir logs
sudo chown $OWNER:www-data logs
chmod g+ws logs
touch logs/error.log
touch logs/combined.log
#
# create Apache virtual site
#
echo creating Apache virtual site file in /etc/apache2/sites-available
echo "<VirtualHost *:80>
ServerName $URL
DocumentRoot $SITESROOT/$URL/public
DirectoryIndex index.php
LogLevel warn
ErrorLog $SITESROOT/$URL/logs/error.log
CustomLog $SITESROOT/$URL/logs/combined.log combined
</VirtualHost>
" > /etc/apache2/sites-available/$URL
#
# finish up
#
echo -e "Drupal website was created in: $SITESROOT/$URL\n"
echo -e "To finish, do the following:\n"
echo "1. Enable site with: sudo a2ensite $URL"
echo "2. Restart Apache with: sudo /etc/init.d/apache2 restart"
echo "3. Make a crontab entry ('sudo crontab -e') like: 0 * * * * wget -O - -q -t 1 http://$URL/cron.php"
echo -e "4. Create a MySQL database with:\n"
echo -e "\t user: $DBUSER"
echo -e "\t pwd: $DBPWD"
echo -e "\t host: $DBHOST"
echo -e "\t database: $DBDATABASE"
echo "5. Open the site in your browser and finish the set-up."
echo "6. Remove write privileges with: sudo chmod og-w public/sites/default/settings.php"
echo " ..and: sudo chmod og-w public/sites/default"
Create the database and user with phpMyAdmin
- click the “Privileges” tab and add a new user
- in the “User name” field, enter the sitename that you entered for DBUSER in the script
- in the Host field, select “Local”
- enter or generate a password – make it a long one, like 20 characters
- in the Database for User list, select “Create database with same name and grant all privileges”
- click Go
Enable the new Drupal website
This will tell Apache to enable the virtual site that was made in the script and then restart Apache.
sudo a2ensite www.sitename.com
sudo /etc/init.d/apache2 restart
Configure the site
Open site in browser and configure it
Remove temporary write privileges
During site creation, write privileges were needed, after site set-up, remove them. In a shell, while in the sites directory (where the public directory is run these commands:
sudo chmod og-w public/sites/default/settings.php
sudo chmod og-w public/sites/default
by Andrew Ault, on June 23rd, 2010
I wanted a command language translator that can be used in bash shell scripts. There are a couple of options available, but none that were versatile enough. A little research resulted in finding that Google Translate offered what I wanted and that there was a JSON interface, which I could use with Perl’s JSON module.
I added a few niceties such as help, multiple source options and a listing of available languages.
Entering:
tranny -t de ‘I am a citizen of Berlin’
gets this result:
Ich bin ein Bürger von Berlin
and entering:
tranny ‘Ich bin ein Bürger von Berlin’
results:
I am a citizen of Berlin
I created a Google Code project for tranny at: http://code.google.com/p/tranny/
If you need a scriptable translator, give it a try. If you run into trouble or would like to suggest changes, leave comments here or at the project.
#!/usr/bin/perl
#
# what: tranny, a language translator
# project: https://code.google.com/p/tranny/
# copyright: Copyright 2010, Andrew Ault
# license: This content is released under the http://code.google.com/p/tranny/wiki/license MIT License.
#
# Uses the JSON module from CPAN. To install: "sudo cpan JSON"
#
use strict;
use warnings;
use POSIX;
use Getopt::Std;
use JSON;
use LWP;
require 'sys/ioctl.ph';
die "no TIOCGWINSZ " unless defined &TIOCGWINSZ;
my $original;
my $winsize;
my $has_tty = 1;
my ( $screen_rows, $screen_cols, $screen_xpixels, $screen_ypixels );
my %languages = (
'afrikaans' => 'af',
'albanian' => 'sq',
'amharic' => 'am',
'arabic' => 'ar',
'armenian' => 'hy',
'azerbaijani' => 'az',
'basque' => 'eu',
'belarusian' => 'be',
'bengali' => 'bn',
'bihari' => 'bh',
'breton' => 'br',
'bulgarian' => 'bg',
'burmese' => 'my',
'catalan' => 'ca',
'cherokee' => 'chr',
'chinese' => 'zh',
'chinese simp' => 'zh-cn',
'chinese trad' => 'zh-tw',
'corsican' => 'co',
'croatian' => 'hr',
'czech' => 'cs',
'danish' => 'da',
'dhivehi' => 'dv',
'dutch' => 'nl',
'english' => 'en',
'esperanto' => 'eo',
'estonian' => 'et',
'faroese' => 'fo',
'filipino' => 'tl',
'finnish' => 'fi',
'french' => 'fr',
'frisian' => 'fy',
'galician' => 'gl',
'georgian' => 'ka',
'german' => 'de',
'greek' => 'el',
'gujarati' => 'gu',
'haitian creole' => 'ht',
'hebrew' => 'iw',
'hindi' => 'hi',
'hungarian' => 'hu',
'icelandic' => 'is',
'indonesian' => 'id',
'inuktitut' => 'iu',
'irish' => 'ga',
'italian' => 'it',
'japanese' => 'ja',
'javanese' => 'jw',
'kannada' => 'kn',
'kazakh' => 'kk',
'khmer' => 'km',
'korean' => 'ko',
'kurdish' => 'ku',
'kyrgyz' => 'ky',
'lao' => 'lo',
'latin' => 'la',
'latvian' => 'lv',
'lithuanian' => 'lt',
'luxembourgish' => 'lb',
'macedonian' => 'mk',
'malay' => 'ms',
'malayalam' => 'ml',
'maltese' => 'mt',
'maori' => 'mi',
'marathi' => 'mr',
'mongolian' => 'mn',
'nepali' => 'ne',
'norwegian' => 'no',
'occitan' => 'oc',
'oriya' => 'or',
'pashto' => 'ps',
'persian' => 'fa',
'polish' => 'pl',
'portuguese' => 'pt',
'punjabi' => 'pa',
'quechua' => 'qu',
'romanian' => 'ro',
'russian' => 'ru',
'sanskrit' => 'sa',
'scots_gaelic' => 'gd',
'serbian' => 'sr',
'sindhi' => 'sd',
'sinhalese' => 'si',
'slovak' => 'sk',
'slovenian' => 'sl',
'spanish' => 'es',
'sundanese' => 'su',
'swahili' => 'sw',
'swedish' => 'sv',
'syriac' => 'syr',
'tajik' => 'tg',
'tamil' => 'ta',
'tatar' => 'tt',
'telugu' => 'te',
'thai' => 'th',
'tibetan' => 'bo',
'tonga' => 'to',
'turkish' => 'tr',
'ukrainian' => 'uk',
'urdu' => 'ur',
'uzbek' => 'uz',
'uighur' => 'ug',
'vietnamese' => 'vi',
'welsh' => 'cy',
'yiddish' => 'yi',
'yoruba' => 'yo',
);
# get window size for country listing
open( TTY, "+</dev/tty" ) or $has_tty = 0;
if ($has_tty) {
unless ( ioctl( TTY, &TIOCGWINSZ, $winsize = '' ) ) {
die sprintf "$0: ioctl TIOCGWINSZ (%08x: $!)\n", &TIOCGWINSZ;
}
( $screen_rows, $screen_cols, $screen_xpixels, $screen_ypixels ) = unpack( 'S4', $winsize );
}
getopts( 'f:t:o:hl', \my %opts );
if ( defined $opts{h} && $opts{h} == 1 ) { usage() }
if ( defined $opts{l} && $opts{l} == 1 ) { usage() }
my ( $from, $to ) = ( $opts{f}, $opts{t} );
if ( !defined $opts{f} ) { $from = ''; }
if ( !defined $opts{t} ) { $to = 'en'; }
# if text was passed on the command line
if ( defined $ARGV[0] ) {
$original = $ARGV[0];
# text is from a file
} elsif ( defined $opts{o} ) {
# slurp from file
open FILE, $opts{o} or die "-o argument: couldn't open file: $!";
local $/ = undef;
$original = <FILE>;
close FILE;
# text is from STDIN
} else {
# slurp STDIN
local $/ = undef;
$original = <STDIN>;
}
my $ua = LWP::UserAgent->new;
$ua->agent("PGDict/1.0");
my $request =
HTTP::Request->new( GET => "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&langpair=$from|$to&q=$original" );
my $response = $ua->request($request);
if ( $response->is_success ) {
my $perl_res = from_json( $response->content );
if ( $perl_res->{'responseStatus'} eq '200' ) {
print $perl_res->{'responseData'}->{'translatedText'} . "\n";
} else {
warn "error " . $perl_res->{'responseDetails'} . "\n";
}
} else {
print $response->status_line . "\n";
}
sub usage {
print "usage: ";
print "\ttranny -f language_code -t language_code [original text]\n\n";
print "-f language_code (optional)\n\n";
print "-t language_code (optional)\n\n";
print "-o original_file (optional)\n\n";
print "-h this help\n\n";
print "-l language list\n\n";
print "Tranny uses Google Translate and requires an Internet connection to work.\n";
print "Text is translated from STDIN, from the command line or a file with -o.\n\n";
print "By default,the 'from' language is automatically detected and translated to English (en).\n\n";
if ( defined $opts{l} && $opts{l} == 1 ) { list_languages() }
exit;
}
sub list_languages {
my $num_columns = ceil( $screen_cols / 23 );
my $num_rows = ceil ( keys(%languages) /$num_columns );
my $row = 0;
my $col = 0;
my @formatted_languages = ( );
foreach my $key ( sort ( keys(%languages) ) ) {
$row++;
$formatted_languages[$col][$row] = sprintf( "%-14s %-6s", $key, $languages{$key} );
if ( $row == $num_rows ){
$row = 0;
$col++;
}
}
for ($row = 0; $row <= $num_rows; $row++) {
for ($col = 0; $col <= $num_columns; $col++) {
if ( defined $formatted_languages[$col][$row] ){
print $formatted_languages[$col][$row];
}
}
print "\n";
}
}
by Andrew Ault, on June 10th, 2010
It’s a WYSIWYG world. After all, we’re nearly in the future, which I define as 2019, the year Rick Deckard chases down replicants in the Blade Runner. Still no flying cars, which is disappointing. Even so, we have Steve Jobs, so the future coming, right?
GUI everything isn’t all that it could be. For many, many tasks, it is more expeditious to open a terminal and get a bash prompt. CLI. Character. Text. It could be green on black, or it would be a rainbow on white, but it is not different from a Televideo terminal, or a Teletype for that matter. As good as the Bourne Again Shell is, it is not graphical or fancy.
What it is is efficient. For a sharp mind and one given to efficiency, the terminal is power. Want to replace frick with frack in 800 HTML files?
find ~/web/project3 -name '*.php' | xargs perl -pi -e 's/frick/frack/g'
Bam. Done.
This is why I have 3 terminals open right now. One is connected to a server somewhere in Texas. I just fixed some text on a site with a command much like the one above.
But you already knew all that. You Googled and found this page, so you are already 1337 or whatnot. How about some word power on the command line?
Install some packages
This will install the packages we will use on an Ubuntu or Debian system. For other distributions, you will need to use your distributions package system.
To install on Ubuntu or Debian, just install the needed APT packages:
sudo aptitude -y install wordnet wamerican-large curl wget an
Definitions
This grabs a definition for a word from dict.org. For “unusual” for example:
curl --stderr /dev/null dict://dict.org/d:unusual | sed '/^[.,0-9].*$/d'
Which returns:
Unusual \Un*u"su*al\, a.
Not usual; uncommon; rare; as, an unusual season; a person of
unusual grace or erudition. -- {Un*u"su*al*ly}, adv. --
{Un*u"su*al*ness}, n.
[1913 Webster]
As you can see you are using curl to request a definition for “unusual”, then using sed to filter the results, to exclude extra stuff you don’t want. You could just enter “curl dict://dict.org/d:unusual” for the raw deal. Good on ya.
You can turn this into a script:
#! /bin/bash
# display definition of a word
#
curl --stderr /dev/null dict://dict.org/d:$1 | sed '/^[.,0-9].*$/d'
Save that in a file called “def” and run “chmod +x def” to make it executable. Then “def unusual” will return the same definition. You just created your own tool. You rock.
Wordnet
How about more power? Princeton has a project called Wordnet, which organizes nouns, verbs, adjectives and adverbs into set of “cognitive synonyms” and provides tools to use this data. With Wordnet, synonyms, antonyms and other lexical relations can be found for a given word.
To show a definition, (still using “unusual” as an example):
wn unusual -over
Here’s the output:
Overview of adj unusual
The adj unusual has 3 senses (first 3 from tagged texts)
1. (24) unusual -- (not usual or common or ordinary; "a scene of unusual beauty"; "a man of unusual ability"; "cruel and unusual punishment"; "an unusual meteorite")
2. (1) strange, unusual -- (being definitely out of the ordinary and unexpected; slightly odd or even a bit weird; "a strange exaltation that was indefinable"; "a strange fantastical mind"; "what a strange sense of humor she has")
3. (1) unusual -- (not commonly encountered; "two-career families are no longer unusual")
This uses the “-over” option. Some other options are:
| -synsa |
adjective synonyms |
| -synsn |
noun synonyms |
| -synsr |
adverb synonyms |
| -antsa |
adjective antonymns |
| -antsn |
noun antonymns |
| -antsr |
adverb antonymns |
Wordnet is extensive and there are many more options, run “man wn” for more.
Crossword help
This is simply a use of grep to pattern match words in a word list file.
Use a regular expression to find a word. In quotes, start your pattern with a “^” character and end with a “$” character. Use a period “.” for each unknown character.
grep '^.a...f.c.n...$' /usr/share/dict/words
Magnificent!
Rhyming
This uses the rhyme project, which provides a rhyming dictionary for the command line.
To get, build and install rhyme on your system:
sudo aptitude -y install build-essential libgdbm-dev libreadline-dev
cd ~
DIR="src" && [ -d "$DIR" ] || mkdir "$DIR"
cd src
wget http://softlayer.dl.sourceforge.net/project/rhyme/rhyme/0.9/rhyme-0.9.tar.gz
tar -xzf rhyme-0.9.tar.gz
cd rhyme-0.9
make
sudo make install
Holy smokes, you just built software! There is no stopping you. To find a rhyme, using “house” as an example:
rhyme house
House rhymes! Lots of them.
Anagrams
Anagrams are pretty much pure word fun. It is fun to see what an anagram of your name is.
Print single-word anagrams of “andrew”:
an -l 1 andrew
Just call me the wander warden.
by Andrew Ault, on June 8th, 2010
Dropbox is so useful! Wouldn’t it be great to have that same convenience and function for your user account on your server, just like you have on your workstation?
This has been tested on Ubuntu Lucid and Jaunty. This procedure will create a system with:
- Install a separate Dropbox client (daemon) for individual users
- Each user has a separate Dropbox account
- All the daemons will be managed together with normal daemon controls
This allows individuals to have their own Dropbox accounts, each with a separate process syncing their individual ~/Dropbox directory. With one command an admin can start or stop all the daemons at once.
Install Prerequisites
Later, we’ll need to read a sqlite3 database record, so install sqlite3.
sudo aptitude -y install sqlite3
Install Dropbox client for an individual user
This step is repeated for each user that wants a Dropbox client. Start by setting up your own account, then repeat for each user. This is run with a user’s own account. The changes made all take place in their home directory.
First, determine whether you have 32-bit or 64-bit Ubuntu Server. You must install the correct version, either 32 or 64 bit or it will not work. The following command will tell you which is installed:
uname -a | grep '_64' >/dev/null && echo 'A 64-bit OS is installed'; uname -a | grep '_64' >/dev/null || echo 'A 32-bit OS is installed'
Run the correct installation, based on whether a 32-bit or 64-bit OS is installed.
32-bit installation:
cd ~
wget -O dropbox.tar.gz http://www.dropbox.com/download/?plat=lnx.x86
tar -zxof dropbox.tar.gz
or
64-bit installation:
cd ~
wget -O dropbox.tar.gz http://www.dropbox.com/download/?plat=lnx.x86_64
tar -zxof dropbox.tar.gz
Link user’s Dropbox client to their Dropbox account:
wget http://dl.dropbox.com/u/6995/dbmakefakelib.py
python dbmakefakelib.py
The above will run for a little while without printing anything, then print “dropboxd ran for 15 seconds without quitting – success?”. When it does so, press control-c twice. Yes, it is unusual. What this does is populate a sqlite3 database with an ID from the Dropbox server. Next, we’ll extract that code and use it to link your Dropbox user account with this CLI Dropbox client instance.
On the server, via SSH
Get the URL with:
echo https://www.dropbox.com/cli_link?host_id=`echo '.dump config' | sqlite3 ~/.dropbox/dropbox.db | grep host_id | cut -d \' -f 4 | python -c 'print raw_input().decode("base64")' | grep '^V' | cut -b 2-`
On your local machine, in a web browser
Copy the URL that the above printed and paste it into a web browser. When you do so, Dropbox will register your client on the server with your Dropbox account.
At this point, Dropbox will not be quite working yet. The next steps will take care of that.
Create Dropbox daemon control
The next task is to create a system to start and stop the dropbox daemon for each user on the system that has Dropbox installed for his/her user account. The following daemon init script was lifted from: http://wiki.dropbox.com/TipsAndTricks/TextBasedLinuxInstall.
Of course, this is for the use of the system admin. This creates a normal daemon init start/stop script and installs it so the Dropbox daemons are started when the system boots. The admin can also use this to control the Dropbox daemons manually.
sudo vi /etc/init.d/dropbox
Paste in the following code. Then, modify line 3, replacing “user1 user2″ with your username. For future reference, additional user’s Dropbox daemons can be controlled with this one script – add additional username separated with spaces.
# dropbox service
# separate usernames in the following line with spaces.
DROPBOX_USERS="user1 user2"
DAEMON=.dropbox-dist/dropbox
start() {
echo "Starting dropbox..."
for dbuser in $DROPBOX_USERS; do
HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
if [ -x $HOMEDIR/$DAEMON ]; then
HOME="$HOMEDIR" start-stop-daemon -b -o -c $dbuser -S -u $dbuser -x $HOMEDIR/$DAEMON
fi
done
}
stop() {
echo "Stopping dropbox..."
for dbuser in $DROPBOX_USERS; do
HOMEDIR=`getent passwd $dbuser | cut -d: -f6`
if [ -x $HOMEDIR/$DAEMON ]; then
start-stop-daemon -o -c $dbuser -K -u $dbuser -x $HOMEDIR/$DAEMON
fi
done
}
status() {
for dbuser in $DROPBOX_USERS; do
dbpid=`pgrep -u $dbuser dropbox`
if [ -z $dbpid ] ; then
echo "dropboxd for USER $dbuser: not running."
else
echo "dropboxd for USER $dbuser: running (pid $dbpid)"
fi
done
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload|force-reload)
stop
start
;;
status)
status
;;
*)
echo "Usage: /etc/init.d/dropbox {start|stop|reload|force-reload|restart|status}"
exit 1
esac
exit 0
Make the init script executable and restart the daemon:
sudo chmod +x /etc/init.d/dropbox
sudo /etc/init.d/dropbox restart
When you restart the daemon, it will be running correctly linked to your Dropbox account. It will create a Dropbox directory in your home directory and will start to populate it with files you have on Dropbox.
Have the daemon(s) run automatically at boot time:
sudo update-rc.d dropbox defaults
The above correctly copies links as needed so the daemon(s) start when the server boots.
Managing the Daemons
A separate daemon will be run for each user that has the Dropbox client installed – with only one command. This makes it easy for individual users to have separate Dropbox accounts, each syncing to ~/Dropbox for their user account. Here are the commands to manage these daemons:
Start Dropbox services for all users:
sudo /etc/init.d/dropbox start
Stop Dropbox services for all users:
sudo /etc/init.d/dropbox stop
Restart Dropbox services for all users:
sudo /etc/init.d/dropbox restart
Get service status for each user Dropbox service:
sudo /etc/init.d/dropbox status
by Andrew Ault, on June 7th, 2010
This is a copy of the script I use to create an instance of a Kohana project. It works on Ubuntu server (tested with Lucid). Adapt it to your own needs, of course.
#! /bin/bash
CODE="upc"
OWNER="myusername"
GROUP="mygroupname"
KOHANA_URL="http://dev.kohanaframework.org/attachments/download/1355/kohana-v2.3.4.zip"
KOHANA_ZIP="kohana-v2.3.4.zip"
SITESROOT="/usr/local/var/www"
URL="www.mysite.com"
# database.php parameters
DBUSER="mydbuser"
DBPWD="mydbpasswd"
DBHOST="localhost"
DBDATABASE="mydbname"
#
# make sure this is run as root
#
if [[ $UID -ne 0 ]]; then
echo "Not running as root"
exit
fi
if [ ! -d $SITESROOT ]; then
echo "$SITESROOT directory does not exist"
exit
fi
#
# change to the siteroot directory
#
cd $SITESROOT
if [ -d $URL ]; then
echo "$SITESROOT/$URL directory already exists"
exit
fi
mkdir $URL
chown $OWNER:$GROUP $URL
cd $URL
#
# create kohana directory
#
echo creating kohana directory
mkdir kohana
chown $OWNER:$GROUP kohana
chmod g+ws kohana
wget "$KOHANA_URL"
unzip "$KOHANA_ZIP"
rm "$KOHANA_ZIP"
chown -R $OWNER:$GROUP kohana
chmod -R g+w kohana
chown $OWNER:www-data kohana/application/logs
rm kohana/example.htaccess
rm kohana/install.php
rm kohana/Kohana\ License.html
rm kohana/kohana.png
#
# create public directory
#
echo creating public directory
mkdir public
chown $OWNER:$GROUP public
chmod g+ws public
mv kohana/index.php public/
sed -i "s\\kohana_application =.*$\\kohana_application = '../kohana/application';\\" public/index.php
sed -i "s\\kohana_modules =.*$\\kohana_modules = '../kohana/modules';\\" public/index.php
sed -i "s\\kohana_system =.*$\\kohana_system = '../kohana/system';\\" public/index.php
echo "# turn on URL rewriting
RewriteEngine On
RewriteBase /
" > public/.htaccess
#
# create Apache logs directory
#
echo creating logs directory
mkdir logs
chown $OWNER:www-data logs
chmod g+ws logs
touch logs/error.log
touch logs/combined.log
#
# create utilties directory
#
echo creating utilties directory
mkdir utilities
chown $OWNER:$GROUP utilities
chmod g+ws utilities
#
# modify kohana/application/config/config.php
#
echo modifying kohana/application/config/config.php
sed -i "s\\'/kohana/'\\'$URL/'\\" kohana/application/config/config.php
#
# create kohana/application/config/database.php
#
echo creating kohana/application/config/database.php
cp kohana/system/config/database.php kohana/application/config/database.php
chown $OWNER:$GROUP kohana/application/config/database.php
chmod g+w kohana/application/config/database.php
sed -i "s\\'user'.*$\\'user' => '$DBUSER',\\" kohana/application/config/database.php
sed -i "s\\'pass'.*$\\'pass' => '$DBPWD',\\" kohana/application/config/database.php
sed -i "s\\'host'.*$\\'host' => '$DBHOST',\\" kohana/application/config/database.php
sed -i "s\\'database'.*$\\'database' => '$DBDATABASE',\\" kohana/application/config/database.php
#
# create kohana/application/config/routes.php
#
echo creating kohana/application/config/routes.php
cp kohana/system/config/routes.php kohana/application/config/routes.php
chown $OWNER:$GROUP kohana/application/config/routes.php
chmod g+w kohana/application/config/routes.php
sed -i "s\\'welcome'\\'/index'\\" kohana/application/config/routes.php
#
# copy kohana/application/config/profiler.php
#
echo creating kohana/application/config/profiler.php
cp kohana/system/config/profiler.php kohana/application/config/profiler.php
chown $OWNER:$GROUP kohana/application/config/profiler.php
chmod g+w kohana/application/config/profiler.php
#
# create Apache virtual site
#
echo creating Apache virtual site file in /etc/apache2/sites-available
echo "<VirtualHost *:80>
ServerName $URL
DocumentRoot $SITESROOT/$URL/public
DirectoryIndex index.php
LogLevel warn
ErrorLog $SITESROOT/$URL/logs/error.log
CustomLog $SITESROOT/$URL/logs/combined.log combined
</VirtualHost>
" > /etc/apache2/sites-available/$URL
#
# finish up
#
echo "enable site with: sudo a2ensite $URL"
echo "restart Apache with: sudo /etc/init.d/apache2 restart"
by Andrew Ault, on May 27th, 2010
If you have a process that need to run in the background, creating a daemon is the key. Some examples of what your can do:
- Wait for video files to be dropped via FTP and then process them
- Check for a recurring problem and report it to admin
- Monitor system load and report to the admin at a certain threshold
In this post we’ll take a look at how to create a Perl daemon script and how to get it to run as a daemon whenever the system is started. For the Perl script, a template will be created and for control of the daemon process, standard Debian facilities will be used.
What will be covered:
- Create the start/stop script in /etc/init.d
- Create the Perl daemon script in /usr/bin/
- Create the log file in /var/log/
- Get the daemon to start automatically when the system boots
- How to manage your daemon
The start/stop script starts and stops the daemon. It is also used by the system to start your daemon when the system starts.
The Perl daemon script contains your custom Perl code to run in the background. It executes your code every x number of seconds.
Create the Start/stop Script
Happily, Debian and therefore Ubuntu, supplies a template that uses the Debian start-stop-daemon command to start and stop daemons. It is only necessary to copy this template to a new file of the correct name and modify it for the purpose.
These scripts reside in the /etc/init.d/ directory. The script you create will have the exact same filename as the Perl daemon script you will create and the name cannot have a file extension – just the bare name.
Substitute your daemon’s name for “mydaemon” in the examples.
sudo cp /etc/init.d/skeleton /etc/init.d/mydaemon
sudo chmod +x /etc/init.d/mydaemon
sudo vi /etc/init.d/mydaemon
Make changes to your start/stop script. Locate the header in the script:
PATH=/sbin:/usr/sbin:/bin:/usr/bin
DESC="Description of the service"
NAME=daemonexecutablename
DAEMON=/usr/bin/$NAME
DAEMON_ARGS="--options args"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
Change the content of DESC to a meaningful description of your daemon.
Change the content for NAME to the exact name of your daemon.
Create the Perl Daemon Script
Use the Perl Daemon Script Template located at the end of this post. In the following, the script template is copied to /usr/bin/where executables specific to this system are located. Actually, you would copy the template to your home directory, make the changes and then copy the daemon script to /usr/bin/.
sudo cp daemon_template.pl /usr/bin/mydaemon
sudo vi /usr/bin/mydaemon
sudo chmod +x /usr/bin/mydaemon
The script template is commented with the changes that are needed. In particular, change the value of $daemonName to the exact name of your new daemon. Ten, add your custom code where # do something appears.
Prerequisites
The script uses File::Pid and POSIX from CPAN, so you’ll need to install that module:
sudo aptitude install cpan
sudo cpan POSIX
sudo cpan File::Pid
Create the Log File
We’ll create the log file so it has the correct ownership and permissions. The log fie has the daemon name appended with “.log”. It is located in the /var/log/ directory.
sudo touch /var/log/mydaemon.log
sudo chmod 640 /var/log/mydaemon.log
sudo chown root:adm /var/log/mydaemon.log
The permissions and ownership change allow adm group members to read the log, per convention. Add yourself to adm group to view logs.
Run your Daemon
As you have created a standard start/stop script for your daemon, you can start it the standard way:
sudo /etc/init.d/mydaemon start
Stop your Daemon
Similarly, your daemon is easy to stop and restart:
sudo /etc/init.d/mydaemon stop
sudo /etc/init.d/mydaemon restart
Automatically Start your Daemon
This command tells your system to start your daemon automatically when the system starts:
update-rc.d mydaemon defaults 99
To keep your daemon from starting, if needed:
update-rc.d -f mydaemon remove
Managing Daemons
List running daemons
Well-behaved daemons generally create a PID file in /var/run/. List that directory and your have a fair list of running daemons.
sudo ls /var/run/
Get PID for a particular daemon
If you know the name of a daemon and want to see if it is running and get its PID, use pgrep.
pgrep mydaemon
Also useful is the ps command, which list processes.
ps aux
The aux switch limits output to processes not associated with a terminal.
Show programs running as daemons
This line attempts to show programs that are running as daemons.
which `ps aux | cut -c 66- | cut -d\ -f 1` | sort | uniq
The Perl Daemon Script Template
#!/usr/bin/perl -w
#
# mydaemon.pl by Andrew Ault, www.andrewault.net
#
# Free software. Use this as you wish.
#
# Throughout this template "mydaemon" is used where the name of your daemon should
# be, replace occurrences of "mydaemon" with the name of your daemon.
#
# This name will also be the exact name to give this file (WITHOUT a ".pl" extension).
#
# It is also the exact name to give the start-stop script that will go into the
# /etc/init.d/ directory.
#
# It is also the name of the log file in the /var/log/ directory WITH a ".log"
# file extension.
#
# Replace "# do something" with your super useful code.
#
# Use "# logEntry("log something");" to log whatever your need to see in the log.
#
use strict;
use warnings;
use POSIX;
use File::Pid;
# make "mydaemon.log" file in /var/log/ with "chown root:adm mydaemon"
# TODO: change "mydaemon" to the exact name of your daemon.
my $daemonName = "mydaemon";
#
my $dieNow = 0; # used for "infinte loop" construct - allows daemon mode to gracefully exit
my $sleepMainLoop = 120; # number of seconds to wait between "do something" execution after queue is clear
my $logging = 1; # 1= logging is on
my $logFilePath = "/var/log/"; # log file path
my $logFile = $logFilePath . $daemonName . ".log";
my $pidFilePath = "/var/run/"; # PID file path
my $pidFile = $pidFilePath . $daemonName . ".pid";
# daemonize
use POSIX qw(setsid);
chdir '/';
umask 0;
open STDIN, '/dev/null' or die "Can't read /dev/null: $!";
open STDOUT, '>>/dev/null' or die "Can't write to /dev/null: $!";
open STDERR, '>>/dev/null' or die "Can't write to /dev/null: $!";
defined( my $pid = fork ) or die "Can't fork: $!";
exit if $pid;
# dissociate this process from the controlling terminal that started it and stop being part
# of whatever process group this process was a part of.
POSIX::setsid() or die "Can't start a new session.";
# callback signal handler for signals.
$SIG{INT} = $SIG{TERM} = $SIG{HUP} = \&signalHandler;
$SIG{PIPE} = 'ignore';
# create pid file in /var/run/
my $pidfile = File::Pid->new( { file => $pidFile, } );
$pidfile->write or die "Can't write PID file, /dev/null: $!";
# turn on logging
if ($logging) {
open LOG, ">>$logFile";
select((select(LOG), $|=1)[0]); # make the log file "hot" - turn off buffering
}
# "infinite" loop where some useful process happens
until ($dieNow) {
sleep($sleepMainLoop);
# TODO: put your custom code here!
# do something
# logEntry("log something"); # use this to log whatever you need to
}
# add a line to the log file
sub logEntry {
my ($logText) = @_;
my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime(time);
my $dateTime = sprintf "%4d-%02d-%02d %02d:%02d:%02d", $year + 1900, $mon + 1, $mday, $hour, $min, $sec;
if ($logging) {
print LOG "$dateTime $logText\n";
}
}
# catch signals and end the program if one is caught.
sub signalHandler {
$dieNow = 1; # this will cause the "infinite loop" to exit
}
# do this stuff when exit() is called.
END {
if ($logging) { close LOG }
$pidfile->remove if defined $pidfile;
}
UPDATE:
Looks like there’s a page in French about this post: http://www.duncane.net/2011/02/25/perl-daemon/. Merci Duncane!
|
|