Monthly Archives: February 2010

My remote Subversion dump/tar/rotating file Perl script

This is the script I use to SSH remotely dump Subversion repositories on various servers for which I am responsible.

Before you can use this script, you need to set up SSH so your local cron can access the remote servers without a password.

One thing to note about this script is that it automatically rotates the archived dump files; keeping a fie for the 1st of the week on a month, 1st of the month and 1st of the year.

see: Using Public/Private Key Pairs with SSH

Then, just modify the script for your database/servers (the block @ about line 22).

This will create a series of files over time with daily/weekly/monthly Subversion dump backup tar files. The point is not so much to have every state of every repository, but to grab the daily changes without clobbering the last know good one. More is better, no?

#!/usr/bin/perl -w
#
# rtar_svn.pl
#
# by Andrew Ault www.andrewault.net
#
# No arguments. The program is to be modified to include each Subversion repository to be archived.
#
# Saves a tar of a remote Subversion dump in a rotating file.
#
# Of course you have to have SSH authentication already set up.
#
# This get cron'd daily on my local workstation.
#
use strict;
use warnings;

use DateTime;

my $fileError;
my $jobError  = 0;
my $jobErrors = "";
my $result;

# Specify a data block for each remote repository to be archived.
my %dumpJobs = (
				 'servername-repositoryname' => {
							'remoteServer' => 'servername',
							'repository'     => 'repositoryname',
							'dumpFilename' => 'servername-repositoryname.dump.svn',
							'svnDumpCmd' => '/usr/bin/svnadmin dump', # find svnadmin on your server
							'tarCmd'       => '/bin/tar', # find tar on your server
				 },
				 'servername-repositoryname2' => {
							'remoteServer' => 'servername',
							'repository'     => 'repositoryname2',
							'dumpFilename' => 'servername-repositoryname2.dump.svn',
							'svnDumpCmd' => '/usr/bin/svnadmin dump',
							'tarCmd'       => '/bin/tar',
				 },
);

# Process each specified repository dump/archive job.
for my $dumpJob ( sort keys %dumpJobs ) {
	$fileError = 0;
	my $tarballFilename = "$dumpJobs{$dumpJob}{'dumpFilename'}-" . tarDateSegment() . ".tgz";
	my $svnDumpCmd    = $dumpJobs{$dumpJob}{'svnDumpCmd'};
	my $tarCmd          = $dumpJobs{$dumpJob}{'tarCmd'};
	print "$dumpJob\n";

	my $dumpCommand = "ssh $dumpJobs{$dumpJob}{'remoteServer'} '$svnDumpCmd ";
	$dumpCommand .= "/var/lib/svn/$dumpJobs{$dumpJob}{'repository'} > $dumpJobs{$dumpJob}{'dumpFilename'}'";
	print $dumpCommand . "\n";
	$result = system($dumpCommand );
	if ($result) { $fileError = 1; }

	if ( !$fileError ) {
		my $remoteMakeTarball = "ssh $dumpJobs{$dumpJob}{'remoteServer'} '$tarCmd ";
		$remoteMakeTarball .= "cvfz $tarballFilename $dumpJobs{$dumpJob}{'dumpFilename'}'";
		print $remoteMakeTarball . "\n";
		$result = system($remoteMakeTarball );
		if ($result) { $fileError = 1; }
	}

	if ( !$fileError ) {
		my $downloadCommand = "scp $dumpJobs{$dumpJob}{'remoteServer'}:$tarballFilename .";
		print $downloadCommand . "\n";
		$result = system($downloadCommand );
		if ($result) { $fileError = 1; }
	}

	if ($fileError) {
		$jobError = 1;
		$jobErrors .= "$dumpJob ";
	}
}
if ($jobError) {
	warn "Errors were encountered: $jobErrors\n";
	exit(1);
}

sub tarDateSegment {
	my $dt = DateTime->now();

	my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime(time);
	$year += 1900;
	my $dateTime = sprintf "%4d-%02d-%02d %02d:%02d:%02d", $year, $mon + 1, $mday, $hour, $min, $sec;
	my $date     = sprintf "%4d-%02d-%02d",                $year, $mon + 1, $mday;
	my @weekdays = qw( sun mon tue wed thu fri sat );
	my $weekday  = $weekdays[$wday];
	my @months   = qw( jan feb mar apr may jun jul aug sep oct nov dec );
	my $month    = $months[$mon];

	my $weekOfMonth = $dt->week_of_month;

	my $dateTar = "";

	# if the first day of the year, set $dateTar like: 2009-1st
	if ( $yday == 1 ) {
		$dateTar = "$year-1st";
	}

	# if the first day of the month, set $dateTar like: feb-1st
	elsif ( $mday == 1 ) {
		$dateTar = "$month-1st";
	}

	# if the first day of the week, set $dateTar like: mon-1
	# where the number is the week of the month number
	elsif ( $wday == 1 ) {
		$dateTar = "$weekday-$weekOfMonth";
	}

	# otherwise, set the $dateTar like: mon
	else {
		$dateTar = "$weekday";
	}

	# $sec      seconds          54
	# $min      monutes          37
	# $hour     hour             11
	# $mon      month            4
	# $year     year             2009
	# $wday     weekday          3
	# $yday     day of the year  146
	# $isdst    is DST           1
	# $weekday  day of the week  wed
	# $month    month            may
	# $dateTime date and time    2009-05-27 11:37:54
	# $date     date             2009-05-27
	return $dateTar;
}

=head1 NAME

rtar_svn.pl - Andrew's remote Subversion repository archive program.

=head1 SYNOPSIS

    use: rtar_svn.pl

=head1 DESCRIPTION

This is a program I wrote to SSH/dump/tar/download/rotate archives of Subversion repositories.

=over

=back

=head1 LICENSE

Use this as you will.

=head1 AUTHOR

Andrew Ault 

=cut

Installing Net::Amazon::S3 Perl module on an Ubuntu server

The following is the same on recent Ubuntu releases, including Karmic, Lucid and Maverick.

What will not work

There seems to be a problem if you install Net::Amazon::S3 from CPAN. This will not work:

sudo cpan Net::Amazon::S3

Just about every dependency in the world installs, but fails in the home stretch when XML::LibXML::XPathContext and XML::LibXML fail to install.

What will work

sudo aptitude install libnet-amazon-s3-perl
sudo cpan Net::Amazon::S3::Client

Test your install with this

After throwing some data into S3 with S3Fox, test your installation. You will need to set values for aws_access_key_id and aws_secret_access_key, of course.

#!/usr/bin/perl
use warnings;
use strict;
use Net::Amazon::S3;
use Net::Amazon::S3::Client;

my %s3_hash = (
				aws_access_key_id     => "XXXXXXXXXXXXXXXXX",
				aws_secret_access_key => "YYYYYYYYYYYYYYYYYYYYYYYYYY",
				retry                 => 1,
);

my $s3 = Net::Amazon::S3->new( \%s3_hash );
my $client = Net::Amazon::S3::Client->new( s3 => $s3 );

my @buckets = $client->buckets;
foreach my $bucket (@buckets) {
	print $bucket->name . "\n";
}

XAMMP erroneous error message

While setting up a test system for a new MVC PHP web project, I ran into a hiccup when I restarted Apache under XAMPP on my Mac (OSX).

Googling this error turns out not to be very helpful. It is shown as an error, but no solutions.

This is a hard-to-track-down XAMPP error because the error issued has nothing to do with the problem.

Here is the text from the error dialog, so Google et al can find it: “/Applications/XAMPP/xamppfiles/bin/apachectl: line 70: ulimit: open files: cannot modify limit: Invalid argument”.

I had simply created a typo in the CustomLog line in the httpd-vhosts.conf file.

This was in the httpd-vhosts.conf file in the code block sort like this:


    ServerName projectname.dev
    ServerAlias www.projectname.dev
    DocumentRoot "/Users/andrewault/www/my.projectname.com/public"
    ErrorLog "/Users/andrewault/www/my.projectname.com/logs/error.log"
    CustomLog "/Users/andrewault/www/my.projectname.com/logs/combined.log" common

The directory part of the CustomLog line was wrong, causing the error.

Emulating a Z80 and CP/M on Ubuntu Linux

Here is how to emulate a Z-80 processor running CP/M on Ubuntu Linux.

This method is very easy and achieves an excellent, easy to use and understand system. Essentially, the trick is to use a DOS-based emulator that works really well in a DOS emulator under Linux. I haven’t found a good Z80 emulator that runs directory under Linux.

To begin though, a mystery must be told. There, apparently was a fellow named Simon Cran in Australia who wrote a lovely CP/M Z-80 emulator for DOS. If you Google his name and “CPM” you can delve into the mysterious Simon Cran who created MyZ80 as shareware in the early nineties and then, seemingly vanished into ‘net anonymity.

I found that MyZ80 works well run in the dosemu DOS Emulator on Linux. I used to use this setup when I used Suse and it also works well on Ubuntu.

Install DOS Emulator

Install DOS Emulator:

sudo aptitude install dosemu

On my system dosemu has the equivalent of a DOS C: drive inside ~/.dosemu/drive_c/.

I have DOS in a Box installed as well as dosemu. When I run dosemu, it opens this window…seems to work fine.

Install MyZ80

Download MyZ80 from http://www.gaby.de/edownl.htm and unzip it into ~/.dosemu/drive_c/myz80/ – as illustrated, above. An easy way to do this is to plop the ZIP into ~/.dosemu/drive_c/ and then right-click myz80.zip and select Extract here.

Run MyZ80

In a Terminal, run:

dosemu

Then run MyZ80:

cd myz80
myz80

You will be greeted with this friendly text:

A couple of return key presses will then show how to import and export data into the files that Simon Cran uses for the CP/M drives:

The command to exit MyZ80 is exit. The command to exit dosemu is exitemu.

I have run Wordstar and Turbo Pascal using MyZ80, re-living my experience with my Kaypro 10…a machine I miss very much!

Some useful links

http://en.wikipedia.org/wiki/Digital_Research
http://www.cpm.z80.de/drilib.html
http://www.digitalresearch.biz/
http://www.retroarchive.org/cpm/lang/lang.htm
http://www.chiark.greenend.org.uk/~jacobn/cpm/cpmfiles.html

Have fun!