Tag Archives: admin

My remote Subversion dump/tar/rotating file Perl script

This is the script I use to SSH remotely dump Subversion repositories on various servers for which I am responsible.

Before you can use this script, you need to set up SSH so your local cron can access the remote servers without a password.

One thing to note about this script is that it automatically rotates the archived dump files; keeping a fie for the 1st of the week on a month, 1st of the month and 1st of the year.

see: Using Public/Private Key Pairs with SSH

Then, just modify the script for your database/servers (the block @ about line 22).

This will create a series of files over time with daily/weekly/monthly Subversion dump backup tar files. The point is not so much to have every state of every repository, but to grab the daily changes without clobbering the last know good one. More is better, no?

#!/usr/bin/perl -w
#
# rtar_svn.pl
#
# by Andrew Ault www.andrewault.net
#
# No arguments. The program is to be modified to include each Subversion repository to be archived.
#
# Saves a tar of a remote Subversion dump in a rotating file.
#
# Of course you have to have SSH authentication already set up.
#
# This get cron'd daily on my local workstation.
#
use strict;
use warnings;

use DateTime;

my $fileError;
my $jobError  = 0;
my $jobErrors = "";
my $result;

# Specify a data block for each remote repository to be archived.
my %dumpJobs = (
				 'servername-repositoryname' => {
							'remoteServer' => 'servername',
							'repository'     => 'repositoryname',
							'dumpFilename' => 'servername-repositoryname.dump.svn',
							'svnDumpCmd' => '/usr/bin/svnadmin dump', # find svnadmin on your server
							'tarCmd'       => '/bin/tar', # find tar on your server
				 },
				 'servername-repositoryname2' => {
							'remoteServer' => 'servername',
							'repository'     => 'repositoryname2',
							'dumpFilename' => 'servername-repositoryname2.dump.svn',
							'svnDumpCmd' => '/usr/bin/svnadmin dump',
							'tarCmd'       => '/bin/tar',
				 },
);

# Process each specified repository dump/archive job.
for my $dumpJob ( sort keys %dumpJobs ) {
	$fileError = 0;
	my $tarballFilename = "$dumpJobs{$dumpJob}{'dumpFilename'}-" . tarDateSegment() . ".tgz";
	my $svnDumpCmd    = $dumpJobs{$dumpJob}{'svnDumpCmd'};
	my $tarCmd          = $dumpJobs{$dumpJob}{'tarCmd'};
	print "$dumpJob\n";

	my $dumpCommand = "ssh $dumpJobs{$dumpJob}{'remoteServer'} '$svnDumpCmd ";
	$dumpCommand .= "/var/lib/svn/$dumpJobs{$dumpJob}{'repository'} > $dumpJobs{$dumpJob}{'dumpFilename'}'";
	print $dumpCommand . "\n";
	$result = system($dumpCommand );
	if ($result) { $fileError = 1; }

	if ( !$fileError ) {
		my $remoteMakeTarball = "ssh $dumpJobs{$dumpJob}{'remoteServer'} '$tarCmd ";
		$remoteMakeTarball .= "cvfz $tarballFilename $dumpJobs{$dumpJob}{'dumpFilename'}'";
		print $remoteMakeTarball . "\n";
		$result = system($remoteMakeTarball );
		if ($result) { $fileError = 1; }
	}

	if ( !$fileError ) {
		my $downloadCommand = "scp $dumpJobs{$dumpJob}{'remoteServer'}:$tarballFilename .";
		print $downloadCommand . "\n";
		$result = system($downloadCommand );
		if ($result) { $fileError = 1; }
	}

	if ($fileError) {
		$jobError = 1;
		$jobErrors .= "$dumpJob ";
	}
}
if ($jobError) {
	warn "Errors were encountered: $jobErrors\n";
	exit(1);
}

sub tarDateSegment {
	my $dt = DateTime->now();

	my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst ) = localtime(time);
	$year += 1900;
	my $dateTime = sprintf "%4d-%02d-%02d %02d:%02d:%02d", $year, $mon + 1, $mday, $hour, $min, $sec;
	my $date     = sprintf "%4d-%02d-%02d",                $year, $mon + 1, $mday;
	my @weekdays = qw( sun mon tue wed thu fri sat );
	my $weekday  = $weekdays[$wday];
	my @months   = qw( jan feb mar apr may jun jul aug sep oct nov dec );
	my $month    = $months[$mon];

	my $weekOfMonth = $dt->week_of_month;

	my $dateTar = "";

	# if the first day of the year, set $dateTar like: 2009-1st
	if ( $yday == 1 ) {
		$dateTar = "$year-1st";
	}

	# if the first day of the month, set $dateTar like: feb-1st
	elsif ( $mday == 1 ) {
		$dateTar = "$month-1st";
	}

	# if the first day of the week, set $dateTar like: mon-1
	# where the number is the week of the month number
	elsif ( $wday == 1 ) {
		$dateTar = "$weekday-$weekOfMonth";
	}

	# otherwise, set the $dateTar like: mon
	else {
		$dateTar = "$weekday";
	}

	# $sec      seconds          54
	# $min      monutes          37
	# $hour     hour             11
	# $mon      month            4
	# $year     year             2009
	# $wday     weekday          3
	# $yday     day of the year  146
	# $isdst    is DST           1
	# $weekday  day of the week  wed
	# $month    month            may
	# $dateTime date and time    2009-05-27 11:37:54
	# $date     date             2009-05-27
	return $dateTar;
}

=head1 NAME

rtar_svn.pl - Andrew's remote Subversion repository archive program.

=head1 SYNOPSIS

    use: rtar_svn.pl

=head1 DESCRIPTION

This is a program I wrote to SSH/dump/tar/download/rotate archives of Subversion repositories.

=over

=back

=head1 LICENSE

Use this as you will.

=head1 AUTHOR

Andrew Ault 

=cut

Perl Low Disk Space Warning Cron Script

This is a quick little script I wrote to warn me when disk space is getting low on a server I’m responsible for.

i just stuck this into a daily cron and now I know when to act!

#!/usr/bin/perl
#
# lowdiskspacewarning.pl
#
use strict;
use Filesys::DiskFree;

# init
my $sendmail = "/usr/lib/sendmail -t";

# file system to monitor
my $dirFilesystem = "/";
my $systemName = "putYourSystemNameHere";

# low diskspace warning threshhold
my $warningThreshhold=20 ; # in percent

# fs disk freespace
my $fsHandle = new Filesys::DiskFree;
$fsHandle->;df();
my $fsSpaceAvail = $fsHandle->;avail($dirFilesystem);
my $fsSpaceTotal = $fsHandle->;total($dirFilesystem);
my $fsSpaceUsed = $fsHandle->;used($dirFilesystem);
my $fsSpaceAvailPct = (($fsSpaceAvail) / ($fsSpaceAvail+$fsSpaceUsed)) * 100.0;

# email setup
my $emailTo='it@yourdomain.com';
my $emailFrom='root@yourdomain.com';
my $emailSubject="WARNING Low Disk Space for: $systemName";
my $emailBody = sprintf("WARNING Low Disk Space on '$systemName $dirFilesystem': %0.2f%%\n", $fsSpaceAvailPct);

# If free space is below threshhold, e-mail a warning message.
if ($fsSpaceAvailPct < $warningThreshhold) {
        open(MAIL, "|$sendmail");
        print MAIL "To: $emailTo\n";
        print MAIL "From: $emailFrom\n";
        print MAIL "Subject: $emailSubject\n";
        print MAIL $emailBody;
        close(MAIL);
}

Installing Trac and Subversion on Ubuntu Ibex

These are basically my notes from installing Trac and Subversion on an Ubuntu Intrepid Ibex server.

Install Software Packages

sudo aptitude install libapache2-mod-python libapache2-svn python-setuptools subversion python-subversion
sudo easy_install Trac

Create the Trac Environments Directory

sudo mkdir /var/lib/trac
sudo chown www-data:www-data /var/lib/trac

Setup Apache2

Using the default Ubuntu Apache virtual server setup, create a virtual server instance for
Trac. This single instance will be used for all Trac projects on the server.

sudo vi /etc/apache2/sites-available/trac.websitename.com

Your file will look a lot like the following. Use your website name instead of websitename.com and use a valid e-mail addressinstead of webmaster@websitename.com.


	ServerAdmin it@sitename.com
	ServerName trac.sitename.com
	DocumentRoot /hkm/var/www/trac.sitename.com/public
	LogLevel warn
	ErrorLog /hkm/var/www/trac.sitename.com/logs/error.log
	CustomLog /hkm/var/www/trac.sitename.com/logs/combined.log combined
	# DirectoryIndex index.html
 
	
		SetHandler mod_python
		PythonInterpreter main_interpreter
		PythonHandler trac.web.modpython_frontend
		PythonOption TracEnvParentDir /var/lib/trac
		PythonOption TracUriRoot /projects
		PythonOption PYTHON_EGG_CACHE /tmp
	
 
	# use the following for one authorization for all projects 
	# (names containing "-" are not detected):
	
		AuthType Basic
		AuthName "trac"
		AuthUserFile /path/htpasswd_filename_here
		Require valid-user
	

Now, use the Debian/Ubuntu utility a2ensite to create a link in apache2/sites-enabled to the file you just created in apache2/sites-available. Then, reload apache2 to enable the new subdomain.

sudo a2ensite trac
sudo  /etc/init.d/apache2 reload

You will also need to create a DNS A record to point to the server for this subdomain, but this is outside the scope of this blog entry.

Create Trac and Subversion Environments

Here we will make directories and create the Subversion repositories.

sudo mkdir /var/lib/svn
sudo mkdir /usr/share/trac
sudo svnadmin create /var/lib/svn/project1
sudo svnadmin create /var/lib/svn/project2

Set some permissions:

sudo chown -R www-data /var/lib/svn
sudo chown -R www-data /usr/share/trac
sudo  /etc/init.d/apache2 reload

Set up Trac

sudo mkdir /var/lib/trac

Initialize Trac for one of your projects.

sudo trac-admin /var/lib/trac/project1 initenv

The utility will ask for:

Project Name [My Project]>: project1
Database connection string [sqlite:db/trac.db]>:
Repository type [svn]>:
Path to repository [/path/to/repos]>: /var/lib/svn/project1

Repeat the above for the following, substituting the project code for each:

sudo trac-admin /var/lib/trac/project2 initenv
sudo chown -R www-data /var/lib/trac

Get Subversion Running

Modify /etc/apache2/mods-available/dav_svn.conf

sudo vi /etc/apache2/mods-available/dav_svn.conf
<Location /svn>
  DAV svn
 
  SVNParentPath /var/lib/svn
  SVNListParentPath On
 
  AuthType Basic
  AuthName "Subversion Repository"
  AuthUserFile /path/htpasswd_filename_here
 
  # To enable authorization via mod_authz_svn
  #AuthzSVNAccessFile /etc/apache2/dav_svn.authz
 
  <LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
  </LimitExcept> 
 
</Location>

Install More aptitude packages for Trac plugins

Install some packages that will be needed.

sudo aptitude install build-essential graphviz
sudo aptitude install enscript
sudo aptitude install htmldoc

Using easy_install to Install more Trac Plugins

Because these are installed, you will be able to turn these on easily as Trac administrator.

sudo easy_install http://svn.edgewall.org/repos/genshi/trunk/
sudo easy_install http://trac-hacks.org/svn/accountmanagerplugin/trunk
sudo easy_install http://trac-hacks.org/svn/customfieldadminplugin/0.11
sudo easy_install http://trac-hacks.org/svn/eclipsetracplugin/tracrpcext/0.10
sudo easy_install http://trac-hacks.org/svn/iniadminplugin/0.11
sudo easy_install http://trac-hacks.org/svn/masterticketsplugin/0.11
sudo easy_install http://trac-hacks.org/svn/pagetopdfplugin/0.10/
sudo easy_install http://trac-hacks.org/svn/progressmetermacro/0.11
sudo easy_install http://trac-hacks.org/svn/ticketdeleteplugin/0.11
sudo easy_install http://trac-hacks.org/svn/tracwysiwygplugin/0.11
sudo easy_install http://wikinotification.ufsoft.org/svn/trunk

Nautilus as Root in Ubuntu

Firstly, you can just run Nautilus as root from the command line with:

gksudo nautilus

The following procedure will add a root file browser to the Applications | System Tools menu.

I got this from http://ubuntuforums.org/archive/index.php/t-256998.html.

Create a text file with:

gksudo gedit /usr/share/applications/Nautilus-root.desktop

Add these lines to the file:

[Desktop Entry]
Name=File Browser (Root)
Comment=Browse the filesystem with the file manager
Exec=gksudo "nautilus --browser %U"
Icon=file-manager
Terminal=false
Type=Application
Categories=Application;System;

Open Nautilus with: Applications -> System Tools -> File Browser (Root)

An Alternative

This is from a comment by Cerebrux to this post:

1) Open Synaptic and install the “nautilus-gksu”

2) Logout- Login

3) And now do right-click on a folder and “open as administrator”.

P.S. If you want the “open terminal here” in your right-click, then just install “nautilus-open-terminal”

Firstly, you can just run Nautilus as root from the command line with:

gksudo nautilus

The following procedure will add a root file browser to the Applications | System Tools menu.

I got this from http://ubuntuforums.org/archive/index.php/t-256998.html.

Create a text file with:

gksudo gedit /usr/share/applications/Nautilus-root.desktop

Add these lines to the file:

[Desktop Entry]
Name=File Browser (Root)
Comment=Browse the filesystem with the file manager
Exec=gksudo "nautilus --browser %U"
Icon=file-manager
Terminal=false
Type=Application
Categories=Application;System;

Open Nautilus with: Applications -> System Tools -> File Browser (Root)

An Alternative

This is from a comment by Cerebrux to this post:

1) Open Synaptic and install the “nautilus-gksu”

2) Logout- Login

3) And now do right-click on a folder and “open as administrator”.

P.S. If you want the “open terminal here” in your right-click, then just install “nautilus-open-terminal”