Subversion repository creation script

words by Brian Racer

Here is a simple script I use to create subversion repositories and setup common hook scripts. This assumes you are making svn available through an HTTPS/Apache/DAV.

My directory svn structure looks like the following:

$ tree -L 2 /home/svn
 
/home/svn
|-- conf
|   |-- permissions.conf
|   `-- mailer.conf
|-- repos
|   |-- repo1
|   |-- repo2
|   `-- repo3 
`-- scripts
    |-- post-commit
    |-- pre-commit
    `-- svncreate.sh

There are 3 main directories:

repos – This is where the actual subversion repositories are stored
scripts – Various scripts, including the following creation script, and also various hook scripts that repositories can symlink to
conf – Various configuration files that might contain mailer configuration or repository permissions(both outside the scope of this article)

The following script will create the new repository, set proper user and permissions, symlink common pre and pos commit scripts, and then make the initial import that contains the trunk/branches/tags structure.

#!/bin/bash
 
REPOS_URL=https://svn.example.com
REPOS_PATH=/home/svn/repos
SCRIPTS_PATH=/home/svn/scripts
APACHE_USER=www-data
 
SVNADMIN=`which svnadmin`
EXPECTED_ARGS=1
E_BADARGS=65
REPO=$1
 
if [ $# -ne $EXPECTED_ARGS ]
then
  echo "Usage: $0 reponame"
  exit $E_BADARGS
fi
 
$SVNADMIN create --fs-type fsfs $REPOS_PATH/$REPO
 
rm -rf /tmp/subversion-layout/
mkdir -pv /tmp/subversion-layout/{trunk,branches,tags}
 
ln -s $SCRIPTS_PATH/pre-commit $REPOS_PATH/$1/hooks/pre-commit
ln -s $SCRIPTS_PATH/post-commit $REPOS_PATH/$1/hooks/post-commit
 
chown $APACHE_USER:$APACHE_USER -R $REPOS_PATH/$1
chmod -R 2775 $REPOS_PATH/$1
 
svn import -m "Initial Import" /tmp/subversion-layout/ $REPOS_URL/$REPO
rm -rf /tmp/subversion-layout/

As you root you just run it as /home/svn/scripts/svncreate.sh reponame

Install Phusion Passenger (a.k.a. mod_rails) on cPanel Server

words by Brian Racer

Although cPanel has built in support for running Ruby or Rails apps, it uses Mongrel as the server and doesn’t allow more than one instance per user. That makes it pretty useless for any application that gets even a moderate amount of traffic. Instead we can install Phusion Passenger (a.k.a mod_rails), which in my opinion is a much nicer solution anyway.

First we need to make sure Ruby is installed via a cpanel script:

sudo /script/installruby

Now we can install the passenger gem:

sudo gem install passenger

Next, compile the apache2 module

sudo passenger-install-apache2-module

The installer may tell you that the the Apache development headers are needed and will suggest ‘yum install httpd-devel’. Since cPanel compiles it’s own version of apache, yum is configured to ignore that package. That is OK, because the program we need is already installed, we just have to tell Passenger where to find it.

APXS2=/usr/local/apache/bin/apxs PATH=$PATH:/usr/local/apache/bin passenger-install-apache2-module

Everything should go OK this time, and the installer will give you a few lines to add to your apache config file. It’s best practice with cPanel not to put these in your main httpd.conf, but rather the pre_main_global.conf:

vi /usr/local/apache/conf/includes/pre_main_global.conf
 
LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-X.X.X/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-X.X.X
PassengerRuby /usr/bin/ruby

Now we need to setup passenger to run on a per virtual host basis. Open up the httpd.conf file and find the virtual host you want to run a Rails app and add this line:

Include "/usr/local/apache/conf/userdata/std/2/username/domain_name/*.conf"

Replace username with the username of the account.

Now we need to create the directory we just specified, and also create a configuration file letting passenger know it should load for this host:

mkdir -p /usr/local/apache/conf/userdata/std/2/username/domain_name/
vi /usr/local/apache/conf/userdata/std/2/username/domain_name/rails.conf
 
RailsBaseURI /

To make sure those files load, run this:

/scripts/ensure_vhost_includes --user=username

We need to make sure cPanel records the changes we have for when it rebuilds those files, so run the following two commands:

/usr/local/cpanel/bin/apache_conf_distiller --update
/usr/local/cpanel/bin/build_apache_conf

We can now restart apache:

/etc/init.d/httpd restart

Since by default the Apache Document Root for each host is /home/username/public_html, you will probably need to symlink that to your applications public directory:

ln -s /home/username/railsapp/public /home/username/public_html

To restart that application, you just need to touch the restart.txt file:

touch /home/username/railsapp/tmp/restart.txt

And there you have it, a working high performance rail application server on cPanel! For more information on tuning the Passenger configuration, read the complete docs.

Bash script to create MySQL database and user

words by Brian Racer

Here is a little script I made to quickly and easily create users and databases for MySQL. I only use this for development, for actual deployed applications you would probably want to be more specific about the privileges given:

#!/bin/bash
 
EXPECTED_ARGS=3
E_BADARGS=65
MYSQL=`which mysql`
 
Q1="CREATE DATABASE IF NOT EXISTS $1;"
Q2="GRANT ALL ON *.* TO '$2'@'localhost' IDENTIFIED BY '$3';"
Q3="FLUSH PRIVILEGES;"
SQL="${Q1}${Q2}${Q3}"
 
if [ $# -ne $EXPECTED_ARGS ]
then
  echo "Usage: $0 dbname dbuser dbpass"
  exit $E_BADARGS
fi
 
$MYSQL -uroot -p -e "$SQL"

To use it, just run:

./createdb testdb testuser secretpass

That command would create a database named testdb, and user testuser with the password of secretpass.

Useful PHP Subversion Commit Hook

words by Brian Racer

Here is a subversion pre-commit hook script we use on PHP projects to make sure the developer making the commit is providing a meaningful description, and then PHP lint is run on each PHP script to make sure it will compile correctly.

#!/bin/bash
 
REPOS="$1"
TXN="$2"
 
PHP="/usr/bin/php"
SVNLOOK="/usr/bin/svnlook"
AWK="/usr/bin/awk"
GREP="/bin/egrep"
SED="/bin/sed"
 
CHANGED=`$SVNLOOK changed -t "$TXN" "$REPOS" | $AWK '{print $2}' | $GREP \\.php$`
 
for FILE in $CHANGED
do
    MESSAGE=`$SVNLOOK cat -t "$TXN" "$REPOS" "$FILE" | $PHP -l`
    if [ $? -ne 0 ]
    then
        echo 1>&2
        echo "***********************************" 1>&2
        echo "PHP error in: $FILE:" 1>&2
        echo `echo "$MESSAGE" | $SED "s| -| $FILE|g"` 1>&2
        echo "***********************************" 1>&2
        exit 1
    fi
done
 
# Make sure that the log message contains some text.
SVNLOOKOK=1
SVNLOOK=/usr/bin/svnlook
$SVNLOOK log -t "$TXN" "$REPOS" | \
   grep "[a-zA-Z0-9]" > /dev/null || SVNLOOKOK=0
if [ $SVNLOOKOK = 0 ]; then
  echo Empty log messages are not allowed. Please provide a proper log message. 1>&2
  exit 1
fi
 
# Make sure text might be meaningful
LOGMSGLEN=$($SVNLOOK log -t "$TXN" "$REPOS" | grep [a-zA-Z0-9] | wc -c)
if [ "$LOGMSGLEN" -lt 6 ]; then
  echo -e "Please provide a meaningful comment when committing changes." 1>&2
  exit 1
fi
 
# All checks passed, so allow the commit.
exit 0

Override PHP’s mail() function during development

words by Brian Racer

When doing local development we generally don’t want our test servers sending out mail to the world. And it would be ideal to be able to review the emails our application does send out before deploying the changes to the world. An easy way to achieve this functionality is to override PHP’s sendmail_path config variable. First lets install a few packegs that will allow us to send mail, and some useful scripts to rewrite the mail:

sudo apt-get install procmail sendmail

Next create the following script that will rewrite any mail that all mail generated by PHP’s mail() function to the local user of your choice:

vi /usr/local/bin/trapmail
formail -R cc X-original-cc \
-R to X-original-to \
-R bcc X-original-bcc \
-f -A"To: username@localhost" \
| /usr/sbin/sendmail -t -i

Replace username@localhost with your local username or an external email address.

Now update your php.ini file’s sendmail_path:

grep sendmail_path /etc/php5/apache2/conf/php.ini
 
sendmail_path=/usr/local/bin/trapmail

You can then use mail client like mutt or Thunderbird to review the emails, or just tail your mbox file.

Useful Linux Trick: cron @reboot

words by Brian Racer

There are various ways to make sure something is run at system startup – Redhat has /etc/rc.local script, and it and many others have /etc/init.d/* scripts – but many times you might not have access to those files or creating init scripts might be overkill for your needs.

People are always amazed when I tell them they can achieve this basic functionality by using cron. Many of our websites use Sphinx, the excellent full text indexer, to allow site searches. Should the server ever reboot, we need to make multiple search daemons start back up. Take the following line from a crontab:

crontab -l
 
@reboot /usr/local/bin/searchd --config ~/conf/sphinx.conf

This will make sure the searchd daemon starts on bootup.

Also there are a few other shortcuts you can use:

@yearly        Run once a year, "0 0 1 1 *".
@annually      (same as @yearly)
@monthly       Run once a month, "0 0 1 * *".
@weekly        Run once a week, "0 0 * * 0".
@daily         Run once a day, "0 0 * * *".
@midnight      (same as @daily)
@hourly        Run once an hour, "0 * * * *".

See man 5 cron for more information.

(And to be pedantic, @reboot is run when cron is started or restarted, not necessarily the OS itself. So /etc/init.d/cron restart would trigger that line to be run. You may want to keep that in mind.)

How to fix Munin’s Exim Graph on cPanel

words by Brian Racer

If you notice the Exim graphs on your server have stop updating, you might want to check /var/log/munin/munin-node.log and see if you have lots of these entries:

tail /var/log/munin/munin-node.log
 
...
Plugin "exim_mailstats" exited with status 768. ----
...

You just need to remove the state file, and the graph will start updating again. Don’t worry about deleting the file, no data should be lost.

rm /var/lib/munin/plugin-state/plugin-exim_mailstats.state

How to fix Munin’s MySQL Graph on cPanel

words by Brian Racer

We have a few cPanel servers deployed and to visually monitor performance we use the Munin monitoring tool. We were having issues where the MySQL graphs would stop updating. The bug has to do with one of the perl libraries the script uses, which causes the MySQL plugin to have problems location the mysqladmin program.

To fix these, create or edit the following file:

vi /etc/munin/plugin-conf.d/cpanel.conf

It should contain the following:

[mysql*]
user root
group wheel
env.mysqladmin /usr/bin/mysqladmin
env.mysqlopts --defaults-extra-file=/root/.my.cnf

This would load the username and password from root’s mysql config. If that file doesn’t exist you can create it:

sudo vi /root/.my.cnf
[client]
user="root"
pass="secret!password"

Now just restart Munin, wait a a little bit, and the graphs should start populating again!

sudo /etc/init.d/munin-node restart

Using Maatkit to batch convert MySQL storage engines

words by Brian Racer

Recently I was working with a client who imported a couple databases, each that had thousands of InnoDB tables taking up tens of gigabytes of data to a new server. Unfortunately the InnoDB engine was misconfiguration and therefore not loaded at the time of import. MySQL silently created all these tables as MyISAM instead. Not wanting to wait hours for the import process to proceed again, I used a program from the excellent Maatkit package: mk-find. It works similar to the unix find command, except it works on the MySQL server.

The following command will find all MyISAM tables from a certain database and convert them to InnoDB:

mk-find <db_name> --engine MyISAM --exec "ALTER TABLE %D.%N ENGINE=INNODB" --print

You can download the latest version from Maatkit’s Google Code page.

Installing fonts in Ubuntu

words by Brian Racer

Recently I wanted to add some Mac-style fonts to my Ubuntu system. Although it requires slightly more work than just dragging the font files into a folder, it is still quite simple.

If you have full root or sudo access and you would like the font to be shared with all users of the system, you may want to put the files in /usr/local/share/fonts/truetype. An easy way to navigate to that location with root privileges is to press Alt+F2 and type:

gksudo nautilus /usr/local/share/fonts/truetype

You can then create a new directory and place your fonts into it.

Alternatively you may place the fonts in a directory inside your home folder. The files get placed in a hidden directory called .fonts. You may have to create this directory. Only you will be able to access these fonts.

mkdir ~/.fonts

Whichever you choose, you will need to reset the font cache . You can reset the cache with the following command:

sudo fc-cache -f -v

And for anyone interested, here are a nice set of free MacOS font alternatives to use in Windows or Linux

Mac-Fonts – this includes AppleGaramond, Aquabase, LITHOGRL, Lucida Grande, Lucida Mac, Lucon, MacGrand. I downloaded these from this site, however you need to go through multiple pages of referral sign-ups to access the file.

Monaco Linux – This is the font I use in vim, it looks great.

Inconsolata – This is a variation of Luc(as) de Groot’s Consolas font, which is another nice mono-spaced programming font featured in Windows Vista. Read more about it at the authors webpage.

You may also want to install the Miscrosoft Web Fonts(msttcorefonts):

sudo aptitude install msttcorefonts