Jan 08 2010

Pimp your $PS1 with source control information

Category: bash, linux — Brian Racer @ 8:17 pm

I recently found a useful tip of appending source control information of the current working directory to your shell’s $PS1 line. It might look something like the following image:

The method I saw suggested using vcprompt, a small program that outputs a short string basic version control info. Although it worked well, there were a couple issues I had with it. First, it’s subversion support was somewhat lacking. Second, it was written in C which made it more of a pain to modify, and I wasn’t a huge fan of keeping the binary in version control.

After a little searching I stumbled across vcprompt.py, a python script that did the same thing. This version also had wider support for source control systems, and being a standard python text script it was something I could easily modify and put into my dotfiles git repo. I wasn’t happy with how this one displayed subversion information either(just a revision number which I didn’t find very helpful), so I made my own modification to display the branch you are in. Please pardon my lacking Python skills.

Anyway, on to pimping your prompt. Before we modify the PS1 variable, we need to make sure the vcprompt.py is in your $PATH. I like to put scripts like this in a custom bin directory in my homedir. One way to accomplish that might be the following:

$ mkdir -p ~/bin
$ cd ~/bin
$ wget http://github.com/anveo/dotfiles/raw/master/scripts/vcprompt.py
$ chmod +x vcprompt.py
$ export PATH=~/bin:$PATH

Displayed next is the PS1 line I use – it takes up two lines: the first line contains the current user, hostname, current working directory, and possibly version control info, and the second is just a nice looking arrow for my input. Your terminal will need to support Unicode if you want to use that symbol.

# .bashrc
PS1="\n\u@\h:\w \[\e[1;30m\]$(vcprompt)\[\e[0m\] \n→"
 
# If you are using zsh you may also need the following in .zshrc
setopt prompt_subst

If you use the colors specified, you may need to define those too.

You should now have a prompt similar to the image above! For more shell customizations checkout the rest of my dotfiles, and consider buying Peepcode’s Advanced Command Line screencasts for more productive tips!

Tags: , , , , , ,

Nov 02 2009

Monitoring delayed_job with god on CentOS

Category: linux, rails, ruby, sysadmin — Brian Racer @ 4:24 pm

I recently started using god rather than monit for process monitoring. god lets me be a bit more expressive with how I want processes monitored using the the power of Ruby.

The current project I am working on has a number of tasks that I want processed asynchronously so I will setup god to monitor my delayed_jobs. If you are not familiar with awesome delayed_job gem, watch the excellent Railscast tutorial.

First install the god gem:

$ sudo gem install god

Next we will create a Redhat compatible init script for god:

$ sudo vi /etc/init.d/god
 
#!/bin/bash
#
# God
#
# chkconfig: - 85 15
# description: start, stop, restart God (bet you feel powerful)
#
 
RETVAL=0
 
case "$1" in
    start)
      /usr/bin/god -P /var/run/god.pid -l /var/log/god.log
      /usr/bin/god load /etc/god.conf
      RETVAL=$?
      ;;
    stop)
      kill `cat /var/run/god.pid`
      RETVAL=$?
      ;;
    restart)
      kill `cat /var/run/god.pid`
      /usr/bin/god -P /var/run/god.pid -l /var/log/god.log
      /usr/bin/god load /etc/god.conf
      RETVAL=$?
      ;;
    status)      
      /usr/bin/god status
      RETVAL=$?
      ;;
    *)
      echo "Usage: god {start|stop|restart|status}"
      exit 1
  ;;
esac
 
exit $RETVAL
(adapted from debian version at http://mylescarrick.com/articles/simple_delayed_job_with_god)

Now adjust the permissions, and set the init script to start on system boot:

$ sudo chmod a+x /etc/init.d/god
$ sudo /sbin/chkconfig --add god
$ sudo /sbin/chkconfig --level 345 god on

Before we start god up, we need to create a configuration file that tells it what configuration files to load:

$ sudo vi /etc/god.conf
 
God.load "/srv/apps/your_app/current/config/god/*.god"

You will need to adjust the above depending on how you have your app setup. When working in a Rails project I like to put my god scripts in config/god.

We will use a script from the guys at github to monitor our job daemon. I tweaked it slightly to have less workers, and to set the environment properly.

RAILS_ROOT = "/srv/apps/your_app/current"
 
1.times do |num|
  God.watch do |w|
    w.name = "dj-#{num}"
    w.group = 'dj'
    w.interval = 30.seconds
    w.start = "rake -f #{RAILS_ROOT}/Rakefile RAILS_ENV=production jobs:work"
 
    w.uid = 'your_app_user'
    w.gid = 'your_app_user'
 
    # retart if memory gets too high
    w.transition(:up, :restart) do |on|
      on.condition(:memory_usage) do |c|
        c.above = 300.megabytes
        c.times = 2
      end
    end
 
    # determine the state on startup
    w.transition(:init, { true => :up, false => :start }) do |on|
      on.condition(:process_running) do |c|
        c.running = true
      end
    end
 
    # determine when process has finished starting
    w.transition([:start, :restart], :up) do |on|
      on.condition(:process_running) do |c|
        c.running = true
        c.interval = 5.seconds
      end
 
      # failsafe
      on.condition(:tries) do |c|
        c.times = 5
        c.transition = :start
        c.interval = 5.seconds
      end
    end
 
    # start if process is not running
    w.transition(:up, :start) do |on|
      on.condition(:process_running) do |c|
        c.running = false
      end
    end
  end
end

It’s now time to start the daemon:

$ sudo /etc/init.d/god start
$ sudo /etc/init.d/god status
dj:
  dj-0: up

Looks good! If you want to make sure it’s working, kill the rake task running jobs:work. god will see that it is stopped and automatically restart it!

Tags: , ,

Oct 29 2009

Ubuntu 9.10 (Karmic Koala) and Broadcom BCM4312

Category: desktop, linux, ubuntu — Brian Racer @ 3:35 pm

Ubuntu 9.10 (Karmic Koala) launched today and I figured it was time to do an install from scratch onto my Dell D830 Latitude laptop. Everything went quite smoothly but when it started up I noticed two issues:

Problem 1: No wireless

I know the Broadcom card inside the laptop isn’t the greatest, but the last two Ubuntu releases it has worked out of the box. The following command enabled the card after a reboot:

sudo apt-get install bcmwl-kernel-source

Problem 2: Really slow DNS lookups (because of IPV6)

As documented on Launchpad, there still doesn’t seem to be an official fix. Strangely disabling IPV6 in /etc/sysctl.conf didn’t solve anything, however disabling it in Firefox at least fixes the issue in the browser. Just type about:config in the address bar, and set network.dns.disableIPv6 to false.

Otherwise things seem to be working well, although I don’t understand why they stick with a color scheme that looks like mud.

Tags: ,

Sep 23 2009

pbcopy / pbpaste in Ubuntu (command line clipboard)

Category: linux — Brian Racer @ 1:13 pm

OS X has a neat command-line tool called pbcopy which takes the standard input and places it in the clipboard to paste into other applications.

In Ubuntu(or any Linux distro with Xwindows), a similar tool is xclip. I like to make this alias:

alias pbcopy='xclip -selection clipboard'
alias pbpaste='xclip -selection clipboard -o'

or the following also works if you would rather use xsel:

alias pbcopy='xsel --clipboard --input'
alias pbpaste='xsel --clipboard --output'

Now you can pipe any text to pbcopy

$ cat ~/.ssh/id_dsa.pub | pbcopy

Your public ssh key is transferred to your clipboard and is ready to be pasted(perhaps with pbpaste).

Tags: ,

Sep 16 2009

Unobtrusive viewing of MySQL queries with tcpdump

Category: linux, mysql, sysadmin — Brian Racer @ 11:33 am

There are times when you need to monitor the queries coming in to MySQL, but turning on query logging would create too much of a disk I/O hit, or you can’t restart the server to setup MySQL Proxy. Instead we can just monitor the network traffic and extract data that might be interesting using tcpdump and an inline perl script:

sudo tcpdump -i lo -s 0 -l -w - dst port 3306 | strings | perl -e '
while(<>) { chomp; next if /^[^ ]+[ ]*$/;
  if(/^(SELECT|UPDATE|DELETE|INSERT|SET|COMMIT|ROLLBACK|CREATE|DROP|ALTER)/i) {
    if (defined $q) { print "$q\n"; }
    $q=$_;
  } else {
    $_ =~ s/^[ \t]+//; $q.=" $_";
  }
}'

This will only work for clients communicating via TCP – if you are connecting through ‘localhost’ you will be going through a unix socket instead. If you switch ‘localhost’ to ‘127.0.0.1′ then your queries will go through the network stack.

If you just want to dump the traffic to a file for a little bit and analyze it later, do this instead:

sudo tcpdump -i lo port 3306 -s 65535 -x -n -q -tttt> tcpdump.out

You can then use mk-query-digest from Maatkit with–type=tcpdump. See more about this at the MySQL Performance Blog.

Tags:

Sep 14 2009

Linux Tip: Keep track of packages you have installed

Category: linux, sysadmin — Brian Racer @ 7:12 pm

During development on a linux system, you probably install many packages using your favorite package manager. When you have to use a new system, or reimage your current one, it can be a pain to remember all the packages you had setup. One solution is to keep a list of the packages installed after the OS load, and then periodically generate a list of what has been added since.

On a freshly installed system, create the starting baseline list of packages:

# On Debian based systems(Ubuntu):
dpkg --get-selections > packages-alpha.txt
 
# Or CentOS/Fedora:
yum list installed > packages-alpha.txt

You can run the command again at a later time, concatenating the output into a different file so you can view what has changed since the original system setup. Use a diff tool like diff3, vimdiff, or meld:

meld packages-alpha.txt packages-omega.txt

On Debian systems, once you have that file you can use it in a new or different system to mark packages to install using the –set-selection parameter:

dpkg --set-selections < packages-omega.txt
sudo apt-get upgrade

Tags: ,

Aug 19 2009

Determining linux distribution

Category: linux, sysadmin — Brian Racer @ 12:00 pm

I work with a wide variety of client server deployments, and sometimes it isn’t obvious(via uname) what distribution and version a server is running. Here is a quick list of common files which contain that information:

Debian          /etc/debian_release, /etc/debian_version,
Fedora          /etc/fedora-release
Gentoo          /etc/gentoo-release
Mandrake        /etc/mandrake-release
Novell SUSE     /etc/SUSE-release
Red Hat         /etc/redhat-release, /etc/redhat_version
Slackware       /etc/slackware-release, /etc/slackware-version
Solaris/Sparc   /etc/release
Sun JDS         /etc/sun-release
Ubuntu          /etc/lsb-release
UnitedLinux     /etc/UnitedLinux-release
Yellow dog      /etc/yellowdog-release

Aug 06 2009

Ubuntu Tip: Force new windows to start centered on the desktop

Category: desktop, linux — Brian Racer @ 9:57 am

I use a pretty generic Gnome + Compiz desktop setup in Ubuntu, but one thing that really irks me is my applications always seem to start snapped to a corner. What I really want is for them to open centered on my desktop. You can achieve this by doing a little registry modification(I’m pretty sure there is a nice GUI app to adjust these settings, but I don’t believe it is installed by default).

Press Alt+F2 and enter gconf-config. This will open up Gnome’s registry editor.

Set the following two values:
Key: /apps/metacity/general/focus_new_windows Value: smart
Key: /apps/compiz/plugins/place/screen0/options/mode Value: 1

Now your applications should start up nice and centered :)

Tags: , ,

Aug 05 2009

Using a sane user-agent for Ubuntu’s Firefox 3.5 – Shiretoko

Category: desktop, linux — Brian Racer @ 1:59 pm

Ubuntu’s current release version of Firefox 3.5 is named Shiretoko and sends a user-agent of Shiretoko/3.5 rather than Firefox/3.5. This broke a number of sites I use that rely on browser sniffing such as Facebook Chat and DailyMotion. There are two ways to adjust this behavior:

1) Type about:config in the address bar. Search for ‘general.useragent.extra.firefox’. Double click “Shiretoko/3.5″ replace it with “Firefox/3.5″

2) Use the User Agent Switcher plugin. This I prefer this option as it also lets me set IE user agents so I can use a few sites that think they require IE, and also set iPhone header’s for development.

Tags:

Jul 22 2009

Subversion repository creation script

Category: sysadmin — Brian Racer @ 12:12 pm

Here is a simple script I use to create subversion repositories and setup common hook scripts. This assumes you are making svn available through an HTTPS/Apache/DAV.

My directory svn structure looks like the following:

$ tree -L 2 /home/svn
 
/home/svn
|-- conf
|   |-- permissions.conf
|   `-- mailer.conf
|-- repos
|   |-- repo1
|   |-- repo2
|   `-- repo3 
`-- scripts
    |-- post-commit
    |-- pre-commit
    `-- svncreate.sh

There are 3 main directories:

repos – This is where the actual subversion repositories are stored
scripts – Various scripts, including the following creation script, and also various hook scripts that repositories can symlink to
conf – Various configuration files that might contain mailer configuration or repository permissions(both outside the scope of this article)

The following script will create the new repository, set proper user and permissions, symlink common pre and pos commit scripts, and then make the initial import that contains the trunk/branches/tags structure.

#!/bin/bash
 
REPOS_URL=https://svn.example.com
REPOS_PATH=/home/svn/repos
SCRIPTS_PATH=/home/svn/scripts
APACHE_USER=www-data
 
SVNADMIN=`which svnadmin`
EXPECTED_ARGS=1
E_BADARGS=65
REPO=$1
 
if [ $# -ne $EXPECTED_ARGS ]
then
  echo "Usage: $0 reponame"
  exit $E_BADARGS
fi
 
$SVNADMIN create --fs-type fsfs $REPOS_PATH/$REPO
 
rm -rf /tmp/subversion-layout/
mkdir -pv /tmp/subversion-layout/{trunk,branches,tags}
 
ln -s $SCRIPTS_PATH/pre-commit $REPOS_PATH/$1/hooks/pre-commit
ln -s $SCRIPTS_PATH/post-commit $REPOS_PATH/$1/hooks/post-commit
 
chown $APACHE_USER:$APACHE_USER -R $REPOS_PATH/$1
chmod -R 2775 $REPOS_PATH/$1
 
svn import -m "Initial Import" /tmp/subversion-layout/ $REPOS_URL/$REPO
rm -rf /tmp/subversion-layout/

As you root you just run it as /home/svn/scripts/svncreate.sh reponame

Next Page »