Learning Haml/Sass? Try Rendera!

words by Brian Racer

Fellow ECRuby group member Brian Hogan has released Rendera, a great web-based tool to learn Haml and Sass markup interactively in your browser. You can paste in your HTML/CSS code and it will convert to Haml and Sass, and vice versa. It also includes a number of interesting preset examples to study – a few exploiting some very cool HTML5 features. Check it out!

(and if you haven’t checked out Haml or Sass yet, please do!)


Monitoring delayed_job with god on CentOS

words by Brian Racer

I recently started using god rather than monit for process monitoring. god lets me be a bit more expressive with how I want processes monitored using the the power of Ruby.

The current project I am working on has a number of tasks that I want processed asynchronously so I will setup god to monitor my delayed_jobs. If you are not familiar with awesome delayed_job gem, watch the excellent Railscast tutorial.

First install the god gem:

$ sudo gem install god

Next we will create a Redhat compatible init script for god:

$ sudo vi /etc/init.d/god
 
#!/bin/bash
#
# God
#
# chkconfig: - 85 15
# description: start, stop, restart God (bet you feel powerful)
#
 
RETVAL=0
 
case "$1" in
    start)
      /usr/bin/god -P /var/run/god.pid -l /var/log/god.log
      /usr/bin/god load /etc/god.conf
      RETVAL=$?
      ;;
    stop)
      kill `cat /var/run/god.pid`
      RETVAL=$?
      ;;
    restart)
      kill `cat /var/run/god.pid`
      /usr/bin/god -P /var/run/god.pid -l /var/log/god.log
      /usr/bin/god load /etc/god.conf
      RETVAL=$?
      ;;
    status)      
      /usr/bin/god status
      RETVAL=$?
      ;;
    *)
      echo "Usage: god {start|stop|restart|status}"
      exit 1
  ;;
esac
 
exit $RETVAL
(adapted from debian version at http://mylescarrick.com/articles/simple_delayed_job_with_god)

Now adjust the permissions, and set the init script to start on system boot:

$ sudo chmod a+x /etc/init.d/god
$ sudo /sbin/chkconfig --add god
$ sudo /sbin/chkconfig --level 345 god on

Before we start god up, we need to create a configuration file that tells it what configuration files to load:

$ sudo vi /etc/god.conf
 
God.load "/srv/apps/your_app/current/config/god/*.god"

You will need to adjust the above depending on how you have your app setup. When working in a Rails project I like to put my god scripts in config/god.

We will use a script from the guys at github to monitor our job daemon. I tweaked it slightly to have less workers, and to set the environment properly.

RAILS_ROOT = "/srv/apps/your_app/current"
 
1.times do |num|
  God.watch do |w|
    w.name = "dj-#{num}"
    w.group = 'dj'
    w.interval = 30.seconds
    w.start = "rake -f #{RAILS_ROOT}/Rakefile RAILS_ENV=production jobs:work"
 
    w.uid = 'your_app_user'
    w.gid = 'your_app_user'
 
    # retart if memory gets too high
    w.transition(:up, :restart) do |on|
      on.condition(:memory_usage) do |c|
        c.above = 300.megabytes
        c.times = 2
      end
    end
 
    # determine the state on startup
    w.transition(:init, { true => :up, false => :start }) do |on|
      on.condition(:process_running) do |c|
        c.running = true
      end
    end
 
    # determine when process has finished starting
    w.transition([:start, :restart], :up) do |on|
      on.condition(:process_running) do |c|
        c.running = true
        c.interval = 5.seconds
      end
 
      # failsafe
      on.condition(:tries) do |c|
        c.times = 5
        c.transition = :start
        c.interval = 5.seconds
      end
    end
 
    # start if process is not running
    w.transition(:up, :start) do |on|
      on.condition(:process_running) do |c|
        c.running = false
      end
    end
  end
end

It’s now time to start the daemon:

$ sudo /etc/init.d/god start
$ sudo /etc/init.d/god status
dj:
  dj-0: up

Looks good! If you want to make sure it’s working, kill the rake task running jobs:work. god will see that it is stopped and automatically restart it!


Ubuntu 9.10 (Karmic Koala) and Broadcom BCM4312

words by Brian Racer

Ubuntu 9.10 (Karmic Koala) launched today and I figured it was time to do an install from scratch onto my Dell D830 Latitude laptop. Everything went quite smoothly but when it started up I noticed two issues:

Problem 1: No wireless

I know the Broadcom card inside the laptop isn’t the greatest, but the last two Ubuntu releases it has worked out of the box. The following command enabled the card after a reboot:

sudo apt-get install bcmwl-kernel-source

Problem 2: Really slow DNS lookups (because of IPV6)

As documented on Launchpad, there still doesn’t seem to be an official fix. Strangely disabling IPV6 in /etc/sysctl.conf didn’t solve anything, however disabling it in Firefox at least fixes the issue in the browser. Just type about:config in the address bar, and set network.dns.disableIPv6 to false.

Otherwise things seem to be working well, although I don’t understand why they stick with a color scheme that looks like mud.


Rails 2.3.4 and SWFUpload – Rack Middleware for Flash Uploads that Degrade Gracefully

words by Brian Racer

Browser upload controls have been pretty much the same for years. They are very difficult to style, and do not look consistent across browsers. Perhaps the biggest issue with them is they provide no feedback to the user about how long the submission will take. One alternative is to use Flash for the uploads. There are numerous libraries available, I like SWFUpload. Since the reason you are here is probably because you can’t get it working in Rails, I’m going to try and help you deal with the quirks associated with using Flash and Rails together.

It used to be you would monkeypatch the CGI class to get Flash uploaders to work due to issues with Flash. With the introduction of Rack in Rails 2.3 things now work quite differently. What we will do is create some rack middleware to intercept traffic from Flash to deal with it’s quirks. I have created a small example application of an mp3 player and uploader. You will probably want to download it, as it contains a few files not displayed in this article. You can clone it from the github project page.

First lets create a simple Song model:

./script generate model Song title:string artist:string length_in_seceonds:integer track_file_name:string track_content_type:string track_file_size:integer

title, artist, and length_in_seconds are meta-data we will pull from the ID3 tags of the uploaded mp3 file, and the rest will be used by Paperclip to handle the attachment. Lets add the paperclip attachment and a few simple validations to our new Song model:

class Song < ActiveRecord::Base
 
  has_attached_file :track,
                    :path => ":rails_root/public/assets/:attachment/:id_partition/:id/:style/:basename.:extension",
                    :url => "/assets/:attachment/:id_partition/:id/:style/:basename.:extension"
 
  validates_presence_of :title, :artist, :length_in_seconds
  validates_attachment_presence :track
  validates_attachment_content_type :track, :content_type => [ 'application/mp3', 'application/x-mp3', 'audio/mpeg', 'audio/mp3' ]
  validates_attachment_size :track, :less_than => 20.megabytes
 
  attr_accessible :title, :artist, :length_in_seconds
 
  def convert_seconds_to_time
    total_minutes = length_in_seconds / 1.minutes
    seconds_in_last_minute = length_in_seconds - total_minutes.minutes.seconds
    "#{total_minutes}m #{seconds_in_last_minute}s"
  end
end

Next comes an upload form and some containers to hold the SWFUploader:

- form_tag songs_path, :multipart => true do
  #swfupload_degraded_container
    %noscript= "You should have Javascript enabled for a nicer upload experience"
    = file_field_tag :Filedata
    = submit_tag "Add Song"
  #swfupload_container{ :style => "display: none" }
    %span#spanButtonPlaceholder
  #divFileProgressContainer

The container that holds the SWFUploader will be hidden until we know the user can support it. Initially a standard file upload form will display. A number of things can go wrong, so we need to think about a few levels of degradation here. The user might not have flash installed, the user might have an outdated version of flash, he might not have javascript installed or enabled(which is needed to load the flash), and there may be a problem downloading the flash swf file. Yikes. Luckily using the swfobject library we can easily handle all these potential issues.

If the user is missing javascript, he will see the message in the noscript tag and be presented a standard upload control.

If the user is missing flash or it is outdated, he will be presented a dialog with an upgrade link. Otherwise he can use the standard upload control.

If everything goes okey-dokey, then some function handlers we write will hide the the degradation container, and display the flash container.

Oh, and just so you know the current version of Flash Player for linux do not fire the event that monitors upload progress, so you will not get the status bar until the upload finishes. No work around for that right now.

So lets initialize the SWFUpload via some javascript. Many tutorials out there seem to put the authentication token and session information in the URL, but there are some options with current version of SWFUpload to POST and avoid that.

:javascript
  SWFUpload.onload = function() {
    var swf_settings = {
 
      // SWFObject settings
      minimum_flash_version: "9.0.28",
      swfupload_pre_load_handler: function() {
        $('#swfupload_degraded_container').hide();
        $('#swfupload_container').show();
      },
      swfupload_load_failed_handler: function() {
      },
 
      post_params: {
        "#{session_key_name}": "#{cookies[session_key_name]}",
        "authenticity_token": "#{form_authenticity_token}",
      },
 
      upload_url: "#{songs_path}",
      flash_url: '/flash/swfupload/swfupload.swf',
 
      file_types: "*.mp3",
      file_types_description: "mp3 Files",
      file_size_limit: "20 MB",
 
      button_placeholder_id: "spanButtonPlaceholder",
      button_width: 380,
      button_height: 32,
      button_text : '<span class="button">Select Files <span class="buttonSmall">(20 MB Max)</span></span>',
      button_text_style : '.button { font-family: Helvetica, Arial, sans-serif; font-size: 24pt; } .buttonSmall { font-size: 18pt; }',
      button_text_top_padding: 0,
      button_text_left_padding: 18,
      button_window_mode: SWFUpload.WINDOW_MODE.TRANSPARENT,
      button_cursor: SWFUpload.CURSOR.HAND,
      file_queue_error_handler : fileQueueError,
      file_dialog_complete_handler : fileDialogComplete,
      upload_progress_handler : uploadProgress,
      upload_error_handler : uploadError,
      upload_success_handler : uploadSuccess,
      upload_complete_handler : uploadComplete,
 
      custom_settings : {
        upload_target: "divFileProgressContainer"
      }
    }
    var swf_upload = new SWFUpload(swf_settings);
  };

You will want to check out the official SWFUpload docs to understand what all of these variable do. There are many handlers we have to define to handle various events, and if you clone the project you can review them in detail.

We also need to set styles for the containers that will be generated. You can see the Sass file I created for SWFUpload here, and another one for Ryan Bates nifty_generators.

Another quirk we have to be aware of when dealing with flash uploads is that everything gets a content-type of an octet stream. We will use the mime-types library to identify it for validation. Keep in mind it only uses the extension to determine the file type. (I haven’t tested it yet, but I believe mimetype-fu will actually check file-data and magic numbers). By default SWFUpload calls the file parameter ‘Filedata’.

  def create
    require 'mp3info'
 
    mp3_info = Mp3Info.new(params[:Filedata].path)
 
    song = Song.new
    song.artist = mp3_info.tag.artist
    song.title = mp3_info.tag.title
    song.length_in_seconds = mp3_info.length.to_i
 
    params[:Filedata].content_type = MIME::Types.type_for(params[:Filedata].original_filename).to_s
    song.track = params[:Filedata]
    song.save
 
    render :text => [song.artist, song.title, song.convert_seconds_to_time].join(" - ")
  rescue Mp3InfoError => e
    render :text => "File error"
  rescue Exception => e
    render :text => e.message
  end

Another annoyance with flash uploads is that it doesn’t send cookie data. That is why we are sending the session information in the POST data. We will intercept requests from Flash, check for the session key, and if so inject it into the cookie header. We can do this with some pretty simple middleware.

require 'rack/utils'
 
class FlashSessionCookieMiddleware
  def initialize(app, session_key = '_session_id')
    @app = app
    @session_key = session_key
  end
 
  def call(env)
    if env['HTTP_USER_AGENT'] =~ /^(Adobe|Shockwave) Flash/
      params = ::Rack::Request.new(env).params
      env['HTTP_COOKIE'] = [ @session_key, params[@session_key] ].join('=').freeze unless params[@session_key].nil?
    end
    @app.call(env)
  end
end

This is a modified version from code the appears in a few tutorials about flash uploads. It will allow the session information to be in the query string *or* POST data. Next we have to make sure this middleware gets put to use so in config/initializers/session_store.rb add:

ActionController::Dispatcher.middleware.insert_before(ActionController::Base.session_store, FlashSessionCookieMiddleware, ActionController::Base.session_options[:key])

And that’s, uhh, all there is too it. Again, I really suggest you checkout the example project. It also uses the nifty WordPress Audio Player flash control to play the music you upload!



High Quality Ruby on Rails Example Applications

words by Brian Racer

Ruby on Rails LogoSometimes to best way to get up to speed with a new technology is learning by example. I have compiled a list of fully featured, production ready example applications that I consider to be of very decent quality.

Most are RESTful and all have good-great test coverage. I listed components like the authentication, templating, and testing frameworks they employ – perhaps useful if you are looking for examples of say cucumber stories, or maybe how to use haml markup. Also listed are some of the gems and plugins they leverage which I think are either useful or popular and worth checking out if you are not already familiar with them.

If you know of other quality apps I have missed just let me know in the comments section and I will add them. So in no particular order here they are – Enjoy!

 

 


Homepage: http://railscasts.com/
Github: http://github.com/ryanb/railscasts

You’ve probably seen his excellent video tutorials, but did you know Ryan Bates also gives away the code that powers his site?

Plugins: redcloth, acts-as-list, will_paginate, whenever
Templates: ERB
Testing: RSpec(controllers, helpers, models)
Search: ThinkingSphinx


Homepage: http://gemcutter.org/
Github: http://github.com/qrush/gemcutter

“Awesome gem hosting”. Good source of cucumber story examples. Also a few Rack examples, including middleware that uses Sinatra to serve files from Amazon’s S3 service. Also some memcache action going on, and some delayed_jobs.

Authentication: clearance
Plugins: pacecar, will_paginate, high_voltage, jrails
Templates: ERB, also uses the 960.gs grid framework
Testing: test unit(functional, unit), Factory Girl, shoulda, cucumber
Search: named_scopes


Homepage: http://www.spot.us/
Github: http://github.com/spot-us/spot-us

Crowdsourcing and Crowdfunding local news stories.

Authentication: restful-authentication
E-Commerce: active_merchant, ssl_requirement
Plugins: acts_as_state_machine(aasm), fastercsv, attribute_fu, paperclip, subdomain_fu
Templates: haml (no sass), compass
Testing: RSpec(models, views, controllers, helpers), Factory Girl


Homepage: http://spreecommerce.com/
Github: http://github.com/railsdog/spree

An excellent base to build ecommerce sites from.

Authentication: authlogic
E-Commerce: active_merchant, ssl_requirement
Plugins: pluginaweek-state_machine, will_paginate, whenever, chronic, acts-as-list, attribute_fu, awesome_nested_set, paperclip
Templating: haml, compass, sass
Testing: RSpec(controllers, models), test unit(functional, unit)


Homepage: http://bostonrb.org/
Github: http://github.com/bostonrb/bostonrb

This is a pretty cutting edge rails app. Follows ThoughtBot’s best practices.

Authentication: clearance
Plugins: inherited_resources, acts_as_versioned, autochronic, geokit, jrails
Templating: haml, sass
Testing: mocha, shoulda, cucumber, Factory Girl, webrat, fakeweb


Homepage: http://www.railsdevelopment.com/
Github: http://github.com/engineyard/rails_dev_directory

Here is a fresh new app just released from the guys at EngineYard. An web directory of professional Rails development firms. Good example for dealing with internationalization(I18N) and localization(L10n). Also many cucumber features.

Authentication: authlogic
Authorization: can_has
Plugins: acts_as_state_machine(aasm), acts-as-list, paperclip, recaptcha, redcloth, shortcode_url, ssl_requirement, will_paginate, xss_terminate, jrails
Search: ThinkingSphinx
Testing: RSpec(controllers, helpers, models), cucumber, Factory Girl


Homepage: http://dogfood.insoshi.com/
Github: http://github.com/insoshi/insoshi

An OpenSource social network platform in Rails

Authentication: restful-authentication, open_id_authentication
Plugins: acts-as-list, annotate_models, attachment_fu, jrails
Search: ultrasphinx
Testing: RSpec(controllers, helpers, models, views)

Bucketwise

Homepage: http://wiki.github.com/jamis/bucketwise
Github: http://github.com/jamis/bucketwise

A simple web based personal finance application. A pretty straight forward application that doesn’t leverage many plugins.

Testing: test unit(unit, functional)

Fat Free CRM

Homepage: http://fatfreecrm.com/
Github: http://github.com/michaeldv/fat_free_crm

Fat Free CRM is an open source Ruby on Rails-based customer relationship management platform. Out of the box it features group collaboration, campaign and lead management, contact lists, and opportunity tracking.

Authentication: authlogic
Plugins: acts_as_commentable, advanced_errors, annotate_models, paperclip, will_paginate
Search: simple_column_search
Templating: haml, sass
Testing: RSpec(controllers, helpers, models, routing, views), faker, Factory Girl


Rails autocompletion in MacVim when using Macports

words by Brian Racer

In moving much of my development over to OS X, I started receiving errors when trying to use vim’s omnicompletion in Rails projects. An excerpt from my vim config to enable that functionality looks like this:

" Turn on language specific omnifuncs
autocmd FileType ruby,eruby set omnifunc=rubycomplete#Complete
autocmd FileType ruby,eruby let g:rubycomplete_buffer_loading = 1
autocmd FileType ruby,eruby let g:rubycomplete_rails = 1
autocmd FileType ruby,eruby let g:rubycomplete_classes_in_global = 1
autocmd FileType ruby,eruby let g:rubycomplete_include_object = 1
autocmd FileType ruby,eruby let g:rubycomplete_include_objectspace = 1

When I tried to auto-complete something(Ctrl^X^O), I would receive the following error:

"-- Omni completion (^O^N^P) -- Searching...Rails requires RubyGems >= 1.3.5 (you have 1.0.1). Please `gem update --system` and try again. Error loading rails environment"

Long story short, I was using MacPort’s ruby/gem packages, but a binary snapshot of MacVim that I downloaded off their website was using the libraries that come with OSX. There is not really a clean workaround for that, but luckily it turns out macport’s macvim builds the latest snapshot. So all you need to is to install macvim with ruby support:

sudo port install macvim +ruby

And you will get nice auto-completion:

macvim_omnicomplete


Munin Graphs for Phusion Passenger (a.k.a. mod_rails)

words by Brian Racer

The goal of this article is to get fairly nice looking graphs of Phusion Passenger’s performance and memory metrics:


This specific setup focuses on CentOS (on cPanel none the less) – but instructions should apply for most linux distros. It assumes you already have Passenger and Munin successfully setup. See my previous article on getting Phusion Passenger setup if you have not already.

First we need to allow the munin user sudo privileges for a few passenger related commands:

$ sudo /usr/sbin/visudo
 
# Add the following line to the file
munin ALL=(ALL) NOPASSWD:/usr/bin/passenger-status, /usr/bin/passenger-memory-stats
 
# Depending on your setup, you may also have to comment out the following line:
Defaults requiretty

If you see the error sorry, you must have a tty to run sudo in /var/log/munin/munin-node.log, comment out the final line shown above.

The following two files will glean some performance and memory statistics.

Passenger Status:

sudo vi /usr/share/munin/plugins/passenger_status
#!/usr/bin/env ruby
 
def output_config
  puts <<-END
graph_category App
graph_title passenger status
graph_vlabel count
 
sessions.label sessions
max.label max processes
running.label running processes
active.label active processes
END
  exit 0
end
 
def output_values
  status = `sudo /usr/bin/passenger-status`
  unless $?.success?
    $stderr.puts "failed executing passenger-status"
    exit 1
  end
  status =~ /max\s+=\s+(\d+)/
  puts "max.value #{$1}"
 
  status =~ /count\s+=\s+(\d+)/
  puts "running.value #{$1}"
 
  status =~ /active\s+=\s+(\d+)/
  puts "active.value #{$1}"
 
  total_sessions = 0
  status.scan(/Sessions: (\d+)/).flatten.each { |count| total_sessions += count.to_i }
  puts "sessions.value #{total_sessions}"
end
 
if ARGV[0] == "config"
  output_config
else
  output_values
end

Memory Stats:

sudo vi /usr/share/munin/plugins/passenger_memory_status
#!/usr/bin/env ruby
# put in /etc/munin/plugins and restart munin-node
# by Dan Manges, http://www.dcmanges.com/blog/rails-application-visualization-with-munin
# NOTE: you might need to add munin to allow passwordless sudo for passenger-memory-stats
 
def output_config
  puts <<-END
graph_category App
graph_title Passenger memory stats
graph_vlabel count
 
memory.label memory
END
  exit 0
end
 
def output_values
  status = `sudo /usr/bin/passenger-memory-stats | tail -1`
  unless $?.success?
    $stderr.puts "failed executing passenger-memory-stats"
    exit 1
  end
  status =~ /(\d+\.\d+)/
  puts "memory.value #{$1}"
end
 
if ARGV[0] == "config"
  output_config
else
  output_values
end

Now we will link these to the active plugins, and make them executable:

sudo chmod +x /usr/share/munin/plugins/passenger_status
sudo chmod +x /usr/share/munin/plugins/passenger_memory_status
sudo ln -s /usr/share/munin/plugins/passenger_status /etc/munin/plugins/passenger_status
sudo ln -s /usr/share/munin/plugins/passenger_memory_status /etc/munin/plugins/passenger_memory_status

Last thing we need to do is make sure those scripts run as the munin user:

sudo vi /etc/munin/plugin-conf.d/munin-node
[passenger_*]
user munin
command ruby %c

Restart the munin node, and wait and you should see the graphs start to propagate.

sudo /etc/init.d/munin-node restart

For even more detailed performance analytics, checkout NewRelic monitoring. And thanks to Dan Mange for the munin scripts.


pbcopy / pbpaste in Ubuntu (command line clipboard)

words by Brian Racer

OS X has a neat command-line tool called pbcopy which takes the standard input and places it in the clipboard to paste into other applications.

In Ubuntu(or any Linux distro with Xwindows), a similar tool is xclip. I like to make this alias:

alias pbcopy='xclip -selection clipboard'
alias pbpaste='xclip -selection clipboard -o'

or the following also works if you would rather use xsel:

alias pbcopy='xsel --clipboard --input'
alias pbpaste='xsel --clipboard --output'

Now you can pipe any text to pbcopy

$ cat ~/.ssh/id_dsa.pub | pbcopy

Your public ssh key is transferred to your clipboard and is ready to be pasted(perhaps with pbpaste).


Clean Webalizer Referrer Stats in cPanel

words by Brian Racer

I find it useful to override some of cPanel’s default webalizer configurations. There can be a lot of misleading or unnecessary data when you don’t ignore traffic that is being referred to from your own site.

Create the following file:

vi /home/username/tmp/webalizer/webalizer.conf
HideSite *example.com
HideSite localhost
HideReferrer example.com/
MangleAgents 4
 
TopSites        10
TopKSites       10
TopURLs         10
TopKURLs        50
TopReferrers    250
TopAgents       15
TopCountries    10
TopEntry        10
TopExit         10
TopSearch       20
TopUsers        20
 
# Usually you want to hide these
HideURL         *.gif
HideURL         *.GIF
HideURL         *.jpg
HideURL         *.JPG
HideURL         *.png
HideURL         *.PNG
HideURL         *.bmp
HideURL         *.BMP

Replace example.com with the domain you are working on. The next time webalizer runs it will not be clogged with useless referrers generated by your site. To force webalizer to run(in a cPanel environment), execute:

/scripts/runweblogs username

Textmate and GNU screen error in Leopard

words by Brian Racer

When running mate from inside a GNU screen session in Leopard, I was getting this error:

$ mate .
mate: failed to establish connection with TextMate.

I read reports of falling back to a previous version in macports, but a simple solution is to put this line in your .profile config:

alias mate='open -a TextMate.app'

That will fix the problem.