Deployment with Vlad

When I ask people about how they deploy their Rails apps, most of them answer that they are using good old Capistrano. It is the most popular solution because it has been around since almost when people started deploying Rails applications and it works well in most cases. However, if you want to do a lot of custom tasks and need to look under the hood to extend it, Capistrano seems somewhat complicated. At least it did at the time I left it in favor of Vlad - an imho more intuitive deployment tool.

Though one might not like the offense against Capistrano that was shown when Vlad was released in 2007, it addresses some valid points when arguing against needless complexity. But enough of the history, this was almost five years ago, both of the tools have improved, made their way and all I wanted to share is some snippets and insights I got from using Vlad over the last years.

The main thing I like abot Vlad is that it is based on Rake, which makes it so easy to learn, understand and extend. If you want to do something custom, all you have to do is to write a Rake task. And as you will want to use them across different projects, you can even bundle them up and distribute them as a gem, like I’ve done with vlad-extras, a set of extensions for tools like DelayedJob, ThinkingSphinx, Monit, Whenever and so on.

Here is what a simple config/deploy.rb file might look like:

# Custom variables used in the following config
set :application, "appname"

# Required variables
set :domain, "#{application}.com"
set :deploy_to, "/var/www/#{application}"
set :repository, "git@github.com:myaccount/#{application}.git"

# Optional variables
set :user, "deploy" # if different from your current login
set :bundle_cmd, "/usr/local/bin/bundle"
set :rake_cmd, "#{bundle_cmd} exec rake"
set :revision, "origin/master"

# vlad-extras config
set :copy_shared, {
  'config/maintenance.html' => 'config/maintenance.html',
  'config/database.yml'     => 'config/database.yml' }
set :symlinks, {
  'assets'              => 'public/assets',
  'config/database.yml' => 'config/database.yml' }
set :deploy_tasks, %w(
  vlad:maintenance:on
  vlad:update
  vlad:symlink
  vlad:bundle:install
  vlad:assets:precompile
  vlad:migrate
  vlad:start_app
  vlad:maintenance:off
  vlad:cleanup)

# Bundler ships with vlad integration you'll have to require
# in order to use the vlad:bundle tasks
require 'bundler/vlad'

# Require custom vlad tasks or recipes from vlad-extras
require 'vlad/maintenance'

# Some custom tasks, can be included directly in the deploy file
namespace :vlad do
  namespace :custom_script do
    %w(start stop restart).each do |task|
      desc "#{task.capitalize} the custom script"
      remote_task task, :roles => :app do
        run "cd #{current_release}; RAILS_ENV=#{rails_env}
          #{bundle_cmd} exec ruby script/custom #{task}"
      end
    end
  end
end

Here is an example of how easy it is for instance to add staging support:

task :staging do
  set :rails_env, "staging"
  set :deploy_to, "/var/www/#{application}-#{rails_env}"
end

task :production do
  set :rails_env, "production"
  set :deploy_to, "/var/www/#{application}-#{rails_env}"
end

All you need to do is to change some variables based on the environment that is set with a simple Rake task. You’d invoke it by running

$ bundle exec rake staging vlad:deploy

Gemfile

You can include Vlad and it’s requirements in the development group, because it you do not need it in any other environment. Requiring is done in the Rakefile like shown below.

group :development do
  # Deployment
  gem 'vlad', :require => false
  gem 'vlad-git', :require => false
  gem 'vlad-extras', :require => false
end

Rakefile

Add this snippet to your Rakefile, just before MyApp::Application.load_tasks

if Rails.env.development?
  begin
    require 'vlad'
    require 'vlad-extras'
    Vlad.load(scm: :git, web: :nginx, app: :passenger, type: :rails)
  rescue LoadError
    puts 'Could not load Vlad'
  end
end

You might have to configure the options given to Vlad.load.

If you want to get started, take a look at the documentation to find out about the parts that this post leaves out.

Troubleshooting

This part is about some trouble I’ve had and it is not really Vlad specific. You might encounter these problems when trying to deploy to your production server and maybe these tips and links will help.

The deployment shell environment

If Vlad does not find some commands, try the following:

To enable per user PATH environments for ssh logins you need to add to your sshd_config:

PermitUserEnvironment yes

After that, restart sshd!

Then in your users ssh home directory (~/.ssh/environment), add something to this effect (your mileage will vary):

PATH=/opt/ruby-1.9.3/bin:/usr/local/bin:/bin:/usr/bin

For details on that, see this article on setting the deployment shell environment

SSH Agent Forwarding

Maybe you also need to configure SSH Agent Forwarding:

$ ssh-add ~/.ssh/<private_keyname>

Edit your ~/.ssh/config file and add something like this:

Host <name>
  HostName <ip or host>
  User <username>
  IdentityFile ~/.ssh/<filename>
  ForwardAgent yes

For details on that see this article on SSH Agent Forwarding

Seeds for different environments

Here’s a little Rails tip for splitting up seed data for various environments: Create the folder db/seeds/. For each environment you want to seed put a file in there named after the env. Your general seeds are put into db/seeds/all.rb. Here’s what it might look like:

|___seeds
| |___all.rb
| |___development.rb
| |___staging.rb
| |___production.rb
|___seeds.rb

Change the content of your db/seeds.rb to something like this:

['all', Rails.env].each do |seed|
  seed_file = "#{Rails.root}/db/seeds/#{seed}.rb"
  if File.exists?(seed_file)
    puts "*** Loading #{seed} seed data"
    require seed_file
  end
end

Running rake db:seed will now load up the seed data defined in db/seeds/all.rb and the current environment.

Configuring Nginx for the Asset Pipeline

Following up on the series of posts about the Rails Asset Pipeline, here are some things to keep in mind when you are using Nginx as your webserver.

Installation via Passenger

When you are installing Nginx automatically with the script that comes with Passenger, you will not be able to serve the gzipped file variants that get precompiled. That’s because Passenger currently does not configure Nginx with the http_gzip_static_module that would be necessary for including the directive gzip_static on. I’ve opened a pull request for that, but for now you’ll have to choose the manual installation, if that’s what you are after - and why wouldn’t you?

# download and untar nginx
wget -O /tmp/nginx.tar.gz http://www.nginx.org/download/nginx-1.0.10.tar.gz
tar xzvf /tmp/nginx.tar.gz
rm /tmp/nginx.tar.gz

# install nginx via passenger install script, already includes
# --with-http_ssl_module
# --add-module for passenger
passenger-install-nginx-module --auto --prefix='/opt/nginx' --nginx-source-dir='/tmp/nginx-1.0.10' --extra-configure-flags="--with-http_gzip_static_module"

You might want to change the version and prefix to fit your needs. Like mentioned in the comments you don’t have to take care of including the passenger or ssl module, because the installation script adds them automatically.

Nginx configuration

This is already part of the Asset Pipeline Guide, but I’ll mention it here again, so that this article is complete - here are the directives for the assets location config:

location ~ ^/(assets)/  {
  root /path/to/site;
  gzip_static on;
  expires max;
  add_header Cache-Control public;
  # access_log /dev/null;
}

I’ve also disabled access logging, so that asset requests do not clutter up the log.

Mathias brought up a good point, why one might not want to do that.

Sendfile Header

In your app’s production.rb you should uncomment to following line, to let Nginx send static files:

config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect'

Verify that it’s working

After deploying the changes, you can check if Nginx is now serving the gzipped compressed versions of your asset files. For that you can use tools like this HTTP Compression Tester and enter the address of your application.js and application.css

Referencing Rails Assets in CoffeeScript

Sometimes you’ll want to refer to your image assets from inside of your CoffeeScript (or JavaScript). While we have nice helpers for that in SASS (namely image-path and image-url), we would have to resort to appending .erb to every .js or .coffee file, we want to reference images in, so that we could use something like this in there:

icon = <%= image_path('icon.png') %>

I didn’t like it that way, because ERB inside of CoffeeScript looks odd and having the file end with .erb messes up syntax highlighting. But we can circumvent this easily by adding this (admittedly not the most beautiful) piece of code to one single file that ends on .erb (i.e. images.js.coffee.erb) and let it provide the helper method we need:

<%
imgs = {}
Dir.chdir("#{Rails.root}/app/assets/images/") do
  imgs = Dir["**"].inject({}) {|h,f| h.merge! f => image_path(f)}
end
%>

window.image_path = (name) ->
  <%= imgs.to_json %>[name]

This adds up all images inside of app/assets/images and makes them available via the global helper method image_path - your milage may vary, but you get the idea. From there on it’s simply

icon = image_path('icon.png')

inside of your CoffeeScript (or JavaScript) files and it also works in production as the returned filename contains the digest.

OpenData Bremen

Im Rahmen des OpenData-Wettbewerbs Apps4Deutschland fand in Bremen am Freitag, den 11. November 2011 die Auftaktveranstaltung zu Apps4Bremen statt. Nach dem Vorbild Berlin macht nun auch das Land Bremen Teile der Daten staatlicher Stellen öffentlich und läd dazu ein, diese zu verwerten und sinnvoll aufzubereiten - auf der Apps4Deutschland-Website heißt es dazu:

Behörden in Deutschland erheben, speichern und verarbeiten viele interessante Daten. Dazu gehören Wetterdaten ebenso wie Informationen über die Luftqualität oder über die Verwendung der Steuergelder. Diese Daten sind von öffentlichem Interesse und bergen ein großes Potenzial für unsere Gesellschaft, weil sie Transparenz, Beteiligung, Innovationen und neue Dienstleistungen fördern.

Mit dem Wettbewerb „Apps für Deutschland“ laden wir Designer, Entwickler, Journalisten, Forscher und die breite Öffentlichkeit ein, Anwendungen zu schaffen, um diese Daten nutzbar zu machen.

Da ich das Thema Open Government spannend finde und für sehr wichtig halte, war der Wettberwerb ein guter Anlass, um mir eine neue Spielwiese für die offenen Daten Bremens zu schaffen. Das Projekt besteht aktuell nur aus einer Übersicht der Bremer Kindertagesstätten, lässt sich aber auf dem bisherigen Stand gut erweitern, da es bereits Beispiele für Datenimporter, Geokodierung und Darstellung von Orten auf einer Google Map enthält. Basis dafür sind Ruby on Rails und Backbone.js und der Quelltext ist auf GitHub verfügbar, so dass das Projekt gerne erweitert werden kann. Gerne lasse ich mich auch auf Ideen, Verbesserungswünsche oder Fehler hinweisen.

Wer sich inspirieren lassen möchte, kann auch mal einen Blick in den OpenData Showroom werfen, der Projekte aus aller Welt enthält, die sich offener Daten bedienen. Würde mich freuen, wenn sich viele weitere Leute finden, die mit den Daten was nettes auf die Beine stellen.

Precompiling Rails Assets for Development

First off: I love Rails’ asset pipeline from the bottom of my heart. It’s a great thing and makes frontend development such a lovely task. There are two little drawbacks though when using it in development: It slows down page reloads and clutters up the log output. Here I’ll show you how to get rid of that.

When you are mainly doing backend development or most of your apps frontend code doesn’t change too much do yourself a favor and precompile your assets for the development environment:

$ RAILS_ENV=development bundle exec rake assets:precompile

This really speeds up page reloads as Rails does not need to look up and precompile the asset files every time a request hits the app anymore.

Another change you might want to make is to set config.assets.debug to false in your development environment. This makes Sprocket concatenate all your files (users.css, search.css, …) into one javascript and stylesheet file (application.css):

config.assets.debug = false

This also cleans up the noisy logging info concerning the asset files that were previously loaded one by one.

When you need to precompile additional assets that aren’t already referenced in you application.js or application.css you’ll want to move the config.assets.precompile directive from production.rb to application.rb:

config.assets.precompile += %w( search.js )

This isn’t directly related but while you are at speeding up our development experience you might also want to have a look at the rails-dev-tweaks gem. This gem defines rules for managing what files get reloaded on page requests in development.

Setting up Ubuntu with Teleport

Some time ago I came across Teleport which pitches itself as a lightweight way to set up Ubuntu machines. Having failed at mastering the steep learning curve that full-blown tools like Chef or Puppet provide, I was willing to give Teleport a shot. Teleport stays simple by focusing on Ubuntu and providing just the right set of tools to set up the whole Ruby/Rails eco system. This makes it less powerful than the other aforemention tools, but in my case it’s just what I was looking for.

Some facts: Teleport …

  • consists of a configuration (Telfile) plus files you want to be present on the target
  • copies itself onto the target via ssh and then runs itself there (hence the name)
  • works idempotently - run it over and over again without breaking the target

My setup

For each project I store everything that belongs to Teleport in a specific deployment folder. This folder might look something like this:

|____Telfile
|____recipes
| |____install_passenger_nginx.rb
|____files
| |____etc
| | |____gemrc
| | |____init.d
| | | |____nginx
| |____Gemfile
| |____Gemfile.lock
| |____opt
| | |____nginx
| | | |____conf
| | | | |____nginx.conf

The configuration is done inside the Telfile:

user "deploy"
ruby "1.9.3"
server "my-own-vps"

apt "deb http://ppa.launchpad.net/chris-lea/node.js/ubuntu lucid main", key: "C7917B12"

packages %w(openssl libreadline5 libreadline5-dev curl
  git-core libopenssl-ruby libcurl4-openssl-dev imagemagick
  libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev
  libmysqlclient-dev mysql-client mysql-server nodejs)

recipes %w(
  install_passenger_nginx.rb)

user defines the name of the account you will later use to deploy your app. This user gets added to the sudoers and your ssh key is uploaded for password-less login. ruby tells Teleport which version to install and server is the name you gave to the machine in your ~/.ssh/config.

As you can see you can also define customs apts and packages that should get installed. For performing special setup routines or invoking more complex tasks you can utilize the recipes feature. Recipes are standalone files that can be written in Ruby and allow for sharing and reuse. Here’s an example of how to install nginx via the passenger gem:

version_gem = gem_version("passenger")
version_installed = `passenger-config --version`.strip

if !version_gem
  warn "Passenger is not installed. Please add it to your Gemfile."
elsif version_installed == version_gem.to_s
  banner "Passenger #{version_installed} is already installed"
else
  banner "Installing Nginx via Passenger (installed: #{version_installed}, gem version: #{version_gem})"
  run "passenger-install-nginx-module", %w(--auto --auto-download --prefix=/opt/nginx)  # install without prompting
  run "update-rc.d nginx defaults"                                                      # activate autostart
  run "service nginx restart"                                                           # restart nginx
end

The files directory contains files and directories that get copied over to the target machine - not much magic about that. To install system-wide gems you can place a Gemfile and Gemfile.lock inside the files directory.

Finally you can run the setup with

$ teleport my-own-vps

Keep in mind that you can re-run this every time you make a change to the Telfile or add new files. Teleport strives to be idempotent and can be run repeatedly without breaking the target machine(unless you explicitely messed up something ;)).

Let’s go…

There is much more to Teleport than what I could cover in this short post. For example you can define multiple servers with different roles or reverse engineering existing servers which makes it very easy to generate a Telfile from your current setup. In case you want to get started head over to the project wiki which contains a lot of information and examples.

Capybara Finder for Cucumber Rails’ Deprecated Tableish

The cucumber-rails gem recently deprecated the tableish method which could be used to verify data in tables using Table#diff!. Tableish was easy to use but also had some limitations so that from now on you are advised to use Capybara’s finder API which is much more flexible.

In case you are now searching for a replacement for tableish, look no further:

rows = find("table#selector").all('tr')
table = rows.map { |r| r.all('th,td').map { |c| c.text.strip } }
expected_table.diff!(table)

This maps the table into a two dimensional array containing all cell contents. The last line diffs this array against the expected_table which is the table that’s passed into the step definition like explained on this Cucumber wiki page.

You’ll have to replace table#selector with the actual table selector you are looking for.

When using capybara-webkit for testing elements that get inserted into the page via JavaScript I had to change this a little bit, because the finder did not seem to work with capybara-webkit. One can circumvent this by parsing the page body and using the finder on the new result:

html = Capybara::Node::Simple.new(body)
rows = html.find("table#selector").all('tr')
table = rows.map { |r| r.all('th,td').map { |c| c.text.strip } }
expected_table.diff!(table)

Programmiert Ihr Deutsch?

Mit diesem Post würde ich gerne ein Thema wieder aufgreifen, dass ich vor drei Jahren einmal angeschnitten habe: Programmiert Ihr Deutsch?

Grund dafür ist, dass wir in zwei aktuellen Projekten dazu übergegangen sind, alles was mit der Domäne und Tests zu tun hat komplett in deutsch zu entwickeln. Nach anfänglicher Skepsis darüber muss ich nun sagen, dass ich meine Meinung deutlich geändert habe und es in jedem kommenden Projekt wohl wieder so machen würde – es sei denn natürlich, es gibt Beschränkungen bzw. gute Gründe eben nicht deutsch als Sprache zu verwenden (bspw. englischsprachige Teammitglieder oder Kunden).

Wir haben Modell-, Controller- und Attributnamen auf deutsch definiert, was auch erstaunlich problemlos funktioniert. Die einzigen Anpassungen die in Rails daraus resultieren sind, dass es erfordert die Inflections (config/initializers/inflections.rb) anzupassen. Alles weitere (bspw. Datenbanktabellennamen) resultiert daraus und benötigt weiter keine Konfiguration. Wir waren bislang noch nicht wagemutig genug das auf die Spitze zu treiben und auch Umlaute in Klassen- und Dateinamen zu verwenden, das bleibt dann dem geneigten Leser als Übung überlassen ;)

Testing

Richtig angenehm wird es in den Tests: Cucumber bringt von Haus aus mit, Features auch in anderen Sprachen als Englisch zu verfassen. Alles was dazu nötig ist, ist bei der Installation das Kürzel der Sprache mitzugeben, für die zusätzliche Web Steps generiert werden sollen, in unserem Fall:

rails generate cucumber:install de

Eine Übersicht der in deutsch benötigten Keywords gibt es mittels

bundle exec cucumber --i18n de

Die Testdaten legen wir über Tabellen an, was meiner Meinung nach mittlerweile auch etwas übersichtlicher ist als Bibliotheken zu verwenden, die Factory Steps generieren (wie cucumber_factory oder pickle). Dies ist aber sicherlich Geschmacksache und es gäbe auch Möglichkeiten, die Stepgeneratoren so anzupassen, dass sie auch für deutsche Steps funktionieren. Hier ein kurzes Beispiel für eines unserer Features (wichtig für Cucumber ist dabei der Kommentar in der ersten Zeile):

# language: de

Funktionalität: Kunde-Login
  Als Kunde
  Möchte ich mich mit meinen Zugangsdaten einloggen
  Um an der Auktion teilnehmen zu können

  Grundlage: Bestehender Kunde
    Angenommen es gibt die folgenden Kunden:
      | email        | kundennummer | password   |
      | kunde@web.de | 123456       | top_secret |

  Szenario: Erfolgreiches Login mit E-Mail-Adresse
    Wenn ich auf die Login-Seite gehe
    Und ich "Login" mit "kunde@web.de" ausfülle
    Und ich "Passwort" mit "top_secret" ausfülle
    Und ich auf "Anmelden" drücke
    Dann sollte ich auf der Startseite sein
    Und ich sollte "Sie wurden erfolgreich eingeloggt" sehen

Auch die Specs lassen sich bis auf die Keywords sehr gut auf deutsch schreiben, was bei uns dann in etwa so aussieht:

# encoding: utf-8

require 'spec_helper'

describe Auktionator do
  
  context "bei bevorstehender Auktion" do
    
    before do
      @termin = Factory(:auktionstermin, :start => 1.day.from_now)
      @los1 = Factory(:los, auktionstermin: @termin, losnummer: 1)
      @los2 = Factory(:los, auktionstermin: @termin, losnummer: 2)
      @auktionator = Auktionator.new(@termin)
    end
    
    describe "#phase" do
      
      it "sollte 'auktion_bevorstehend' zurückgeben" do
        @auktionator.phase.should == 'auktion_bevorstehend'
        @auktionator.auktion_bevorstehend?.should be_true
      end
    
    end

    describe "#los" do
    
      it "sollte das aktuelle Los zurückgeben" do
        @auktionator.los.should == @los1
      end
    
    end

end

Mich würde wie gesagt eure Meinung dazu interessieren, vielleicht habt ihr Tips und weitere Tools dafür. Wer darüber hinaus auf der Suche nach Anregungen zum Thema Domänenabbildung ist, dem sei ggf. noch Eric Evans’ Buch Domain Driven Design empfohlen - dies aber wohl eher nur als Randnotiz.

Silencing the Rails log on a per-action basis

In our current project we are having an action that is polled every ten seconds by the clients. This clutters up the log, so I was looking for a way to completely silence the Rails logger for this action. It turns out that you have to swap the default logger with a custom one, because even when you use the Rails.logger.silence block in your action, you will still find a line in your log that is printed out before the request gets dispatched.

Here’s our custon logging class:

# lib/custom_logger.rb
class CustomLogger < Rails::Rack::Logger
  def initialize(app, opts = {})
    @app = app
    @opts = opts
    @opts[:silenced] ||= []
  end
  
  def call(env)
    if env['X-SILENCE-LOGGER'] || @opts[:silenced].include?(env['PATH_INFO'])
      Rails.logger.silence do
        @app.call(env)
      end
    else
      super(env)
    end
  end
end

This way you can either define request paths that should not get logged or send the X-SILENCE-LOGGER header for the request you don’t want to log.

To make use of your own logger, you’ll have to swap the default logger, which is included as a Rack middleware:

# config/application.rb
require File.dirname(__FILE__) + '/../lib/custom_logger.rb'

module MyApp
  class Application < Rails::Application
    # Middlewares
    config.middleware.swap Rails::Rack::Logger, CustomLogger, :silenced => ["/noisy/action.json"]
  end
end

Using Cucumber Step Argument Transforms

In a recent project that is tested with Cucumber we were using „within“ steps with CSS selectors. Though this isn’t basically considered a good idea I’d say it made sense for us, because we had to select specific parts of the page. Cuking it like that, our steps usually looked something like this:

Scenario: Edit something
  Given there is something
  And another something
  When I go to the page of somethings
  And I follow "edit" within "ul.somethings li:nth-child(1)"
  Then I should be on the edit page for "something"

Besides looking just ugly, CSS selectors also don’t offer any business value and are likely to change. Cucumber’s Step Argument Transforms can help you avoid that by refactoring common operations.

Transforms are defined with the Transform keyword and can be a part of your step definition files. Cucumber matches the captures in your steps (i.e. the 1st something) against the defined transforms. If the regexp matches, it yields the transformed value as an argument to the step definition block. This sounds complicated, so here’s an example:

Transform /^the (\d+)(?:st|nd|rd|th) something$/ do |num|
  "ul.somethings li:nth-child(#{num})"
end

This transform allows us to refactor the scenario above to be more readable and maintainable:

Scenario: Edit something

  When I go to the page of somethings
  And I follow "edit" within "the 1st something"

We can even abstract this a little further and make it usable on a common level:

Transform /^the (\d+)(?:st|nd|rd|th) (.+)$/ do |num, type|
  "ul.#{type.pluralize} li:nth-child(#{num})"
end

This is just a basic example of how this often unknown Cucumber feature might help you to refactor your step definitions. For more information and decent examples check out the wiki page on Step Argument Transforms.

Ältere Artikel im Archiv