blog.omdb-beta.org

One of the great features of ruby (and rails) is the ability to automatically create code. A lot of our controller code is generated automatically by some generic functions. For example to add our ‘edit-abstract’ functionality for movies, we just need to add a

set_abstract_for :movie

to our controller. While it took us some time to genericize our controller code, we now have less LoC, an investment, that will surely pay off. While rewriting some of our controller methods, we realized that we try hard to keep our controller DRY, while we repeat ourselves over and over again when writing test cases.

The solution to this is to write helpers for testcases, just like writing helpers for the controllers. We’ve added a test_omdb_helper.rb to the test/ directory and required it in the standard ‘test_helper.rb’ file. One of our first test-case-generators was the view_test_for. This helper method might be of interest to someone out there as it is not OMDB-related. Its purpose is simply to test all of our views. OMDB has a lot of different views but not all of our views have special functionalities. For example, the statistic views for a category or the filmography of a person just displays simple data. Our goal was to be able to test every single view without writing something like that all the time:

def test_movie_index
  get :index, :id => movies(:king_kong).id
  assert_response :success
end

So we started with a small method, implementing exactly the logic above:

def view_test_for( klass, id, opts = {} )
  define_method( "test_view_#{options[:action]}_#{id.to_s}" ) do
    # another way of saying: movies(:king_kong) :-)
    object = self.send(klass.to_s.pluralize, id)
    get options[:action], :id => object.id
    assert_response :success
  end
end

Now we can test all of our views with just a few lines in our movie_controller_test.rb:

view_test_for :movie, :king_kong, :action => :index
view_test_for :movie, :king_kong, :action => :cast
view_test_for :movie, :king_kong, :action => :review

This was a good start of the basic views but it has his flaws in flexiblity, e.g. if you want to do more than just know that it does not throw any exception. One small extension added to the method above will allow you to add custom code to each of the view_tests.

def view_test_for( klass, id, opts = {} )
  define_method( "test_view_#{options[:action]}_#{id.to_s}" ) do
    object = self.send(klass.to_s.pluralize, id)
    get options[:action], :id => object.id
    assert_response :success
    yield self if block_given?
  end
end

I do not know about your experience with ruby, at least i needed some time to feel comfortable with blocks and yields, but now its great to work with them. Lucas Carlson and Leonard Richardson write in their Ruby Cookbook (which I highly recommend):

The yield keyword acts like a special method, a stand-in for whatever code block was passed in. [...] This may seem mysterious if you’re unfamiliar with the practice of passing blocks around, but it is usually the preferred method of calling blocks in Ruby.

You should definitely take a closer look to yield, if you’re not familiar with it. We can now pass in a few lines of code, that will be evalutated inside the test, for example checking for a certain tag:

view_test_for :movie, :king_kong, :action => :cast do |test_case|
  test_case.assert_tag :tag => :a,
      :attributes => { :href => "http://test.host/person/" +
                        instance.people(:peter_jackson).id.to_s }
end

Now that helps a lot to write more complex tests. Actually we found a lot of options, we wanted to pass in to our view-test. Here’s another example

view_test_for :movie, :king_kong, :action => :info, :template => 'info.rhtml',
                                  :xhr => true, :method =>:post do |test_case|
  test_case.assert_tag :tag => :a, :content => instance.people(:peter_jackson).name
end

Here’s the whole test_view_for-method, one of the options is OMDB-related, you might want to remove this option. We placed most of our templates in a common/ directory. So we added a test for the template name, if it uses a common template or one of the controller-view-directories (e.g. view/movie vs. view/common, that is).

def view_test_for( klass, id, opts = {} )
  options = { :action       => :index, 
    :template     => 'overview.rhtml',
    :common_template => false,
    :method       => :get,
    :fetch_method => klass.to_s.pluralize,
    :xhr          => false, 
    :response     => :success,
    :login        => nil }
  options.update(opts)
  define_method( "test_view_#{options[:action]}_#{id.to_s}" ) do
    object = self.send(options[:fetch_method], id)
    login_as options[:login] unless options[:login].nil?
    if options[:xhr] 
      xhr options[:method], options[:action], :id => object.id
    else
      self.send( options[:method], options[:action], :id => object.id )
    end
    assert_response options[:response]
    assert_template options[:template] if options[:response] == :success
    assert @response.rendered_file.include?("views/common/") if options[:common_template]
    yield self if block_given?
  end
end

If your application relies on server-based live searches, you’ll notice that using database queries is far too slow. On OMBD we implemented our first live search tests (some time back in May 2006) with LIKE statements and MySQL full text columns. But both solutions were far too slow, leaving us wondering, how to speed up live searches. Fortunately for us, we discovered Ferret, maybe the only search engine for ruby worth mentioning as of this day.

We used Ferret as an indexer to find the appropriate ActiveRecord-Objects, basically just storing the class and id of the AR-Objects. So, whenever a live search is performed by a user, the ferret index will be queried and the appropriate AR-Objects fetched from the database.

If you are still using SQL-Statements for searches, you should really take a look at the acts_as_ferret plugin for rails. It encapsulates the logic i described above. Jens did a fantastic job writing a plugin that allows you to include a really fast search engine into your project with ease.

However, if you want to do a live search there is no need to fetch the whole object from the database. Most of the time a live search result is just a small title or a name. In OMDB we’ve implemented a lot of live searches, so for example if you want to attach a new cast to a movie, you do a livesearch for a person and for a job. If you’re adding a lot of casts, thats a whole lot of SQL statements. So we thought about removing these SQL Statements by using Daves LazyDocs.

A LazyDoc is one of the newer features of ferret. Whenever you store something in the ferret index you basically just store fields and values, a hash that is. Ferret will return those hashes as results to you, whenever you run a query. So we added a lot more fields to the ferret index, allowing us to fetch all the data we need for our live searches.

In order to keep our templates DRY we wanted to make sure a template does not need to know whether it is currently working with a LazyDoc or with a real AR-Object. Therefore we implemented a small extension to the LazyDoc class.

First of all we need some way to convert a LazyDoc to an AR-Object. We’re storing the class of the AR-Object as the ferret key :type, and the database-id as :object_id, therefore these small methods (extension to LazyDoc) will return the class and the id of the stored object.

def class
  self[:type].constantize
end

def id
  self[:object_id].to_i
end

Having the class and the id we can then fetch the object from the database.

def to_o
  self.class.find( self[:object_id].to_i )
end

Now we can convert a LazyDoc to an AR-Object by calling the to_o method. Most of the AR-Object data is stored in the index, but some complex information or relations cannot or should not be stored in ferret – at least for now. So if an information is missing in the index, we can fallback to the AR-Object and get the information from there Furthermore it would be great to be able to use all of these nifty methods that ActiveRecord is creating automatically from the database fields. A LazyDoc does not know about these methods. But by implementing a method_missing method for LazyDocs, we can automatically fall back on the AR-Object, whenever neccessary.

def method_missing( method_name, *args )
   # Fetch the information from the ferret index.
   value = self[method_name]
   # Return the information from the database, 
   # if there is nothing in the ferret index
   # A LazyDoc - like a Hash - will return nil, if the
   # key is unknown.
   to_o.send(method_name, *args ) if value.nil?
end

Now our live searches can use LazyDocs, reducing our sql statements for live searches to almost zero. It increased the performance for live searches dramatically. Our next step will be to memchache LazyDocs, so we don’t need to fetch the information from disk for every search.