A story about building an application

I was recently working on a Ruby on Rails application that had a section for sending messages. This sounds pretty easy, right? I started with a User model and a Message model and some basic associations:

 class User < ActiveRecord::Base
    has_many :messages
 end

 class Message < ActiveRecord::Base
    belongs_to :user
 end  

Fig.1 - initial User and Message models

But when it came time to actually start building the application, I found this simple model code was not enough. The devil is in the details, as they say. There was a lot of functionality I needed to add beyond just a list of messages connected to a User.

1) Filtered views

I needed different views of the messages such as sent messages, drafts, and deleted messages.

How do I determine 'draft' status? Well one way is to fill in a delivered_at date whenever a message is sent. Then a draft is just a Message with no delivered_at date.

So after adding that field (and a sender_id and receiver_id) to the database I went to my messages_controller.rb file and added a few methods that looked sort of like this:

 def index
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is not NULL and recipient_id = ?', 
       user.id])
 end
 
 def sent_mail
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is not NULL and sender_id = ?', 
       user.id])
 end
 
 def drafts 
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is NULL and sender_id = ?', 
       user.id])
 end

Fig.2 - initial fragment from messages_controller.rb

2) Pagination

Nobody wants to load a page of 1000 messages at a time, so I needed to be able to break up that list into limited sized chunks. I used the excellent plugin will_paginate for that purpose. Then my controllers methods got a little more verbose:

 
 def index
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is not NULL and recipient_id = ?', 
       user.id]
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end

 def sent_mail
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is not NULL and sender_id = ?',
       user.id]
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end
 
 def drafts 
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is NULL and sender_id = ?', 
       user.id]
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end

Fig.3 - fragment from messages_controller.rb with pagination

3) The ability to flag content (i.e. spam, objectionable content etc...)

What if someone gets spam in the message system - or something objectionable in some other way. Well I need to filter that stuff out. I added a Flag model and connected that to messages like so:

  class Message < ActiveRecord::Base
    has_many :flags
    belongs_to :user
  end

Fig.4 - Message model with flags added

However, at this point my controller methods are starting to look like this:

 def sent_mail
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is not NULL and 
         flags.flagged_item_id is NULL and 
         recipient_id = ?', user.id], 
       :include => :flags
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end

 def sent_mail
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is not NULL and 
         flags.flagged_item_id is NULL and 
         sender_id = ?', user.id], 
       :include => :flags
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end
 
 def drafts 
   @messages = user.messages.find(:all, 
     :conditions => ['delivered_at is NULL and
         flags.flagged_item_id is NULL and 
         sender_id = ?', user.id], 
       :include => :flags
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end

Fig.5 - fragment from messages_controller.rb with flags

I'm looking at some ugly code - with a lot of repetition. How do I pare this down?

Begin Pruning

My first thought is that if anything in my application can be flagged, I should be able to do a little meta-programming to create a find method that will give me only un-flagged items. Ideally I could even send in all the rest of the find arguments exactly the same.

There is the named_scope addition to Rails 2.x that does just that - but I also want something I can add to any class as a Mixin. That way I can write code like this:

  Message.unflagged_items.find(:all, 
    :conditions => ['delivered_at is not NULL'])
  SomeOtherThing.unflagged_items.find(:all, :conditions => ...)

Fig.6 - call to imagined method unflagged_items

The method with_scope is a good candidate for sending in some pre-determined find conditions - but leaving it open to add more later. I'm wanting to add the following method to all my classes that need to be flagged:

  def unflagged_items(*args)  
    self.with_scope(:find => {
        :conditions => 'flags.flagged_item_id is NULL', 
        :include => :flags}) do  
      self.find(*args)
    end  
  end

Fig.7 - code for imaginary unflagged_items method

How do I do that? Well, I can turn that code into a Module and add it to any class automatically using a little metaprogramming:

  module Flaggable

    def self.included(base)
      base.class_eval do
        has_many :flags, :as => :flagged_item, :dependent => :destroy
      end
      base.extend(ClassMethods)
    end

    module ClassMethods

      def unflagged_items(*args)  
        self.with_scope(:find => {
            :conditions => 'flags.flagged_item_id is NULL', 
            :include => :flags}) do  
          self.find(*args)
        end  
      end  
    end

  end

Fig.8 - Flaggable module

Any model I put the line include Flaggable in will have that method available. So if I include it in the User class I've added a method user.messages.unflagged_items which returns a sort of incomplete version of the find function - with all the necessary logic to limit the list to unflagged items already filled in. I still have to fill in the :all or :first or any other :conditions I want. But the function is sort of half-called. This is a useful thing - getting half-called functions. In functional programming it's called currying. I'll come back to that in a moment.

Anyway, So now my controller methods now look like this:

 def index
    @messages = user.messages.unflagged_items(:all,
      :conditions => ['delivered_at is not NULL and recipient_id = ?', 
        user.id]
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end

 def sent_mail
    @messages = user.messages.unflagged_items(:all, 
      :conditions => ['delivered_at is not NULL and sender_id = ?', 
        user.id]
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end
 
 def drafts 
   @messages = user.messages.unflagged_items(:all, 
     :conditions => ['delivered_at is NULL and sender_id = ?', 
       user.id]
    ).paginate(:page => (params[:page] == "" ? 1 : params[:page]))
 end

Fig.9 - fragment from new messages_controller.rb

Continue Pruning

It's getting better, but isn't there some way I can pare it down even more? Now I'll go to the User model. Instead of simply using has_many :messages - since has_many supports blocks - I can add some more convenience methods to the User class:

  class User < ActiveRecord::Base

    has_many :received_messages, :foreign_key => 'recipient_id', 
          :class_name => 'Message' do
      def delivered_and_unflagged(page=1)
          unflagged_items(:all, :conditions => 'delivered_at IS NOT NULL'
        ).paginate(:page => page, :per_page => @messages_per_page)
      end
    end

    has_many :sent_messages, :foreign_key => 'sender_id', 
          :class_name => 'Message' do
      def delivered_and_unflagged(page=1)
          unflagged_items(:all, :conditions => 'delivered_at IS NOT NULL'
        ).paginate(:page => page, :per_page => @messages_per_page) 
      end
    end

    has_many :draft_messages, :foreign_key => 'sender_id', 
          :class_name => 'Message', 
          :conditions => 'delivered_at IS NULL' do
      def paginated(page=1)
        paginate(:page => page, :per_page => @messages_per_page)
      end
    end

    def inbox(page=1)
      self.received_messages.delivered_and_unflagged(page)
    end

    def sent_mail(page=1)
      self.sent_messages.delivered_and_unflagged(page)
    end

    def drafts(page=1)
      self.draft_messages.paginated(page)
    end
  end

Fig.10 - more developed User model

I'm doing pretty well with reduction of code in my controller now. The only ugly bit of code leftover is the params[:page]... bit - but I can make that slightly better too by factoring it out. I would like to use params[:page] || 1 but params[:page] returns an empty string if there is no matching parameter and will_paginate interprets an empty string as a request for page 0 and returns an error. So I have to use the longer statement with the ternary operator. Now my controller code looks like this:

 def index
   @messages = user.inbox(figure_page)
 end
 
 def sent_mail
   @messages = user.sent_messages(figure_page)
 end
 
 def drafts 
   @messages = user.drafts(figure_page)
 end

 def figure_page
   params[:page] == "" ? 1 : params[:page]
 end

Fig.11 - pruned fragment from messages_controller.rb

I'm happy enough with that. I've made different lists of messages for the currently logged on User that automatically paginate and filter out flagged items with just one line of code per method.

3) Next and Previous Message

I'm not done yet though - because the view page of a message needs a next and previous link. So if the user is looking at a draft - next should be the next draft - not the next sent message - and previous should be the previous draft - not the previous sent message. Make sense?

One way I could do this is to have a show_draft method, a show_sent_item method etc... and just call the correct link from the correct listing page (i.e. the list of all drafts page has links to show_draft, the sent items page has links to show_sent_item etc...).

There are 2 problems with this though. 1) That is creating several methods for basically one 'show' action. So they will all be virtually the same code over and over again. 2) I'm using a partial to render the list of messages - so I'd have to send in some way to create a different link based on the type of filter ('drafts', 'sent mail' etc...) but I'd rather just call render :partial => "message", :collection => @messages. I don't want the partial to have to worry about what particular filtered list of messages it happens to be rendering.

I'm sure there are a lot of ways to solve this. What I came up with was to add a 'from' value as a parameter for each link_to :action => 'show' in the partial. That way I could just append params[:action] to every url and by the time the controller gets the request, it knows where the request is coming from. This gives me the information I need to respond differently to the show action depending on that parameter. And leaves that logic out of the view.

In order to get the next and previous messages though, I needed to be able to identify and generate a list of messages based on the value of a string (i.e. value of params[:from]).

The code I wrote at first looked something like this and was in the controller:

  def show
    @message = Message.find(params[:id])
    # need @messages for previous, next
    case params[:from]
    when 'sent_mail'
      @messages = user.sent_messages(figure_page)
    when 'drafts'
      @messages = user.drafts(figure_page)
    #...
  end

  def bulk_action
    # ... do bulk action

    # need @messages for previous, next
    case params[:from]
    when 'sent_mail'
      @messages = user.sent_messages(figure_page)
    when 'drafts'
      @messages = user.drafts(figure_page)
    #...
  end

Fig.12 - fragment of messages_controller.rb with new code

So I've lost some of my simplicity, I'm repeating myself again and my code is in need of pruning.

What I need is a function that returns a function waiting to receive arguments. This is similar to the with_scope method I mentioned earlier, and the idea of function currying. I need a function that's partially filled out - but not called yet - waiting for some parameters. This is a good place to use the the fact that a Method is just another object in Ruby - and create a method to return whichever User method I want.

A method that returns a method

  def get_messages_function(param)
    # special case of 'index' action
    if param == 'index'
       self.method('inbox')
    else
       self.method(param)
    end
  end

Fig.13 - fragment from User model

returns a method as an object waiting for arguments. So I can put that code in my User class and I can call it like this in my controller:

 def index
  @messages = user.get_messages_function(params[:action]).call(figure_page)
 end

 def sent_mail
  @messages = user.get_messages_function(params[:action]).call(figure_page)
 end
 
 def drafts 
  @messages = user.get_messages_function(params[:action]).call(figure_page)
 end

 def show
  @message = Message.find(params[:id])
  @messages = user.get_messages_function(params[:from]).call(figure_page)
  # ...
 end

Fig.14 - fragment from new messages_controller.rb

One last trick

I'm almost done. But I can go one step further in minimization of code. Taking advantages of the fact that a method can be converted to a block by putting an & in front of it. In the controller, since all the returned methods are taking that same figure_page parameter - I can factor that out as a method accepting a block and do something like this:

  def index
    @messages = find_messages(&user.get_messages_function(:inbox))
  end

  def sent_mail
    @messages = find_messages(&user.get_messages_function(:sent_mail))
  end

  def show
    @message = Message.find(params[:id])
    @messages = find_messages(&user.get_messages_function(params[:from])) 
    # ...
  end

  def figure_page
    params[:page] == "" ? 1 : params[:page]
  end

  private
  def find_messages(&func)
    yield(figure_page)
  end

Fig.15 - fragment from another revision to messages_controller.rb

It's odd looking, I admit. I've lost a little readability for the sake of density. But I've left myself very little code in the controller and nothing specific about controllers in the model. That much I like.

Conclusion

So if you ever writing a Ruby on Rails application that has messages that need to be filtered, paginated and include a detail view with a previous, next link - you might be able to glean some code from the article to help get started. Also, today's lesson is that it's sometimes handy to pass around functions as objects.

NOTE: I've included a zip file of various items related to this article. It includes some Ruby code as a demonstration which requires a sqlite3 installation. Also, I used Python to generate this document with all the color-coded sections. I've included that in case it is of interest to anyone. It requires the Mako and Pygments packages.

Let's face it, everything seems to move at a breakneck pace in our industry. It seems like every other day there is a new start-up with some amazing product that's supposed to change the way all of us work. At least once a week I get software update notifications for some application or another. It is our job (and usually our passion) as developers to keep up with the tools that we use, and constantly be evaluating these upgrades, new features or sometimes even the alternatives. As a Ruby on Rails shop, there is nowhere better to check the pulse of the Rails community than RailsConf 2008.

RailsConf 2008 was held in Portland, Oregon again this year and Chris and myself had the chance to experience it first hand. It was the first RailsConf for both of us so our expectations were high, and for the most part we weren't disappointed. We met some really cool people from all over the world (including the cool guys from Blue Box Group), picked up a ton of information on where things with the Rails community are headed, and Chris even held a Birds of a Feather (BOF) discussion about his GiftWrap plugin.

There's simply too much to cover in a single blog post for those who couldn't make it, so please excuse my brevity. You can check out O'Reilly's website to read all about the events and schedules, and you can even grab most of the presentation materials there too.

Maybe my surprise was simply because this was my first RailsConf, but some of the best stuff for me wasn't specifically related to Rails! I went to a cool BOF on Rails alternatives, an interesting discussion about frameworks like Merb, Rack, and Mack, and we even got into the DataMapper ORM and replacing Subversion with Git. I also saw some great presentations on meta-programming, the internal workings of Git, and perhaps the topic I was most curious about – Phusion Passenger. I also enjoyed the well-delivered keynotes from Joel Spolsky and DHH. All these things gave me a taste for what else is out there, and how people are using not only Rails but other tools as well in our industry. The BOF sessions were great as well, bringing small groups of really bright people together to just chat about different topics. Oh, and the food rocked – plus all the Starbucks coffee you can handle, sweet.

That said, there were a few things that weren't so great that I thought I should share as well. The topic of scalability has been a hot one since way back when. Maybe it's just because we've built several high-traffic applications, but the scalability issue has been beaten to death in my opinion, and was beaten quite a bit more this year with about 5 talks covering scalability. Also, the tutorials weren't really worth the extra fees in my opinion – don't get me wrong, they were useful and interesting, but several of the topics ended up being covered in other sessions that were part of the conference, some of them were even the exact same presentation! Finally, while it was great to get a feel for a lot of the other tools out there being used, I must admit I was hoping for a bit more Rails-specific content. It was great to see the keynote by Jeremy Kemper announcing Rails 2.1 (some great stuff in there), but I still don't have a feel for the roadmap of development for Rails and where things are going next.

Overall RailsConf was great, and Portland was a great location for it (despite the 4+ hour flight from Chicago). Would I recommend it? Absolutely. You won't find a collection of brighter developers out there than the ones in the Rails community. Everyone seems to 'get it' and they're all interested in advancing the framework and expanding Rails into the main stream. So thanks to everyone who presented, it was quite an experience – see you next year!

Just give me the globalize_with_google plugin now!

Anybody who's looked into localizing or internationalizing a Rails app has probably come across the "Globalize" plugin. It's a bit of an 800 lb. gorilla in the sense that it supports potentially hundreds of languages, automatic generation of validation messages, and even multiple pluralization cases based on the exact number of objects being counted. (There's a story about a people whose language only had three numbers- 1, 2, and 'many'. Globalize can handle that!) But as long as installation is as easy as "script/plugin install ...", who cares how much the gorilla weighs?

On a related note, Google recently released a series of AJAX APIs that are dead-simple to plug in to any web app, including one that does automatic translation. Can you guess where I'm going with this?

As soon as I saw Google's announcement that they were offering a free translation API, I started thinking about how to write a plugin that used it to initialize a Globalize database.

My solution, as sketched on the back of a napkin, had two pieces: The first would override Globalize's "String.translate" method. The other one would cache the translations so we still had a checklist of phrases for professional translators to go over, if necessary, and so we weren't dependent on the uptime of Google's servers for the functionality of our application. (Not that Google has lousy uptime; but if by chance they ever take down the service or start charging for translations, we can't have our translations just turn off).

The Actual Translation

This part was the easiest. We just modify Globalize's ".t" method to use Google's translation service:

module String
  def self.included(base)
    base.send :alias_method_chain, :translate, :google
    base.send :alias_method, :t, :translate
  end
  def translate_with_google(default = nil, arg = nil)
    local_base_language = defined?(BASE_LANGUAGE) ? BASE_LANGUAGE : 'en'

    #don't translate this if it's already written in the target language
    return self if Locale.language.iso_639_1 == local_base_language

    result = Locale.translate(self, '__translate__', arg)
    return result unless result ==  '__translate__' 

    return %Q{<span id="translation_#{self.object_id}">#{self}</span>
                <script type="text/javascript"> 
                ......}
    end
  end

The only flaw is that you can't use this on the labels of buttons or in javascript alert()s. Instead of showing a translated string, it would display a huge mess of javascript. I don't think there's a simple workaround for this, though, since the ".t" method can't know what context it is being called in. So in your views, make sure all of your translated buttons use something like

<input type="submit" value="<%= "Submit".translate_without_google %>" />

The Caching

This part nearly killed me. How do you cache the result of a google translation? It never goes through our server! The solution was a little convoluted, but very educational to a guy who had never written a plugin before.

First, we need to make the Javascript report the result of each translation back to our server. Fortunately, Google's "translate" function offers a callback once the translation is complete. So I just told it to execute the following:

new Ajax.Request('/cache_google_translation',{method: 'post', parameters: "phrase=#{self}&translation="+result.translation});

Next, we need a way for our Rails app to recognize the request for caching. But how can a plugin respond to a request like a controller does? It takes two steps. First you need to make a pseudo-controller that will do the caching:

class TricksController < ActionController::Base
  def cache_google_translation
    bound_vars = [params[:translation], params[:phrase]]
    ActiveRecord::Base.connection.execute("UPDATE globalize_translations SET built_in = 2, text = ? WHERE tr_key = ? AND language_id = #{Locale.language.id}".gsub('?'){ActiveRecord::Base.connection.quote(bound_vars.shift)})
    Locale.translator.put_in_cache(params[:phrase],Locale.language.iso_639_1,params[:translation])
    render :text => ''
  end
end

And then you need to extend Rails' route parser to attach a URL to your controller. (alias_method_chain to the rescue!)

module MapperExtensions
  def self.included(base)
    base.send :alias_method_chain, :initialize, :google_caching
  end
  def initialize_with_google_caching(set)
    #we have to add ours FIRST, otherwise the final line of the regular routes.rb is usually a catchall that would intercept OUR route
    set.add_route('/cache_google_translation',{:controller => 'google/tricks', :action => 'cache_google_translation'})
    initialize_without_google_caching(set)
  end
end

Finally, in your plugin's init file you just attach these classes into Rails:

ActionController::Routing::RouteSet::Mapper.send :include, Google::MapperExtensions
ActionView::Helpers::AssetTagHelper.send :include, Google::Javascript

And that's it! Well, not quite. Did you notice the reference to Locale.translator.put_in_cache? If you want to make sure that the auto-translations in your database are easily distinguishable so that you can have them manually translated later (machine translation isn't quite there yet!) then you have to add an extra step. It was easy enough to use a manual update statement instead of Locale.set_translation, which allowed me to set "built_in = 2" (that's how you recognize the auto-translations). But then the 800 lb. gorilla gets in the way. Globalize maintains a separate cache of translations in memory to avoid wear and tear on the database, but if you don't update the copy in memory as well, Globalize will never actually USE your cached version! It's a protected variable, so one more module extension:

module LocalizeCacheAccess
  def put_in_cache(key,language,translation)
    @cache["#{key}:#{language}:1"] = translation
  end
end

and then include it in your app with

Globalize::DbViewTranslator.send :include, Google::LocalizeCacheAccess

And that's it! Now you're REALLY done! To get all of this code in a simple Rails plugin, download globalize_with_google.zip and unpack it in #{RAILS_ROOT}/vendor/plugins/.