Of the 300 new features in Leopard (Mac OS 10.5), Time Machine may be the one that excites me the most. I've already used to to repair applications after failed updates, pull back browser history items from two months ago, and restore emails that I had inadvertently flushed.

Yesterday, I found a new use for Time Machine: reviving IRb command history.

Over at eduFire, we've been working on converting user profile pictures from file system store to S3 backed storage. Since we're using the attachment_fu plugin, it seemed that the easiest way would be to just create a new instance of the user profile picture and feed in the path to the file.

While using attachment_fu in a controller is beautifully simple, requiring just an uploaded_data parameter, manipulating files through script/console is a bit more difficult. I knew we had wrestled with this back in February, when we changed the default thumbnail resolutions, and had to reprocess everyone's photos. What I had failed to do back then was document how we actually achieved it.

I have my IRb history logging to ~/.irb_history, but with only 500 lines of scrollback, I knew that there was no chance an entry from three months ago had survived. My first attempt to view the file in Time Machine was unsuccessful, as I couldn't seem to figure out how to get Time Machine to display dot-files.

However, the Time Machine archive is browsable in Terminal; after finding the path to the relevant backup (which looked something like this: /Volumes/Time Machine Drive/Backups.backupdb/AlloyCode/2008-03-03-000650/Macintosh HD/Users/jared), I was able to open the .irb_history file in TextMate and find the command I had lost.

For those who might need to do a similar operation in the future, here's how to recreate a file attachment in script/console:

Avatar.create(:filename => filename, :content_type => content_type, :temp_path => temp_path)

In the above example, filename corresponds to the name to save the attachment as on the server, content_type is the MIME type, and temp_path is the full path to the actual asset.

A lot has been said about running Rails applications on shared hosting, most visibly in these two articles by David Heinemeier Hansson and Dallas Kashuba (on the DreamHost blog). For the past eight months, Alloy Code and Your Garage Online have been running on a single 256mb slice from Slicehost, and while it was possible to get the two sites to happily co-exist, I've always regarded it as something of a delicate house of cards, just waiting for a gust of wind or slammed door in a neighboring apartment to knock the whole stack down. Well, two weeks ago, I made a serious change in back-end configuration, and I couldn't be happier. Indulge me for a moment, because I feel that a little history is appropriate…

Initially, the slice was configured with Apache 2.2 acting as a proxy for two mongrel clusters. The resources of the slice itself allowed me to run two mongrels for the blog, and three for Your Garage Online. Since the majority of the blog's content is cached static pages, two mongrels seemed like plenty, and only suffered a slight delay in attempting to access the admin interface, or other dynamic content.

Then I came across Ezra's article on nginx, and I invested the better part of a weekend switching from a pure Apache/Mongrel setup, to something of a strange hybrid. Since nginx wouldn't honor SVN webdav connections properly, I had to keep an apache instance handy, but restricted to listening on a high-numbered port, with nginx forwarding requests intended for my SVN repositories back into Apache. Meanwhile, nginx had two other listeners set up, one forwarding Alloy Code traffic to the blog's mongrel cluster, and one forwarding Your Garage Online traffic to the other mongrel cluster.

A few months ago, I heard about Thin while listening to the Rails Envy Podcast, and the idea of using Unix sockets instead of TCP to forward the proxy requests really appealed to me. Unfortunately, even though I had upgraded my slice's copy of Ruby to 1.8.6, I was never able to get the Thin gem to install. Later, I discovered it was because my RubyGems installation was linked against the older, 1.8.5 version of the ruby binary. I never did find a good way to switch the gem command's installed ruby version, I had to reinstall RubyGems from scratch using the 1.8.6 binary to call the setup.rb file.

So, when Passenger came out, the configuration-tweaker in me was very excited to give it a try. Once again, I set aside the better part of a weekend to get the installation going. After sorting out the gems issue above, and recompiling Apache to include prefork support, I was ready to roll.

The installation instructions provided were enough to get me 90% of the way there. I had overlooked the fact that, eight months ago, I told Apache to only listen for connections on port 8010 (part of the nginx-svn debacle). And, it turns out that when running two rails applications on a single host, with a single IP address, I needed to provide a little extra context to make certain the static content was handled properly.

Since the hostname of the box itself is alloycode.com, the setup for the blog is as sparse as the Passenger sample:

<VirtualHost *:80>
    ServerName alloycode.com
    DocumentRoot /path/to/blog/public
  </VirtualHost>

Setting up Your Garage Online was a little tricker. My first attempt was just to mirror the same code as above, but with the proper ServerName and DocumentRoot settings. Unfortunately, that meant that while the Rails stack did load, and process the requests properly, none of the stylesheets, images, javascrtips, or other static assets could be loaded. After carefully investigating each of the options in the Passenger documentation, I managed to put together this VirtualHost definition that seemed to do the trick:

<VirtualHost *:80>
     ServerName yourgarageonline.com
     ServerAlias www.yourgarageonline.com        
     DocumentRoot /path/to/ygo/public
     <Directory "/path/to/ygo/public">
         Options FollowSymLinks
         AllowOverride None
         Order allow,deny
         Allow from all
     </Directory>
     RailsBaseURI /
  </VirtualHost>

And like that, it was as if a switch was flipped, and everything was working 100%. It's now been two weeks since I put Passenger on that slice, and I haven't had any outages or runaway memory problems, and I no longer have to worry about my web server and my mongrel cluster getting out of sync, resulting in dreaded 503 Service Unavailable errors.

Kudos to the Phusion folks, this is one incredible release.

Heroku, a Y Combinator funded startup, has been making headlines lately! Mentions on the official Ruby on Rails blog, the Ruby on Rails Podcast, and even an article on TechCrunch outlining some of the upcoming features. Personally, I've been using it since late last year, and I've found it to be immensely useful in a couple of situations.

Scenario 1: While interviewing for my current job, I wanted to show my interviewers the application Keith and I had developed for the Rails Rumble. Thanks to some trademark issues and a certain corporation whose devices have become synonymous for copiers, the document collaboration application didn't have an online presence. Rather than attempt to put it back online with my current shared hosting (already stretched to the limit by this blog and the YGO application) or secure new shared hosting, instead I prepared to zip up the entire application and upload it to my Heroku account.

• The first thing I had to do was unpack the gems used by the application, such as attachment_fu into my /vendor directory, and make sure I added them to my config/environment.rb load path.
• Secondly, I removed all my existing log files... no need to push an additional 300mb of data online!

After zipping the app and uploading it, I just needed to click the 'migrate' button inside the Heroku editor window, and I was in business. By virtue of putting the app online in a private state, I could ensure that only my interviewers were able to see it, and I was able to take it back offline once the interview process had concluded.

Scenario 2: In doing some freelance work on a contract tracking app, I wanted to show my client a live preview of the progress I was making, and to get some early feedback on some of the workflow. Again, rather than expose the whole application live through my existing hosting, I again created a zipped version of the application and added it to Heroku.

This time, I was able to show him the current 'snapshot' of the application, and instantly incorporate minor tweaks without even needing to reload a mongrel instance or re-upload any files. Small details, like the text on a form label or the size of an element could easily be adjusted on the fly, in accordance with the client's wishes; then, those changes were easily mirrored back down to my local copy to continue development.

All in all, I'm very impressed with Heroku. While I don't make extensive use of the in-browser editor, the few times I've had to make tweaks to a running instance, it's been very easy to do so right in the browser. Plus, having a browser based interface to my favorite ruby tool, script/console, has been incredibly useful.

Lastly, I think Heroku illustrates an excellent usage of Amazon Web Services as part of a business model, particularly in being able to make using EC2 easily available to the end user, along with an easy way to pass along the associated costs to that user. I'll be very excited to see where the future takes them.

In the first entry, one of the issues I addressed was stubbing a response to an HTTP-Basic Authentication scheme. To get my spec to pass, I stubbed the entire authenticate method which I had written. While this approach was successful, it left me with a hole in my code coverage:

Now that I knew my controllers' actions were being tested, I needed a good way to test the authenticate method itself, to ensure that failed requests were not being let through. A quick glance in the Rails source gave me some clues to implementation.

Since the credentials are sent (encoded) in the HTTP header, the first thing I needed to do was inject those credentials into the header before the request was processed. Rails core (using Test::Unit) uses the following convenience method in /actionpack/test/controller/http_authentication_test.rb

def set_headers(value = @credentials, name = 'HTTP_AUTHORIZATION')
      @controller.request.env[name] = value
    end

The encoded credentials are passed in and assigned to the request environment. Accomplishing this in RSpec wasn't much different. First, we construct a set of encoded credentials, then we assign them to the environment variable 'HTTP_AUTHORIZATION'. Note that in the RSpec code below, we don't call @controller.request.env, but simply request.env

before(:each) do 
    @credentials = ActionController::HttpAuthentication::Basic.encode_credentials("david", "clearly_false")
    request.env['HTTP_AUTHORIZATION'] = @credentials
  end

This allowed me to create a series of tests to check that a user providing invalid credentials was not being allowed into the protected areas of the site.

it "should fail with invalid credentials" do
      get :index
      response.should_not be_success
      response.headers["Status"].should =~ /401/
    end

Two fairly simple expectations, when attempting to access a resource using invalid credentials, the response should not be a success, and the status code should match 401 (Unauthorized).

RSpec is definitely growing on me, and I've noticed some BDD disciplines starting to creep into my approach to testing with Test::Unit in my day job.

The end of 2007 has brought about some exciting changes for me, most notably a new job with Education Revolution. To try and ease the transition during a stressful holiday season, I gave myself a week off between jobs, which left me with a bit of unexpected time on my hands between Christmas and New Year's. I decided to make the most of my time off and try to learn RSpec.

RSpec is quickly becoming a darling among some of the visionaries in the Rails world. With the release of RSpec 1.1, which brings easy integration between Test::Unit style testing and Behavior Driven Development using RSpec, it seemed prudent to try it on myself and see how it fits.

On the whole I really like the idea of specifying behaviors instead of assertions, especially when coupled with doing some design-driven development. As a developer, having both a clear set of expected behaviors and a set of slides which show the application in a near-finished state, it removes a lot of the guesswork which had been plaguing me on previous projects. Also, hearing Adam Williams and John Long talk at November's Raleigh.rb meetup about doing top-down testing (starting with integration tests, then drilling down to functional and unit tests merely to handle edge cases) has turned my opinion of Integration tests on it's head.

Having come from a metaprogramming-driven Test::Unit background using Mike Clark's TestRig framework, RSpec felt like a lot more testing code. And indeed, my rake stats seems to support that, showing a 1:2.8 ratio for my training project. Finding help with RSpec has been tricky, though. There are some really great contrived examples on the RSpec homepage, but I had trouble finding more information on how to deal with presenters, nested resources, and HTTP Basic authentication. I want to show off two solutions that I cobbled together out of solutions found online.

The first problem I encountered was in dealing with the save! method in ActiveRecord. My controllers typically use the save! coupled with a rescue statement for ActiveRecord::RecordInvalid, like so:

class WidgetsController < ApplicationController

        def create
            @widget = Widget.new(params[:widget])
            @widget.save!
            respond_to do |format|
                format.html { redirect_to @widget }
            end
        rescue ActiveRecord::RecordInvalid => e
            respond_to do |format|
                format.html { render :action => 'edit' }
            end
        end

    end

Most of the RSpec examples I had come across show something like this (excerpted from Testing Controllers with RSpec):

def do_create
    post :create, :menu_item=>{:name=>"value"}
  end

  it "should save the menu item" do
    @menu_item.should_receive(:save).and_return(false)
    do_create
  end

The problem being, using save! doesn't return false on failure, it raises an exception. Fortunately, the answer was available on the RSpec-users mailing list. Now my widet_controller_spec.rb contains the following:

it "should fail to save the widget" do
      @widget.should_receive(:save!).and_raise(ActiveRecord::RecordInvalid.new(@widget))
    end

The second major stoppage I encountered was dealing with HTTP Basic authentication. The application I was building didn't require a huge complicated account/password structure, it just needed a few protected pages available to a single administrator.

class ApplicationController < ActionController::Base

        before_filter :authenticate

        def authenticate
          authenticate_or_request_with_http_basic do |username, password|
            username == 'jared' && password == 'secret'
          end
        end
    end

Suddenly, all my controller specs for actions lying behind that authenticate filter were failing. The fix lies in stubbing out the method using the controller local variable in the spec.

describe WidgetsController do

      describe "with successful admin login" do
        before(:each) do
          controller.stub!(:authenticate).and_return(true)
        end
            ...
        end
    end

Of course, I still need to come back and write an integration test which will address both the success and failure states of the authenticate method.

All in all, I'm very impressed with RSpec, and I can see why it's picking up such a following. I'm definitely going to play around with it further, but I'm not quite ready to say that I'm going to switch all my projects over; one important factor to consider is future code maintainability. The pool of talented Rails developers is small enough to begin with, adding the further requirement of finding a Rails developer who is also versed in RSpec limits that result set even further.

I was very excited to use the new Advanced YAML Fixtures introduced in Rails 2.0, so I set about updating my code in an Edge Rails application I've been working on for the past year or so. There are a few caveats in using the new fixture styles, though.

The Fixtures are introspective, so if you've overridden the default name of an association in your model, you'll need to use that new name in your fixture. With the following code,

class Worker < ActiveRecord::Base
  belongs_to :status, :class_name => 'WorkerStatus', :foreign_key => 'worker_status_id'
  ...
end

The key for worker_status: should actually just be status: in the workers.yml fixture file.

The larger problem I encountered was in using a mix of IDs and fixture record names. I use a testing framework, TestRig, which was written by Mike Clark and Dave Thomas for the Pragmatic Studio. The TestRig framework takes in ID parameters of fixtures in the database, and loads them accordingly. This falls apart with the new status: full_time style of declaring relationships in fixtures, because the former relies on a 'known' ID, and the new style generates a fairly random ID instead. This leads to a bunch of broken relationships throughout the test suite.

According to the API documentation, the generated ID is constant, so I could discover that ID and use it in the TestRig calls which require an integer key, but then I've essentially lost all benefit of using the Advanced YAML Fixtures, and added some crazy complexity to my tests as well.

Every Christmas, my father sends gift assortments from Swiss Colony to family members, friends, and former business associates. Last month, he shared with me an interesting story about how Swiss Colony handles repeat customers; which he gave me permission to retell here, because I think it’s a great example of a non-technical company using modern technology effectively.

He started out telling me about a woman who used to work for him 18 years ago, someone whom he has kept in touch with over the years, exchanging a few letters annually. He knew that she had moved recently, but had not yet received her new mailing address. However, when he received his most recent Swiss Colony catalog, he noticed that her address had been updated. He was suitably impressed.

One of my former jobs required me to send periodical mailings to a nationwide group of photojournalists, an activity that probably gave me more insight into business-based bulk mailing than any web application developer would ever care to have. But, it did leave me with an answer as to how her information was likely updated. It would seem that the company runs it’s entire recipient database through an automated service, such as Verimove, in order to keep their mailing address data fresh.

In addition to providing repeat customers with the list of all the recipients who had received gifts the previous year, they also include the previous year’s order for each recipient. It’s a refreshing change of pace from those companies that vacuum up a great deal of information about their customers but then don’t turn it around and use it effectively. Swiss Colony offers, even for it’s mail-order customers, the benefit of relational data.

Taking this a step further for the world of web development, this sort of information can be ideal to use as part of a ‘hook’ strategy to drive return traffic to an application. Instead of just emailing existing users to tell them about new products or services you provide, offer them value by doing a little bit of analysis on the relationships they already have in your systems, and making their experience more convenient.

Recently, I've been delving into some filter-related problems in one of my Rails applications. This has required me to trap and review specific segments of my Rails log files. As part of my initial stack setup when I begin a new project, I install the Query Analyzer and Query Trace plugins.

The upside is I get very detailed trace information for my application, pictured below:

The downside is, when these files are viewed with Textmate or Console, they wind up looking more like this:

    Rendered events/_event (0.00036)  
      [4;35;1mSlot Load (0.000361)   SELECT * FROM 'slots' WHERE (slots.event_id = 23)   
        [35;2mvendor/plugins/query_analyzer/lib/query_analyzer.rb:38:in 'select'  
        [35;2mlib/association_extensions/chronological.rb:9:in 'first'  
        [35;2mapp/models/event.rb:109:in 'begin_at'  
        [35;2mapp/views/events/_event.html.erb:6:in '_run_erb_47app47views47events47_event46html46erb'  
        [35;2mapp/views/events/index.html.erb:39:in '_run_erb_47app47views47events47index46html46erb'

Today's "stretch the brain" exercise centered around creating a Textmate command to clear out this unnecessary cruft. For starters, I cloned the Text bundle's "Remove Unprintable Characters in Document / Selection" command. It looks like the Textmate folks are using Perl to remove unprintable characters:

perl -pe 's/[^\t\n\x20-\xFF]|\x7F|\xC2[\x80-\x9F]//g'

I left all the settings unchanged:
Save: Nothing
Input: Selected Text or Document
Output: Replace Selected Text

And edited the Perl regular expression to match each of the three possible variants of ASCII instruction:

perl -pe 's/\[\d;?(\d+)?;?(\d)?m//g'

Success! Now, with one keystroke, I can remove all that extra information from my log file, and get down to business.

Having been excited about the upcoming Rails 2.0 release since I saw DHH speak in Portland at Railsconf, I've been starting all my new projects on Edge Rails in anticipation of the new version.

While bringing one of my projects up-to-date with the Preview Release, I came across something relatively new that (to my mind) breaks the long-standing logic of “When Things Get Saved.”

I've been trying to keep with the Skinny Controller, Fat Model approach outlined by Jamis Buck, so I've moved the logic for creating my associated models out of the controller, and into the model.

My specific example deals with a semi-complex form, registering new users. I needed to create a Company, a User, an Employee and some defaults, by simply prompting the user for a minimal amount of data (Company Name, First and Last name, email, password). My Company model has the following associations:

belongs_to :owner, :class_name => "User", :foreign_key => "owner_id"
has_many :employees, :class_name => "Employee", :foreign_key => "employer_id"
has_one :company_setting

The following snippet is taken from the Company model, this method instantiates a new company object, sets the owner, creates the first employee, assigns roles to the employee, and sets the default variables for the company's details.

def self.create_with_associations(options)
  returning new do |company|
    company.attributes = options[:company]
    company.create_owner(options[:user])
    employee = company.employees.create(:title => "Owner", :phone => company.phone, :user => company.owner)
    Role.find(:all).each { |role| employee.roles << role }
    company.create_company_setting(:hours_per_day => 8, :hours_per_week => 40)
  end
end

Unfortunately, Changeset 7511 has rendered that snippet useless. The new rule states that the .create method cannot be called on an association for an unsaved parent. Previously, the code above would instantiate a new Company object, then silently save it when the owner was assigned. Now, that line throws an exception instead.

Here's the Rails 2.0-compatible version. I'm not sure I like it as much.

def create_with_associations(options)      
  company = self.create(options[:company])
  owner = company.build_owner(options[:user])
  employee = company.employees.build(:title => "Owner", :phone => company.phone, :user => company.owner)
  Role.find(:all).each { |role| employee.roles << role }
  setting = company.build_company_setting(:hours_per_day => 8, :hours_per_week => 40)

  company
end

I think I need to approach the whole tangle from another angle. Perhaps I need to use the User as the base unit, instead of the Company, and build my associations from there?

OK, title aside, no train puns from me today… this is exciting news!

Rails 2.0 has reached Preview Release status. Despite the Microsoft-esque syntax, the mood is overwhelmingly upbeat. Personally, I’ve been using Edge Rails for all my new development over the past 6 months, but this final drive by the Rails Core team and core contributors has included some great new goodies.

The changes to the Routing system have received plenty of attention elsewhere, but I wanted to pull out one tidbit which I’ve only seen posted one other place, Casper Fabricius’ excellent recap of DHH’s keynote address at Railsconf Europe:

Namespaced Routes are awesome, and a long-overdue addition for any of us who’ve attempted to create an administration section for our sites. There’s a clever bit of form_for syntax for dealing with namespaced routes.

The Old Way

form_for :user, :url => admin_users_path do |f|
  # new user form 
end

form_for :user, :url => admin_user_path(@user), 
                :html => { :method => :put } do |f|
  # edit user form
end

The New Way

form_for([:admin, @user]) do |f|
  # Works for either!
end

Cleaner, more concise, and thoroughly overlooked. Casper, thanks for bringing this out so we could see it!

After a turbo-charged 48 hour development period, our contest entry has launched! Keith did an outstanding job on our interface, far exceeding merely doing justice to my application code. Even more impressive, we managed to implement everything we intended, and simplified the app at the same time.

Here’s the short list of features:

• *No* Account Management Required!
• Easy to share private URLs
• Document Version & Status Tracking
• Version specific Comments

I would argue that our greatest success was removing the account management requirement. Most of the Rumble contest sites we’ve viewed so far put the majority of their content behind a login/password screen. A few of the more generous ones support OpenID, a technology I’ve recently come to embrace.

We take pride in being one of the few applications that offers all the benefit of a session-backed user account without any of the account creation/management overhead. Simply provide your name and email address when you upload a document and your account is created for you. If you’ve uploaded additional documents with the same email address, we’ll automatically group them together for you.

Well, I’m long overdue for a Rumble update, the past 24 hours have been a whirlwind of activity over at team Redkey. Our application now supports team collaboration, ActionMailer backed email, and some really cool routing stuff that enables us to use the concept of users without the overhead of making those users log in.

Also, don’t forget to check out our Flickr photos. We were fortunate enough to have a guest photographer stop by the morning to take our official team photo, capturing us at our most-rested state in the past day and a half.

Inspired by City Cliq's RSpec screenshot, we wanted to brag on our 100% RCov coverage as well. We've been using Test::Unit as opposed to RSpec, along with a healthy dose of Mocha to mock out external services.

For example, our application provides a convenient, tiny url to access user content, which is generated as part of a create action, generated with the ShortURL gem. In our functional test, we mock out the "shorten" method of ShortURL, and have it feed us back consistent data:

def test_create
  WWW::ShortURL.expects(:shorten).returns("http://tinyurl.com/fake")
  assert_difference "Document.count", 1 do
    post :create, :document => @@document_default_values
    assert_equal "http://tinyurl.com/fake", assigns(:document).shorturl
  end
end

Since the generated URL could potentially be different each time the test is run, I've removed the dependency on an external library and service to gain repeatability in my tests. I'll take for granted that the ShortURL gem works and has it's own tests; there's no reason for me to duplicate their efforts. The above test simply assumes the ShortURL library is performing as expected, and allowing me to test my own code atomically.

Well, we’re just past the 14 hour mark since the Rails Rumble opened, and we’re making great progress. Keith Medlin and I have resurrected the “Red Key” brand to bring our revolutionary vision to life: Docustackr, a web-based document revision system.

The bulk of my time so far has been spent setting up our deployment environment, having done this recently for Alloy Code and Your Garage Online, I was able to speed through most of it. We’re using Mongrel, Capistrano and Nginx on Ubuntu 7.04 (breaking from the Debian herd) and have our stack already in place. We won’t be struggling to deploy at the last minute!

Keith has been focusing on our design and interface while I’ve been putting the models in place to support the site. We’ve snapped a few photos for posterity.

Keep watching for previews of our design and code process!

The RubyConf preliminary agenda has been posted, and it looks like a really outstanding lineup!

Even better (for me, at least), is the fact that I’ve already been privileged enough to see some of these talks at the Ruby Hoedown, so I don’t feel quite as bad about having to pick and choose which afternoon sessions I’d like to attend. For those who weren’t lucky enough to make it to the Hoedown, I’d especially recommend Jay Phillips’ talk on Adhearsion. Not only were we all on the edge of our collective seat listening to him, he spent the remainder of the Hoedown with a cloud of people constantly hovering around him, peppering him with questions and was gracious enough to make sure he answered everything we could throw at him.

I’d say the feature I’m most excited about, though, is “Room 3.” I think that offering a more freeform ‘workshop’ environment is a great idea, and I’m going to have to watch my time carefully, because I can definitely see myself accidentally missing sessions I’d like to attend due to spending too long in Room 3.

Online Registration is now open, $250 per person (since the site wasn’t exactly clear on the price unless you click around a bit).