In this lab, we are going to look at some examples of unit and functional testing in Rails. Many of the tests that Rails generates for us automatically need some fixing up to track the changes we made in the code (such as adding authentication), and in this lab we’ll walk through those changes. Then we’ll take some first steps toward more complete tests.
As we’ve discussed, there are multiple approaches to writing tests. In this lab, we are writing tests after we’ve created logic in our program. We encourage you to take a look at test-driven development as well.
Creating automated tests for a Rails application is easier than with most other technologies, but it is still a relatively complex topic. If you’re early in your Rails learning curve, you many want to skip this material and return to it later.
1. Running tests
Many generator scripts, such the scaffold generator, create placeholder test files at the same time they generate the associated application code. All of these test source files get placed in the test directory hierarchy of the application.
Let’s start by seeing what happens when we run our tests:
rake test
You can run more focused sets of the tests by selecting rake test:units or rake test:functionals for instance.
You can also trigger the tests within NetBeans too: just right-click on the project, select the Run Rake Task menu, and select one of the test options.
Running our tests, we get a lot of feedback. Rake runs the tests in sequence, first the unit tests, then the functionals, and so on.
It looks like we have some work to do.
2. Reading the test results
Let’s take apart the results a little at a time. For the unit tests, we scan down and see the lines:
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:test "/Library/Ruby/Gems/1.8/gems/rake-0.8.1/lib/rake/rake_test_loader.rb""test/unit/asset_test.rb""test/unit/category_test.rb""test/unit/contact_mailer_test.rb""test/unit/contact_test.rb""test/unit/content_block_test.rb""test/unit/customer_test.rb""test/unit/link_test.rb""test/unit/subscription_test.rb""test/unit/user_test.rb" Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.8.1/lib/rake/rake_test_loader Started ..E.................. Finished in 0.19429 seconds.
This block of information tells me what tests file were loaded and run, and using the simple command line result format, we see a series of periods and an “E”.
Recall that tests can have one of three states:
- Success, indicated with a period (
.), sometimes also called “green”; - Failure, indicated with an
F, (called “red”), which means one of the assertions we wrote to test an assumption about our code did not work; - Error, indicated by an
E, (also called “red”), which means a program logic problem was detected in our test or application code.
So, apparently, we have a number successful tests from seeing all of those periods, but one errored out.
We look further down in the unit test results and see:
1) Error: test_contact_form(ContactMailerTest): ArgumentError: wrong number of arguments (1 for 4) /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `contact_form' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `__send__' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `create!' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:403:in `initialize' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:351:in `new' /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:351:in `method_missing' ./test/unit/contact_mailer_test.rb:10:in `test_contact_form' /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/active_support/testing/default.rb:7:in `run'
This gives us a clue as to what went wrong and where. The ContactMailerTest class’ test_contact_form method had a programming error in it: an incorrect number of parameters were passed in. We’ll look at that in a minute.
Finally, just below the list of diagnostic messages for failing or error state tests, we get the summary:
21 tests, 33 assertions, 0 failures, 1 errors
Wow, we had 21 tests with 33 assertion calls, and we didn’t write any code. Before we look at the problem in ContactMailerTest, what were all of the successful tests doing?
Recall that unit tests check out our models. If we look in all but ContactMailerTest and UserTest, we see classes that contain this:
# Replace this with your real tests.
def test_truth
assert true
end
Alas, this is a do-nothing placeholder method, so we will ignore it for now. What we should do is write some tests to exercise each model. We’ll try that later.
Peeking inside UserModel, we see a different story. This class is full of tests. It turns out this test file was generated for us when we used the restful_authentication plugin and associated generator to create our user and login system. UserModel tests most of the functionality of our User model and is a good example of how a model should be tested.
3. Fixing the Unit Tests
For now, though, let’s fix our one error in ContactModelTest. This test is exercising the pseudo model that was created when we generated our contact mailer. The test inside of the test class looks like this:
def test_contact_form
@expected.subject = 'ContactMailer#contact_form'
@expected.body = read_fixture('contact_form')
@expected.date = Time.now
assert_equal @expected.encoded, ContactMailer.create_contact_form(@expected.date).encoded
end
Mailer related unit tests are a little strange. What this code is trying to do is test to see if the mail message that gets generated by the ContactMailer.contact_form method is identical to a canned email fixture. @expected is an instance variable created for us by the testing class that embodies a TMailer object and which we’ll load with data and subsequently ask to generate a properly formed email body and headers (that is what encoded does).
Here we see that the other special calling convention on a mailer is being used: create_contact_form. Like deliver_contact_form, this method uses the contact_form method we wrote, but rather than building a mail message and then actually emailing it out, this form just returns a string that contains the body of the email message.
To fix this test, we need to make the generated email body equal to the canned version of the email message we prepare in the @expected instance variable. Let’s do that.
a. Call contact_form correctly
The error we received in the first place indicated that we are calling the underlying contact_form incorrectly (not enough parameters). We need to supply name, email, subject, message, and a sent_at time into that method:
assert_equal @expected.encoded, ContactMailer.create_contact_form('bob', 'bob@bob.com', 'test', 'testing', @expected.date).encoded
If we run the test again (rake test:units), we see the Error is gone, but now we have a Failure: the messages are not equal. Let’s fix that.
b. Set up fixture data
First, lets create a fixture data file with the general body of the email message. This needs to match the view we’ve created to contain the message body (in contact_form.erb from the previous lab):
Email from your web site From: <%= @name %> Message: <%= @message %>
Our fixture (found in the directory test/fixtures/contact_mailer/contact_form, another peculiar convention for mailers) looks like this:
Email from your web site From: bob Message: testing
Note that we use hard-coded values in the fixture to make it an exemplar of a known working message body.
We are closer, but this still fails. We need to set up the rest of the data needed for the encoded message we are using as our “correct” email state.
c. Fill in rest of email data
We encode the rest of the values in test_contact_form's @expected variable with the sample data we choose to pass in to our own contact_form call:
def test_contact_form
@expected.subject = 'test'
@expected.body = read_fixture('contact_form')
@expected.date = Time.now
@expected.to = CONTACT_RECIPIENT
@expected.from = 'bob@bob.com'
assert_equal @expected.encoded, ContactMailer.create_contact_form('bob', 'bob@bob.com', 'test', 'testing', @expected.date).encoded
end
We save and run this now, and Success! Our admittedly mostly empty unit tests are all running in the “green” state. Let’s go on to fix up some of our functional tests.
4. Fixing the Functional Tests
We now know how to read the test results, so switching our attention to the functional test block results, we see we have our work cut out for ourselves:
FFFFFFFFFFFFFFEFFF.FFFFFFFFFF......FFFFFFF..........F......FEEEE
Which in my first run was equivalent to 64 tests, 76 assertions, 36 failures, 5 errors. Ouch.
The Rails scaffold generator does a much better job at creating functional tests than it does with unit tests. Functional tests exercise our controllers. Peeking inside most of the test files in the test/functionals directory, you will note that they follow similar patterns.
So what went wrong? In scanning through the test diagnostics, we see a lot of tests with similar issues. There are many tests with problems with redirects: “Expected response to be a <:success>, but was <302>”. There are many tests where things were not created, updated, or destroyed when expected. Thinking back, the common thread here is that all of these things need you to be logged in, yet, when the tests are automatically generated for us, the generator has no idea about this requirement. Let’s fix that. h3. a. Adding authentication to existing tests
The restful_authentication plugin that we used provides a utility module called AuthenticatedTestHelper. This module contains useful methods that you can mix-in to your test classes to simulate things like logging in, precisely what we need. Since we protected most of our administrative functions with those before_filter :login_required checks, that will cause any of our tests that try to directly exercise an action to instead get redirected to a login screen.
Let’s pick one of our controller’s tests to fix up as our example. The others will follow similar patterns and you can look at the final lab code to see the changes (or try to make the changes yourself as an exercise).
First, add the authentication system’s helper at the top of the test class. While there, add the user fixtures so we have some test user accounts to log in with:
class ContentBlocksControllerTest < ActionController::TestCase
include AuthenticatedTestHelper
fixtures :users
Now, let’s use the new method we have to log in as the user quentin:
def test_should_get_index
login_as :quentin
get :index
assert_response :success
assert_not_nil assigns(:content_blocks)
end
Run the tests again. The redirect failure on test_should_get_index should now be gone.
Go back to each method in this test class and add the login_as :quentin line at the start of each test method. All of your redirect failures should now be cleaned up.
But, there are still failures to fix. Look through the other functional tests and make the same changes to all of the others that are protected with the before_filter (assets_controller_test, categories_controller_test, contacts_controller_test, links_controller_test).
For the controllers that are selective about the login requirement, be sure to not login when it isn’t needed. For instance, contacts_controller_test’s new and create tests are not protected by login. See the controller to confirm this.
Run the tests again. We are getting closer. In my run, the results look like this:
Started FE...EE.......E.............F.......................F......FEEEE Finished in 6.451664 seconds.
b. Testing asset uploader
Our asset_controller_test is responsible for a number of those errors and failures, so let’s clean it up next. Recall we are using the attachment_fu plugin to implement our uploader, and as luck would have it, the associated README has some guidance about implementation, but not about testing. We’ll have to dig in to the code to figure that out. We’ve done that for you, so let’s walk through what we need to do.
First, make sure you have a test image file as an asset to play with. This takes the role of a fixture, so make a directory in the test/fixtures directory called “files” and place a test file in there. We used an image called “railsquickstart-train.gif”; you can use your own if you wish.
Next, we add information we may need later in our regular fixture file, assets.yml, which spells out some sample content. This will fix any issues with preloaded data for our Asset model, since the validates_as_attachment call makes sure the minimum needed data for a legal attachment is present:
one: name: first content_type: 'image/gif' filename: '/files/railsquickstart-train.gif' two: name: second content_type: 'image/gif' filename: '/files/railsquickstart-train.gif'
Just a few more things. Recall that asset uploading is done through a multipart form upload. The default RESTful controller tests that are created for us by the generate script do not know about the multipart form technique, so we need to help things along.
In our test_should_create_asset method, we need to use some testing helpers to simulate the file upload and we need to tell the post method that we want to do a multipart form. Cruising the documentation on BuildingWebApps.com, we find the fixture_file_upload method, and drop that in:
def test_should_create_asset
login_as :quentin
image_file = fixture_file_upload('/files/railsquickstart-train.gif', 'image/gif')
assert_difference('Asset.count') do
post :create, :asset => { :name => 'railslogo', :uploaded_data => image_file },
:html => {:multipart => true }
end
assert_redirected_to asset_path(assigns(:asset))
end
We also know that our update method is going to have the same issue, so let’s fix that at the same time:
def test_should_update_asset
login_as :quentin
image_file = fixture_file_upload('/files/railsquickstart-train.gif', 'image/gif')
put :update, {:id => assets(:one).id, :asset => { :uploaded_data => image_file }}
assert_redirected_to asset_path(assigns(:asset))
end
Run the tests. Indeed, we’ve cleaned up the asset_controller tests.
c. User controller cleanup
We see there are still some issues around the user_controller_test. This was a test file that was generated for us when we used the restful_authentication plugin, so what’s up?
We added a before_filter in the users_controller too, to protect the creation of accounts. Let’s fix that up quickly just like we did earlier. Add the AuthenticatedSystemHelper and use the login_as methods in the tests.
d. Email bits and pieces
Down to the wire, and it looks like we have just a few tests to fix up.
The first appears to be an error in test_should_create_contact(ContactsControllerTest). It can’t seem to locate a constant ContactMailer::CONTACT_RECIPIENT. This is an error, so that means some kind of programming mistake is the cause. If you search the code with your editor, you’ll see that CONTACT_RECIPIENT is indeed defined, but it is in config/environments/development.rb.
Recall that testing runs in its own, completely separate, environment. Here is a case where a constant defined in the environment is just plain missing in the test environment. This is fixed by adding (or copying from development.rb) the CONTACT_RECIPIENT definition to the config/environments/test.rb file at the bottom:
CONTACT_RECIPIENT = 'tester@buildingwebapps.com'
Unless you have a working email connection, you’ll also want to add a line to ignore errors when trying to connect to the mail system (below the CONTACT_RECIPIENT line is ok):
config.action_mailer.raise_delivery_errors = false
Now, this is somewhat of a hack. We put mail configuration into the initializer file config/initializers/mail.rb. Initializers run after the environment is loaded, and since we are globally setting up the sendmail environment there, we actually have our delivery_method setting stepped on and changed from :test to :sendmail.
In retrospect, we probably should not have put the setup code in the mail.rb initializer file, and instead put the mail setup code in the environment-specific files. That way, we can get the test code to act properly without ignoring possible exceptions.
e. Customers use validation
In this lab, we don’t have a lot of detailed testing for our merchant-oriented code. Even so, we still have a couple of failures to clean up.
test_should_create_customer fails with a pattern that will become familiar to you as you test code that uses validations. Here, the code is trying to make a new customer object. The test, however, by default tries to create an empty customer. Looking inside of the Customer model, we see a validates_presence_of on :first_name, :last_name, :address, :city, :state, and :zip. We better fill in some values for those:
def test_should_create_customer
assert_difference('Customer.count') do
post :create, :customer => { :first_name => customers(:one).first_name,
:last_name => customers(:one).last_name,
:address => customers(:one).address,
:city => customers(:one).city,
:state => customers(:one).state,
:zip => customers(:one).zip }
end
assert_redirected_to customer_path(assigns(:customer))
end
But, what is this customers(:one) syntax? Those aren’t straight string constants. No, instead, we are using references to data we are storing in our fixtures file. Here is what is going on. First, we need the test controller class to load the fixture data, so we add the fixtures :customers line right after the class declaration:
...
class CustomersControllerTest < ActionController::TestCase
fixtures :customers
def test_should_get_index
...
Now, in our customers.yml fixtures file, we make sure there is some data:
one: first_name: 'First' last_name: 'Last' address: 'Street' city: 'City' state: 'State' zip: 11111 phone: '123-456-5678'
As a reminder, YAML files are picky about spacing. The label one: starts in the first column and its attributes (e.g. first_name:) are all tab indented.
When loaded, fixtures can be referenced by name, using the name of the file (“customers”), then in parentheses, the name of the record we want as a symbol (e.g., one: is the name in the fixtures file, but in code, we use the symbol :one; note the way that colon moved!). We are now referring to a loaded instance of a model object, so any data field defined in that model is available to us, referred to by a period followed by the attribute name (e.g., customers(:one).first_name). Now our test data and our test code are each in separate files, which makes it easier to maintain them in the future.
f. Subscriptions are compound objects
Subscriptions have a similar problem but are a bit more complex. The ActiveMerchant code and the logic to stitch together subscriptions, customers, and credit cards involves a dance of three models and some business logic. Our remaining error is in the create method, which is the most complex in the controller. Let’s supply all of the data it needs:
def test_should_create_subscription
assert_difference('Subscription.count') do
post :create, :subscription => { :amount => subscriptions(:one).amount,
:period => subscriptions(:one).period },
:customer => { :first_name => customers(:one).first_name,
:last_name => customers(:one).last_name,
:address => customers(:one).address,
:city => customers(:one).city,
:state => customers(:one).state,
:zip => customers(:one).zip },
:creditcard => { :number => '4111111111111111',
:type => 'visa',
:month => '10', :year => '2010',
:verification_value => '999'}
end
assert_redirected_to root_path
end
Wow, that’s a lot!. Here is what we are doing (using a few different techniques for illustration). First, we are supplying a subscription object. The Subscription model requires a value “amount”, so we pull one from our fixtures (defined at the top of our file as usual with fixtures :customers, :subscriptions since we are also using the customers data).
Next, we need to provide a Customer model object, so we reuse the same fixture data we used earlier for the Customer tests.
Lastly, we need to supply a CreditCard object, as used by ActiveMerchant. We should put that data into its own fixture, but here we spell it out explicitly to show that you can do that, if you need or want to.
The create method is now happy, receiving the three model objects it will need. On success, however, it doesn’t send the user back to a show method, like many RESTful controllers, but rather redirects the user to the home page. We adjust our assertion to point to the root_path, which is defined in our routes.rb file as a named route for “home page”.
Run the tests. Success! No problems.
g. Aside: Write a new functional test
We lost a valuable piece of information when we fixed all of those authentication tests. We really should also test that the right thing happens when someone is not logged in. Let’s write a quick example of a test that succeeds when an unauthenticated person tries to get a protected resource. In ContentBlocksControllerTest, add this test:
def test_should_not_get_index_not_logged_in
get :index
assert_redirected_to new_session_path
end
Here, we are not logged in, and we are trying to get the list of content blocks. We should not be able to do this, and we expect to be redirected to the log-in screen. The assert_redirected_to new_session_path uses the RESTful route to the session controller’s new action. This is where login is initiated.
Run the test, and success!
5. Writing New Unit Tests
Now that we have our existing tests working, and perhaps you’ve tried your hands at an extra functional test, let’s take a brief moment to look at the unit tests of our models. We’ll build a couple of simple tests to get you started.
a. Fixtures with Associations
First, we’ve mentioned in our lectures that fixtures have the ability to not only capture simple values representative of working and broken versions of our models, but also can represent associations such as has_many, has_one, belongs_to, and has_and_belongs_to_many (HABTM). The fixture implementation in Rails 2 allows us to set these relationships up in a natural way, specifying these associations by name.
Let’s look at the fixtures in two of our models, Links and Categories, to see this in action. These two models have HABTM associations. When you have models instantiated in code, and you have used the association methods to identify the relationships, you gain a number of useful utility methods to address the members of the relation. In much the same way, we can “name names” in fixtures.
Suppose we had this links.yml file:
apple: name: Apple url: http://www.apple.com/ google: name: Google url: http://www.google.com/ learningrails: name: LearningRails PodCast url: http://www.learningrails.com/
and this categories.yml file:
site: name: Site blog: name: Blog podcast: name: Podcast
To indicate which links are currently categorized, we’ll add an attribute “categories:” and literally list out the name of each category fixture member:
apple: name: Apple url: http://www.apple.com/ categories: site google: name: Google url: http://www.google.com/ categories: site learningrails: name: LearningRails PodCast url: http://www.learningrails.com/ categories: podcast, site
Note that for “apple” and “google”, we’ve only associated one category with each entry, but we’ve still named the attribute “categories”. Rails will be able to tie the “site” category fixture entry to the associated “link” fixture when it creates the models for you during tests.
“learningrails”, on the other hand, has two categories associated with it, podcast and site, so we list these out as a comma separated list.
Let’s look at the categories.yml file:
site: name: site links: apple, google, learningrails blog: name: blog podcast: name: Podcast links: learningrails
Here we do the same thing, and link up (no pun intended) the association from the other side. We have three instances of a “site”, so we list them out as “links” (apple, google, learningrails). We have one podcast, but we still identify it with the plural form “links”. Note also that we don’t have any “blog” items, so that entry in the fixture doesn’t use the “links” identifier.
Naming conventions are important here, and the capitalization and pluralization we’ve used is critical.
Now that we have a data set set up, let’s test it out.
b. Testing Associations in our Model
We are going to load our fixtures into the CategoryTest class, just like we did for functional tests earlier:
class CategoryTest < ActiveSupport::TestCase
fixtures :categories, :links
...
Note that since there are relationships between the models, we need both fixture data files. If one or the other was missing, this would not work.
Now, let’s write a simple test that confirms the test data we just created (and demonstrates that Rails’ use of Test::Unit, the testing framework behind all of this magic, auto loads our data). This test will first check that our “site” category has three links using the notation we explained earlier for referring to fixture members:
def test_category_has_links
catsite = categories(:site)
assert_not_nil catsite
assert_equal 3, catsite.links.length
end
Here, we load a local variable “catsite” with the object defined with the “site” name in the categories.yml file. Recall we set up three links, so after asserting that we actually loaded the object with assert_not_nil, we refer to the HABTM derived method “links”, and check that three objects are in that array. Success!
We could further test that the data is also loaded into the test database by using a finder method:
def test_category_load_site_category
catsite = Category.find_by_name('site')
assert_not_nil catsite
assert_equal 3, catsite.links.length
end
Again, we should test and have success!
c. One more unit test
Let’s do a slightly more realistic unit test. Let’s add a validation to one of our models and make sure it is working. In the Category model, let’s validate for the presence of the name attribute:
class Category < ActiveRecord::Base
has_and_belongs_to_many :links
validates_presence_of :name
end
In our CategoryTest class, let’s add two tests that simply try to create an empty model and another that creates a model with a name. The first should fail and the second should succeed:
def test_should_create_category_with_name
assert_difference 'Category.count' do
Category.create(:name => 'testname')
end
end
def test_should_not_create_category_no_name
assert_no_difference 'Category.count' do
Category.create()
end
end
The assert_difference and assert_no_difference methods run the code snippet provided as a parameter before and after their block executes (in this case, the code that tries to create a Category object). Category.count queries the database and gets the number of Category objects that currently exist. In the case where a create method works, the count should be different (one greater). In the case where it fails, the count should be identical.
We run out tests and… success? Our new tests are working, but one of our old tests is failing!
When we added the validation in the Category model, the test case test_should_create_category(CategoriesControllerTest) broke, because it tries to create an empty Category. To fix this, we can simply supply a name parameter:
def test_should_create_category
login_as :quentin
assert_difference('Category.count') do
post :create, :category => { :name => 'testname' }
end
assert_redirected_to category_path(assigns(:category))
end
Run the tests again, and finally, we have everything working again.
6. Further Exercises
- Install the ZenTest gem and take autotest for a spin (gem install ZenTest). This will continually run your tests as you code and save files. A real time saver.
- Fix the remaining (if any, purposely) broken tests in the lab6 or later source (release 6 in Unfuddle).
- Add a few validations to the existing models and create the associated tests to check good and bad data test cases.
- links_controller_test is missing a couple of tests for two actions we added later in development. What are they and can you write those tests?
- Fix up the email settings by moving the initializer code out of mail.rb and in to the different environment files.
- Add the Spider plugin for a simple Integration Test that crawls your site (script/plugin install svn://caboo.se/plugins/court3nay/spider_test).