Calabash Mobile Testing

Introduction:

This overview provides step-by-step instructions for setting up calabash-android on Windows and explaining the basic components of a calabash-android test suite, with links to relevant resources provided throughout the document.

What are Cucumber and Calabash?

Cucumber is a software tool for writing and running automated acceptance tests using a behavior-driven development style. Test scenarios are written in a domain specific language called Gherkin, which enables the step-by-step description of scenarios in a plain language format intended to be readable by individuals without any programming knowledge. Regular expressions are used to match test steps described in Gherkin to step definitions written in Ruby. These step definitions call methods from an automation API suitable for the target application. Calabash provides these APIs for interacting with native iOS and Android apps.

Requirements:

To create a Calabash test suite for an Android application with calabash-android on Windows, you need a few things in place.

Java SDK
Android SDK from Google
Ruby
Ruby DevKit
Apache Ant
Calabash Android ruby gem
An Android device connected via USB, or the Genymotion Android emulator which is much more stable with Calabash than the emulator from the Android SDK
APK for the application under test

Calabash-Android Setup for Windows

Download and install Oracle JDK: http://www.oracle.com/technetwork/java/javase/downloads/index.html
Download and install the Android SDK from https://developer.android.com/sdk/installing/index.html. Then add the tools and platform-tools directories from the SDK to your PATH environment variable.
Download and install Ruby 1.9.3 (check the option to add Ruby to PATH during installation) http://rubyinstaller.org/downloads/
Download and install apache ant (You can use the self-extractor or WinAnt, but if you use the self-extractor, create an environment variable called ANT_HOME and set it to the directory you extracted it to). https://code.google.com/p/winant/
Download Ruby DevKit from http://rubyinstaller.org/downloads/ (tdm-32-4.5.2 for Ruby 1.9.3), cd to directory you installed it in and run: ruby dk.rb init
Check to make sure the config.yml file contains the path to Ruby 1.9.3 and run: ruby dk.rb install
Install Calabash-Android by running: gem install calabash-android
Install the Genymotion Android emulator, available from: http://www.genymotion.com.

To generate a cucumber skeleton for your Calabash-Android project:

Create a directory for your calabash project files and cd to it
From the directory you created, run: calabash-android gen

Using Calabash Console

The Calabash console allows you to interact with the application under test directly using the calabash API. This is useful for exploring and examining user interface elements and gathering the identifying characteristics necessary to write automation code.

calabash-android resign <path_to_apk>
calabash-android console <path_to_apk>
Start Genymotion Android emulator
In calabash console, run:

reinstall_apps

start_test_server_in_background

Now the calabash-console can be used interactively.

Running a Calabash-Android test suite

To run a Calabash-Android test suite, run:

calabash-android run <path_to_apk>

For more detailed output, you can use the –verbose option.

For colored output from cucumber, install ANSICON:

Unzip the file from https://github.com/adoxa/ansicon/downloads
Navigate to the unzipped directory from a command prompt
Type ansicon –i to install ANSICON

Now you should see colored output from your cucumber tests. If you get an error message when you run ansicon, try installing an older version.

More Information:

Components of Calabash-Android:

Gherkin syntax is concisely described here:

https://github.com/cucumber/cucumber/wiki/Gherkin

Note the Feature Introduction and Given-When-Then links on the bottom of the page.

Cucumber step definitions are described here:

https://github.com/cucumber/cucumber/wiki/Step-Definitions

The Calabash-Android Ruby API is described here:

https://github.com/calabash/calabash-android/blob/master/documentation/ruby_api.md)

Other Ruby methods that are useful can be found by examining the source code found in operations.rb:

https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-android/operations.rb

Note that undocumented methods are more likely to change than the documented methods. You may find it instructive to see how the methods described in operations.rb are built using the documented API methods as building blocks.

Useful Links

Xamarin Test Cloud is a cloud-based service that allows you to run Calabash tests against hundreds of different devices: http://developer.xamarin.com/guides/testcloud/introduction-to-test-cloud/

Xamarin Calabash guide: http://developer.xamarin.com/guides/testcloud/calabash/

Calabash-Android Ruby API: https://github.com/calabash/calabash-android/blob/master/documentation/ruby_api.md

Calabash-Android Canned Steps: https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-android/canned_steps.md

Calabash-Android API code: https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-android/operations.rb
Calabash-Android Query Syntax: https://github.com/calabash/calabash-android/wiki/05-Query-Syntax

Gherkin Syntax: https://github.com/cucumber/cucumber/wiki/Gherkin

Gherkin Scenario Outlines and tables: https://github.com/cucumber/cucumber/wiki/Scenario-Outlines

Criticism of Cucumber: http://www.jimmycuadra.com/posts/please-don-t-use-cucumber

Cucumber Step Definitions: https://github.com/cucumber/cucumber/wiki/Step-Definitions

Cucumber RegEx Cheat Sheet: http://35qk152ejao6mi5pan29erbr9.wpengine.netdna-cdn.com/wp-content/uploads/2011/08/Cucumber-Regular-Expressions-Cheat-Sheet.pdf