1 page.title=Testing In Other IDEs
6 <h2>In this document</h2>
9 <a href="#CreateTestProjectCommand">Working with Test Projects</a>
12 <a href="#CreateTestProject">Creating a test project</a>
15 <a href="#UpdateTestProject">Updating a test project</a>
20 <a href="#CreateTestApp">Creating a Test Package</a>
23 <a href="#RunTestsCommand">Running Tests</a>
26 <a href="#RunTestsAnt">Quick build and run with Ant</a>
29 <a href="#RunTestsDevice">Running tests on a device or emulator</a>
34 <a href="#AMSyntax">Using the Instrument Command</a>
37 <a href="#AMOptionsSyntax">Instrument options</a>
40 <a href="#RunTestExamples">Instrument examples</a>
48 <a href="{@docRoot}guide/topics/testing/testing_android.html">
49 Testing Fundamentals</a>
52 <a href="{@docRoot}guide/developing/tools/adb.html">Android Debug Bridge</a>
58 This document describes how to create and run tests directly from the command line.
59 You can use the techniques described here if you are developing in an IDE other than Eclipse
60 or if you prefer to work from the command line. This document assumes that you already know how
61 to create a Android application in your programming environment. Before you start this
62 document, you should read the topic
63 <a href="{@docRoot}guide/topics/testing/testing_android.html">Testing Fundamentals</a>,
64 which provides an overview of Android testing.
67 If you are developing in Eclipse with ADT, you can set up and run your tests
68 directly in Eclipse. For more information, please read
69 <a href="{@docRoot}guide/developing/testing/testing_eclipse.html">
70 Testing in Eclipse, with ADT</a>.
72 <h2 id="CreateTestProjectCommand">Working with Test Projects</h2>
74 You use the <code>android</code> tool to create test projects.
75 You also use <code>android</code> to convert existing test code into an Android test project,
76 or to add the <code>run-tests</code> Ant target to an existing Android test project.
77 These operations are described in more detail in the section <a href="#UpdateTestProject">
78 Updating a test project</a>. The <code>run-tests</code> target is described in
79 <a href="#RunTestsAnt">Quick build and run with Ant</a>.
81 <h3 id="CreateTestProject">Creating a test project</h3>
83 To create a test project with the <code>android</code> tool, enter:
86 android create test-project -m <main_path> -n <project_name> -p <test_path>
89 You must supply all the flags. The following table explains them in detail:
98 <td><code>-m, --main</code></td>
100 Path to the project of the application under test, relative to the test package
104 For example, if the application under test is in <code>source/HelloAndroid</code>, and
105 you want to create the test project in <code>source/HelloAndroidTest</code>, then the
106 value of <code>--main</code> should be <code>../HelloAndroid</code>.
108 To learn more about choosing the location of test projects, please read
109 <a href="{@docRoot}guide/topics/testing/testing_android.html#TestProjects">
110 Testing Fundamentals</a>.
115 <td><code>-n, --name</code></td>
116 <td>Name that you want to give the test project.</td>
120 <td><code>-p, --path</code></td>
121 <td>Directory in which you want to create the new test project.</td>
123 The <code>android</code> tool creates the test project files and directory structure
124 in this directory. If the directory does not exist, <code>android</code> creates it.
129 If the operation is successful, <code>android</code> lists to STDOUT the names of the files
130 and directories it has created.
133 This creates a new test project with the appropriate directories and build files. The directory
134 structure and build file contents are identical to those in a regular Android application
135 project. They are described in detail in the topic
136 <a href="{@docRoot}guide/developing/other-ide.html">Developing In Other IDEs</a>.
139 The operation also creates an <code>AndroidManifest.xml</code> file with instrumentation
140 information. When you run the test, Android uses this information to load the application you
141 are testing and control it with instrumentation.
144 For example, suppose you create the <a href="{@docRoot}resources/tutorials/hello-world.html">
145 Hello, World</a> tutorial application in the directory <code>~/source/HelloAndroid</code>.
146 In the tutorial, this application uses the package name <code>com.example.helloandroid</code>
147 and the activity name <code>HelloAndroid</code>. You can to create the test for this in
148 <code>~/source/HelloAndroidTest</code>. To do so, you enter:
152 $ android create test-project -m ../HelloAndroid -n HelloAndroidTest -p HelloAndroidTest
155 This creates a directory called <code>~/src/HelloAndroidTest</code>. In the new directory you
156 see the file <code>AndroidManifest.xml</code>. This file contains the following
157 instrumentation-related elements and attributes:
161 <code><application></code>: to contain the
162 <code><uses-library></code> element.
165 <code><uses-library android:name="android.test.runner"</code>:
166 specifies this testing application uses the <code>android.test.runner</code> library.
169 <code><instrumentation></code>: contains attributes that control Android
170 instrumentation. The attributes are:
173 <code>android:name="android.test.InstrumentationTestRunner"</code>:
174 {@link android.test.InstrumentationTestRunner} runs test cases. It extends both
175 JUnit test case runner classes and Android instrumentation classes.
178 <code>android:targetPackage="com.example.helloandroid"</code>: specifies
179 that the tests in HelloAndroidTest should be run against the application with the
180 <em>Android</em> package name <code>com.example.helloandroid</code>. This is the
181 package name of the <a
182 href="{@docRoot}resources/tutorials/hello-world.html">Hello, World</a>
183 tutorial application.
186 <code>android:label="Tests for .HelloAndroid"</code>: specifies a
187 user-readable label for the instrumentation class. By default,
188 the <code>android</code> tool gives it the value "Tests for " plus
189 the name of the main Activity of the application under test.
194 <h3 id="UpdateTestProject">Updating a test project</h3>
196 You use the <code>android</code> tool when you need to change the path to the
197 project of the application under test. If you are changing an existing test project created in
198 Eclipse with ADT so that you can also build and run it from the command line, you must use the
199 "create" operation. See the section <a href="#CreateTestProject">Creating a test project</a>.
202 <strong>Note:</strong> If you change the Android package name of the application under test,
203 you must <em>manually</em> change the value of the <code><android:targetPackage></code>
204 attribute within the <code>AndroidManifest.xml</code> file of the test package.
205 Running <code>android update test-project</code> does not do this.
208 To update a test project with the <code>android</code> tool, enter:
210 <pre>android update-test-project -m <main_path> -p <test_path></pre>
219 <td><code>-m, --main</code></td>
220 <td>The path to the project of the application under test, relative to the test project</td>
222 For example, if the application under test is in <code>source/HelloAndroid</code>, and
223 the test project is in <code>source/HelloAndroidTest</code>, then the value for
224 <code>--main</code> is <code>../HelloAndroid</code>.
228 <td><code>-p, --path</code></td>
229 <td>The of the test project.</td>
231 For example, if the test project is in <code>source/HelloAndroidTest</code>, then the
232 value for <code>--path</code> is <code>HelloAndroidTest</code>.
237 If the operation is successful, <code>android</code> lists to STDOUT the names of the files
238 and directories it has created.
240 <h2 id="CreateTestApp">Creating a Test Package</h2>
242 Once you have created a test project, you populate it with a test package.
243 The application does not require an {@link android.app.Activity Activity},
244 although you can define one if you wish. Although your test package can
245 combine Activities, Android test class extensions, JUnit extensions, or
246 ordinary classes, you should extend one of the Android test classes or JUnit classes,
247 because these provide the best testing features.
250 If you run your tests with {@link android.test.InstrumentationTestRunner}
251 (or a related test runner), then it will run all the methods in each class. You can modify
252 this behavior by using the {@link junit.framework.TestSuite TestSuite} class.
256 To create a test package, start with one of Android's test classes in the Java package
257 {@link android.test android.test}. These extend the JUnit
258 {@link junit.framework.TestCase TestCase} class. With a few exceptions, the Android test
259 classes also provide instrumentation for testing.
262 For test classes that extend {@link junit.framework.TestCase TestCase}, you probably want to
263 override the <code>setUp()</code> and <code>tearDown()</code> methods:
267 <code>setUp()</code>: This method is invoked before any of the test methods in the class.
268 Use it to set up the environment for the test. You can use <code>setUp()</code>
269 to instantiate a new <code>Intent</code> object with the action <code>ACTION_MAIN</code>.
270 You can then use this intent to start the Activity under test.
272 <strong>Note:</strong> If you override this method, call
273 <code>super.setUp()</code> as the first statement in your code.
277 <code>tearDown()</code>: This method is invoked after all the test methods in the class. Use
278 it to do garbage collection and re-setting before moving on to the next set of tests.
279 <p class="note"><strong>Note:</strong> If you override this method, you must call
280 <code>super.tearDown()</code> as the <em>last</em> statement in your code.</p>
284 Another useful convention is to add the method <code>testPreConditions()</code> to your test
285 class. Use this method to test that the application under test is initialized correctly. If this
286 test fails, you know that that the initial conditions were in error. When this happens, further
287 test results are suspect, regardless of whether or not the tests succeeded.
290 To learn more about creating test packages, see the topic <a
291 href="{@docRoot}guide/topics/testing/testing_android.html">Testing Fundamentals</a>,
292 which provides an overview of Android testing. If you prefer to follow a tutorial,
293 try the <a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a>
294 tutorial, which leads you through the creation of tests for an actual Android application.
296 <h2 id="RunTestsCommand">Running Tests</h2>
298 You run tests from the command line, either with Ant or with an
299 <a href="{@docRoot}http://developer.android.com/guide/developing/tools/adb.html">
300 Android Debug Bridge (adb)</a> shell.
302 <h3 id="RunTestsAnt">Quick build and run with Ant</h3>
304 You can use Ant to run all the tests in your test project, using the target
305 <code>run-tests</code>, which is created automatically when you create a test project with
306 the <code>android</code> tool.
309 This target re-builds your main project and test project if necessary, installs the test
310 application to the current AVD or device, and then runs all the test classes in the test
311 application. The results are directed to <code>STDOUT</code>.
314 You can update an existing test project to use this feature. To do this, use the
315 <code>android</code> tool with the <code>update test-project</code> option. This is described
316 in the section <a href="#UpdateTestProject">Updating a test project</a>.
318 <h3 id="RunTestsDevice">Running tests on a device or emulator</h3>
320 When you run tests from the command line with
321 <a href="{@docRoot}http://developer.android.com/guide/developing/tools/adb.html">
322 Android Debug Bridge (adb)</a>, you get more options for choosing the tests
323 to run than with any other method. You can select individual test methods, filter tests
324 according to their annotation, or specify testing options. Since the test run is controlled
325 entirely from a command line, you can customize your testing with shell scripts in various ways.
328 To run a test from the command line, you run <code>adb shell</code> to start a command-line
329 shell on your device or emulator, and then in the shell run the <code>am instrument</code>
330 command. You control <code>am</code> and your tests with command-line flags.
333 As a shortcut, you can start an <code>adb</code> shell, call <code>am instrument</code>, and
334 specify command-line flags all on one input line. The shell opens on the device or emulator,
335 runs your tests, produces output, and then returns to the command line on your computer.
338 To run a test with <code>am instrument</code>:
342 If necessary, rebuild your main application and test package.
345 Install your test package and main application Android package files
346 (<code>.apk</code> files) to your current Android device or emulator</li>
348 At the command line, enter:
350 $ adb shell am instrument -w <test_package_name>/<runner_class>
353 where <code><test_package_name></code> is the Android package name of your test
354 application, and <code><runner_class></code> is the name of the Android test
355 runner class you are using. The Android package name is the value of the
356 <code>package</code> attribute of the <code>manifest</code> element in the manifest file
357 (<code>AndroidManifest.xml</code>) of your test package. The Android test runner
358 class is usually {@link android.test.InstrumentationTestRunner}.
361 Your test results appear in <code>STDOUT</code>.
366 This operation starts an <code>adb</code> shell, then runs <code>am instrument</code>
367 with the specified parameters. This particular form of the command will run all of the tests
368 in your test package. You can control this behavior with flags that you pass to
369 <code>am instrument</code>. These flags are described in the next section.
371 <h2 id="AMSyntax">Using the am instrument Command</h2>
373 The general syntax of the <code>am instrument</code> command is:
376 am instrument [flags] <test_package>/<runner_class>
379 The main input parameters to <code>am instrument</code> are described in the following table:
395 <code><test_package></code>
398 The Android package name of the test package.
401 The value of the <code>package</code> attribute of the <code>manifest</code>
402 element in the test package's manifest file.
407 <code><runner_class></code>
410 The class name of the instrumented test runner you are using.
413 This is usually {@link android.test.InstrumentationTestRunner}.
418 The flags for <code>am instrument</code> are described in the following table:
440 Forces <code>am instrument</code> to wait until the instrumentation terminates
441 before terminating itself. The net effect is to keep the shell open until the tests
442 have finished. This flag is not required, but if you do not use it, you will not
443 see the results of your tests.
454 Outputs results in raw format. Use this flag when you want to collect
455 performance measurements, so that they are not formatted as test results. This flag is
456 designed for use with the flag <code>-e perf true</code> (documented in the section
457 <a href="#AMOptionsSyntax">Instrument options</a>).
468 Provides testing options as key-value pairs. The
469 <code>am instrument</code> tool passes these to the specified instrumentation class
470 via its <code>onCreate()</code> method. You can specify multiple occurrences of
471 <code>-e <test_options></code>. The keys and values are described in the
472 section <a href="#AMOptionsSyntax">am instrument options</a>.
474 The only instrumentation class that uses these key-value pairs is
475 {@link android.test.InstrumentationTestRunner} (or a subclass). Using them with
476 any other class has no effect.
482 <h3 id="AMOptionsSyntax">am instrument options</h3>
484 The <code>am instrument</code> tool passes testing options to
485 <code>InstrumentationTestRunner</code> or a subclass in the form of key-value pairs,
486 using the <code>-e</code> flag, with this syntax:
489 -e <key> <value>
492 Some keys accept multiple values. You specify multiple values in a comma-separated list.
493 For example, this invocation of <code>InstrumentationTestRunner</code> provides multiple
494 values for the <code>package</code> key:
497 $ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 \
498 > com.android.test/android.test.InstrumentationTestRunner
501 The following table describes the key-value pairs and their result. Please review the
502 <strong>Usage Notes</strong> following the table.
515 <Java_package_name>
518 The fully-qualified <em>Java</em> package name for one of the packages in the test
519 application. Any test case class that uses this package name is executed. Notice that
520 this is not an <em>Android</em> package name; a test package has a single
521 Android package name but may have several Java packages within it.
525 <td rowspan="2"><code>class</code></td>
526 <td><class_name></td>
528 The fully-qualified Java class name for one of the test case classes. Only this test
529 case class is executed.
533 <td><class_name><strong>#</strong>method name</td>
535 A fully-qualified test case class name, and one of its methods. Only this method is
536 executed. Note the hash mark (#) between the class name and the method name.
540 <td><code>func</code></td>
541 <td><code>true</code></td>
543 Runs all test classes that extend {@link android.test.InstrumentationTestCase}.
547 <td><code>unit</code></td>
548 <td><code>true</code></td>
550 Runs all test classes that do <em>not</em> extend either
551 {@link android.test.InstrumentationTestCase} or
552 {@link android.test.PerformanceTestCase}.
556 <td><code>size</code></td>
558 [<code>small</code> | <code>medium</code> | <code>large</code>]
561 Runs a test method annotated by size. The annotations are <code>@SmallTest</code>,
562 <code>@MediumTest</code>, and <code>@LargeTest</code>.
566 <td><code>perf</code></td>
567 <td><code>true</code></td>
569 Runs all test classes that implement {@link android.test.PerformanceTestCase}.
570 When you use this option, also specify the <code>-r</code> flag for
571 <code>am instrument</code>, so that the output is kept in raw format and not
572 re-formatted as test results.
576 <td><code>debug</code></td>
577 <td><code>true</code></td>
579 Runs tests in debug mode.
583 <td><code>log</code></td>
584 <td><code>true</code></td>
586 Loads and logs all specified tests, but does not run them. The test
587 information appears in <code>STDOUT</code>. Use this to verify combinations of other
588 filters and test specifications.
592 <td><code>emma</code></td>
593 <td><code>true</code></td>
595 Runs an EMMA code coverage analysis and writes the output to
596 <code>/data//coverage.ec</code> on the device. To override the file location, use the
597 <code>coverageFile</code> key that is described in the following entry.
599 <strong>Note:</strong> This option requires an EMMA-instrumented build of the test
600 application, which you can generate with the <code>coverage</code> target.
605 <td><code>coverageFile</code></td>
606 <td><code><filename></code></td>
608 Overrides the default location of the EMMA coverage file on the device. Specify this
609 value as a path and filename in UNIX format. The default filename is described in the
610 entry for the <code>emma</code> key.
614 <strong><code>-e</code> Flag Usage Notes</strong>
617 <code>am instrument</code> invokes
618 {@link android.test.InstrumentationTestRunner#onCreate(Bundle)}
619 with a {@link android.os.Bundle} containing the key-value pairs.
622 The <code>package</code> key takes precedence over the <code>class</code> key. If you
623 specifiy a package, and then separately specify a class within that package, Android
624 will run all the tests in the package and ignore the <code>class</code> key.
627 The <code>func</code> key and <code>unit</code> key are mutually exclusive.
630 <h3 id="RunTestExamples">Usage examples</h3>
632 The following sections provide examples of using <code>am instrument</code> to run tests.
633 They are based on the following structure:</p>
636 The test package has the Android package name <code>com.android.demo.app.tests</code>
639 There are three test classes:
642 <code>UnitTests</code>, which contains the methods
643 <code>testPermissions</code> and <code>testSaveState</code>.
646 <code>FunctionTests</code>, which contains the methods
647 <code>testCamera</code>, <code>testXVGA</code>, and <code>testHardKeyboard</code>.
650 <code>IntegrationTests</code>,
651 which contains the method <code>testActivityProvider</code>.
656 The test runner is {@link android.test.InstrumentationTestRunner}.
659 <h4>Running the entire test package</h4>
661 To run all of the test classes in the test package, enter:
664 $ adb shell am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner
666 <h4>Running all tests in a test case class</h4>
668 To run all of the tests in the class <code>UnitTests</code>, enter:
671 $ adb shell am instrument -w \
672 > -e class com.android.demo.app.tests.UnitTests \
673 > com.android.demo.app.tests/android.test.InstrumentationTestRunner
676 <code>am instrument</code> gets the value of the <code>-e</code> flag, detects the
677 <code>class</code> keyword, and runs all the methods in the <code>UnitTests</code> class.
679 <h4>Selecting a subset of tests</h4>
681 To run all of the tests in <code>UnitTests</code>, and the <code>testCamera</code> method in
682 <code>FunctionTests</code>, enter:
685 $ adb shell am instrument -w \
686 > -e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera \
687 > com.android.demo.app.tests/android.test.InstrumentationTestRunner
690 You can find more examples of the command in the documentation for
691 {@link android.test.InstrumentationTestRunner}.