1 page.title=Testing and Instrumentation
5 <h2>In this document</h2>
8 <a href="#Overview">Overview</a>
11 <a href="#TestAPI">The Testing API</a>
14 <a href="#Extensions">JUnit test case classes</a>
17 <a href="#Instrumentation">Instrumentation test case classes</a>
20 <a href="#Assert">Assert classes</a>
23 <a href="#MockObjects">Mock object classes</a>
26 <a href="#InstrumentationTestRunner">Instrumentation Test Runner</a>
31 <a href="#TestEnviroment">Working in the Test Environment</a>
34 <a href="#TestAreas">What to Test</a>
37 <a href="#UITesting">Appendix: UI Testing Notes</a>
40 <a href="#RunOnUIThread">Testing on the UI thread</a>
43 <a href="#NotouchMode">Turning off touch mode</a>
46 <a href="#UnlockDevice">Unlocking the Emulator or Device</a>
49 <a href="#UITestTroubleshooting">Troubleshooting UI tests</a>
56 <li>{@link android.test.InstrumentationTestRunner}</li>
57 <li>{@link android.test.ActivityInstrumentationTestCase2}</li>
58 <li>{@link android.test.ActivityUnitTestCase}</li>
59 <li>{@link android.test.ApplicationTestCase}</li>
60 <li>{@link android.test.ProviderTestCase2}</li>
61 <li>{@link android.test.ServiceTestCase}</li>
63 <h2>Related Tutorials</h2>
66 <a href="{@docRoot}resources/tutorials/testing/helloandroid_test.html">Hello, Testing</a>
69 <a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a>
75 <a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a>
78 <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a>
84 Android includes a powerful set of testing tools that extend the industry-standard JUnit test framework with features specific to the Android environment. Although you can
85 test an Android application with JUnit, the Android tools allow you to write much more sophisticated tests for every aspect of your application, both at the unit and at the framework level.
88 Key features of the Android testing environment include:
91 <li>Android extensions to the JUnit framework that provide access to Android system objects.</li>
92 <li>An instrumentation framework that lets tests control and examine the application.</li>
93 <li>Mock versions of commonly-used Android system objects.</li>
94 <li>Tools for running single tests or test suites, with or without instrumentation.</li>
95 <li>Support for managing tests and test projects in the ADT Plugin for Eclipse and at the command line.</li>
98 This document is an overview of the Android testing environment and the way you use it. The document assumes you have a basic knowledge of
99 Android application programming and JUnit testing methodology.
101 <h2 id="Overview">Overview</h2>
103 At the heart of the Android testing environment is an instrumentation framework that your test application uses to precisely control the application under test. With instrumentation, you can
104 set up mock system objects such as Contexts before the main application starts, control your application at various points of its lifecycle, send UI events to the application, and
105 examine the application's state during its execution. The instrumentation framework accomplishes this by running both the main application and the test application in the same process.
108 Your test application's manifest file links it to the application under test. The the <instrumentation> attribute in the manifest file points to the application under test
109 and also tells Android how to run the test application. This is described in more detail in the section <a href="#InstrumentationTestRunner">Instrumentation Test Runner</a>.
112 The following diagram summarizes the Android testing environment:
114 <img src="{@docRoot}images/testing/android_test_framework.png"/>
115 <!-- Something about test tools in Eclipse and in other IDEs -->
117 In Android, test applications are themselves Android applications, so you write them in much the same way as the application you are testing. The SDK tools
118 help you create a main application project and its test project at the same time. You can run Android tests within Eclipse with ADT or from the command line.
119 Eclipse with ADT provides an extensive set of tools for creating tests, running them, and viewing their results. You can also use the <code>adb</code> tool to
120 run tests, or use a built-in Ant target.
123 To learn how to set up and run tests in Eclipse, please refer to <a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a>.
124 If you're not working in Eclipse, refer to <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a>.
127 If you want a step-by-step introduction to Android testing, try one of the testing tutorials:
131 The <a href="{@docRoot}resources/tutorials/testing/helloandroid_test.html">Hello, Testing</a> tutorial introduces basic testing concepts and procedures in the
132 context of the Hello, World application.
135 The <a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a> tutorial is an excellent follow-up to the Hello, Testing tutorial.
136 It guides you through a more complex testing scenario that you develop against a more realistic application.
139 <h2 id="TestAPI">The Testing API</h2>
141 For writing tests and test applications in the Java programming language, Android provides a
142 testing API that is based in part on the JUnit test framework. Adding to that, Android includes
143 a powerful instrumentation framework that lets your tests access the state and runtime objects
144 of the application under tests.
146 <p>The sections below describe the major components of the testing API available in Android.</p>
147 <h3 id="Extensions">JUnit test case classes</h3>
149 Some of the classes in the testing API extend the JUnit {@link junit.framework.TestCase TestCase} but do not use the instrumentation framework. These classes
150 contain methods for accessing system objects such as the Context of the application under test. With this Context, you can look at its resources, files, databases,
151 and so forth. The base class is {@link android.test.AndroidTestCase}, but you usually use a subclass associated with a particular component.
157 {@link android.test.ApplicationTestCase} - A class for testing an entire application. It allows you to inject a mock Context into the application,
158 set up initial test parameters before the application starts, and examine the application after it finishes but before it is destroyed.
161 {@link android.test.ProviderTestCase2} - A class for isolated testing of a single {@link android.content.ContentProvider}. Since it is restricted to using a
162 {@link android.test.mock.MockContentResolver} for the provider, and it injects an {@link android.test.IsolatedContext}, your provider testing is isolated
163 from the rest of the OS.
166 {@link android.test.ServiceTestCase} - a class for isolated testing of a single {@link android.app.Service}. You can inject a mock Context or
167 mock Application (or both), or let Android provide you a full Context and a {@link android.test.mock.MockApplication}.
170 <h3 id="Instrumentation">Instrumentation test case classes</h3>
172 The API for testing activities extends the JUnit {@link junit.framework.TestCase TestCase} class and also uses the instrumentation framework. With instrumentation,
173 Android can automate UI testing by sending events to the application under test, precisely control the start of an activity, and monitor the state of the
174 activity during its life cycle.
177 The base class is {@link android.test.InstrumentationTestCase}. All of its subclasses have the ability to send a keystroke or touch event to the UI of the application
178 under test. The subclasses can also inject a mock Intent.
183 {@link android.test.ActivityTestCase} - A base class for activity test classes.
186 {@link android.test.SingleLaunchActivityTestCase} - A convenience class for testing a single activity.
187 It invokes {@link junit.framework.TestCase#setUp() setUp()} and {@link junit.framework.TestCase#tearDown() tearDown()} only
188 once, instead of once per method call. Use it when all of your test methods run against the same activity.
191 {@link android.test.SyncBaseInstrumentation} - A class that tests synchronization of a content provider. It uses instrumentation to cancel and disable
192 existing synchronizations before starting the test synchronization.
195 {@link android.test.ActivityUnitTestCase} - This class does an isolated test of a single activity. With it, you can inject a mock context or application, or both.
196 It is intended for doing unit tests of an activity, and is the activity equivalent of the test classes described in <a href="#Extensions">JUnit test case classes</a>.
197 <p> Unlike the other instrumentation classes, this test class cannot inject a mock Intent.</p>
200 {@link android.test.ActivityInstrumentationTestCase2} - This class tests a single activity within the normal system environment.
201 You cannot inject a mock Context, but you can inject mock Intents. Also, you can run a test method on the UI thread (the main thread of the application under test),
202 which allows you to send key and touch events to the application UI.
205 <h3 id="Assert">Assert classes</h3>
207 Android also extends the JUnit {@link junit.framework.Assert} class that is the basis of <code>assert()</code> calls in tests.
208 There are two extensions to this class, {@link android.test.MoreAsserts} and {@link android.test.ViewAsserts}:
212 The <code>MoreAsserts</code> class contains more powerful assertions such as {@link android.test.MoreAsserts#assertContainsRegex} that does regular expression matching.
215 The {@link android.test.ViewAsserts} class contains useful assertions about Android Views, such as {@link android.test.ViewAsserts#assertHasScreenCoordinates} that tests if a View has a particular X and Y
216 position on the visible screen. These asserts simplify testing of geometry and alignment in the UI.
219 <h3 id="MockObjects">Mock object classes</h3>
221 Android has convenience classes for creating mock system objects such as applications, contexts, content resolvers, and resources. Android also provides
222 methods in some test classes for creating mock Intents. Use these mocks to facilitate dependency injection, since they are easier to use than creating their
223 real counterparts. These convenience classes are found in {@link android.test} and {@link android.test.mock}. They are:
227 {@link android.test.IsolatedContext} - Mocks a Context so that the application using it runs in isolation.
228 At the same time, it has enough stub code to satisfy OS code that tries to communicate with contexts. This class is useful in unit testing.
231 {@link android.test.RenamingDelegatingContext} - Delegates most context functions to an existing, normal context while changing the default file and database
232 names in the context. Use this to test file and database operations with a normal system context, using test names.
235 {@link android.test.mock.MockApplication}, {@link android.test.mock.MockContentResolver}, {@link android.test.mock.MockContext},
236 {@link android.test.mock.MockDialogInterface}, {@link android.test.mock.MockPackageManager},
237 {@link android.test.mock.MockResources} - Classes that create mock Android system objects for use in testing. They expose only those methods that are
238 useful in managing the object. The default implementations of these methods simply throw an Exception. You are expected to extend the classes and
239 override any methods that are called by the application under test.
242 <h3 id="InstrumentationTestRunner">Instrumentation Test Runner</h3>
244 Android provides a custom class for running tests with instrumentation called called
245 {@link android.test.InstrumentationTestRunner}. This class
246 controls of the application under test, runs the test application and the main application in the same process, and routes
247 test output to the appropriate place. Using instrumentation is key to the ability of <code>InstrumentationTestRunner</code> to control the entire test
248 environment at runtime. Notice that you use this test runner even if your test class does not itself use instrumentation.
251 When you run a test application, you first run a system utility called Activity Manager. Activity Manager uses the instrumentation framework to start and control the test runner, which in turn uses instrumentation to shut down any running instances
252 of the main application, starts the test application, and then starts the main application in the same process. This allows various aspects of the test application to work directly with the main application.
255 If you are developing in Eclipse, the ADT plugin assists you in the setup of <code>InstrumentationTestRunner</code> or other test runners.
256 The plugin UI prompts you to specify the test runner class to use, as well as the package name of the application under test.
257 The plugin then adds an <code><instrumentation></code> element with appropriate attributes to the manifest file of the test application.
258 Eclipse with ADT automatically starts a test application under the control of Activity Manager using instrumentation,
259 and redirects the test output to the Eclipse window's JUnit view.
262 If you prefer working from the command line, you can use Ant and the <code>android</code>
263 tool to help you set up your test projects. To run tests with instrumentation, you can access the
264 Activity Manager through the <a href="{@docRoot}guide/developing/tools/adb.html">Android Debug
265 Bridge</a> (<code>adb</code>) tool and the output is directed to <code>STDOUT</code>.
267 <h2 id="TestEnviroment">Working in the Test Environment</h2>
269 The tests for an Android application are contained in a test application, which itself is an Android application. A test application resides in a separate Android project that has the
270 same files and directories as a regular Android application. The test project is linked to the project of the application it tests
271 (known as the application under test) by its manifest file.
274 Each test application contains one or more test case classes based on an Android class for a
275 particular type of component. The test case class contains methods that define tests on some part of the application under test. When you run the test application, Android
276 starts it, loads the application under test into the same process, and then invokes each method in the test case class.
279 The tools and procedures you use with testing depend on the development environment you are using. If you use Eclipse, then the ADT plug in for Eclipse provides tools that
280 allow you to develop and run tests entirely within Eclipse. This is documented in the topic <a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a>.
281 If you use another development environment, then you use Android's command-line tools, as documented in the topic <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a>.
283 <h3 id="TestProjects">Working with test projects</h3>
285 To start testing an Android application, you create a test project for it using Android tools. The tools create the project directory and the files and subdirectories needed.
286 The tools also create a manifest file that links the application in the test project to the application under test. The procedure for creating a test project in Eclipse with
287 ADT is documented in <a href="{@docRoot}guide/developing/testing_eclipse.html">Testing in Eclipse, with ADT</a>. The procedure for creating a test project for use with development
288 tools other than Eclipse is documented in <a href="{@docRoot}guide/developing/testing_otheride.html">Testing in Other IDEs</a>.
290 <h3 id="TestClasses">Working with test case classes</h3>
292 A test application contains one or more test case classes that extend an Android test case class. You choose a test case class based on the type of Android component you are testing and the
293 tests you are doing. A test application can test different components, but each test case class is designed to test a single type of component.
294 The Android test case classes are described in the section <a href="#TestAPI">The Testing API</a>.
297 Some Android components have more than one associated test case class. In this case, you choose among the available classes based on the type of tests you want to do. For activities,
298 for example, you have the choice of either {@link android.test.ActivityInstrumentationTestCase2} or {@link android.test.ActivityUnitTestCase}.
300 <code>ActivityInstrumentationTestCase2</code> is designed to do functional testing, so it tests activities in a normal system infrastructure. You can inject mocked Intents, but not
301 mocked Contexts. In general, you can't mock dependencies for the activity under test.
304 In comparison, <code>ActivityUnitTestCase</code> is designed for unit testing, so it tests activities in an isolated system infrastructure. You can inject mocked or wrappered dependencies for
305 the activity under test, particularly mocked Contexts. On the other hand, when you use this test case class the activity under test runs in isolation and can't interact with other activities.
308 As a rule of thumb, if you wanted to test an activity's interaction with the rest of Android, you would use <code>ActivityInstrumentationTestCase2</code>. If you wanted to do regression testing
309 on an activity, you would use <code>ActivityUnitTestCase</code>.
311 <h3 id="Tests">Working with test methods</h3>
313 Each test case class provides methods that you use to set up the test environment and control the application under test. For example, all test case classes provide the JUnit {@link junit.framework.TestCase#setUp() setUp()}
314 method that you can override to set up fixtures. In addition, you add methods to the class to define individual tests. Each method you add is run once each time you run the test application. If you override the <code>setUp()</code>
315 method, it runs before each of your methods. Similarly, the JUnit {@link junit.framework.TestCase#tearDown() tearDown()} method is run once after each of your methods.
318 The test case classes give you substantial control over starting and stopping components. For this reason, you have to specifically tell Android to start a component before you run tests against it. For example, you use the
319 {@link android.test.ActivityInstrumentationTestCase2#getActivity()} method to start the activity under test. You can call this method once during the entire test case, or once for each test method. You can even destroy the
320 activity under test by calling its {@link android.app.Activity#finish()} method and then restart it with <code>getActivity()</code> within a single test method.
322 <h3 id="RunTests">Running tests and seeing the results</h3>
324 To run your tests, you build your test project and then run the test application using the system utility Activity Manager with instrumentation. You provide to Activity Manager the name of the test runner (usually
325 {@link android.test.InstrumentationTestRunner}) you specified for your application; the name includes both your test application's package name and the test runner class name. Activity Manager loads and starts your
326 test application, kills any instances of the application under test, loads an instance of the application under test into the same process as the test application, and then passes control to the first test case
327 class in your test application. The test runner then takes control of the tests, running each of your test methods against the application under test until all the methods in all the classes have been run.
330 If you run a test within Eclipse with ADT, the output appears in a new JUnit view pane. If you run a test from the command line, the output goes to STDOUT.
332 <h2 id="TestAreas">What to Test</h2>
334 In addition to the functional areas you would normally test, here are some areas
335 of Android application testing that you should consider:
339 Activity lifecycle events: You should test that your activities handle lifecycle events correctly. For example
340 an activity should respond to pause or destroy events by saving its state. Remember that even a change in screen orientation
341 causes the current activity to be destroyed, so you should test that accidental device movements don't accidentally lose the
345 Database operations: You should ensure that database operations correctly handle changes to the application's state.
346 To do this, use mock objects from the package {@link android.test.mock android.test.mock}.
349 Screen sizes and resolutions: Before you publish your application, make sure to test it on all of the
350 screen sizes and densities on which you want it to run. You can test the application on multiple sizes and densities using
351 AVDs, or you can test your application directly on the devices that you are targeting. For more information, see
352 the topic <a href="{@docRoot}guide/practices/screens_support.html">Supporting Multiple Screens</a>.
356 When possible, you should run these tests on an actual device. If this is not possible, you can
357 use the <a href="{@docRoot}guide/developing/tools/emulator.html">Android Emulator</a> with
358 <a href="{@docRoot}guide/developing/tools/avd.html">Android Virtual Devices</a> configured for
359 the hardware, screens, and versions you want to test.
361 <h2 id="UITesting">Appendix: UI Testing Notes</h2>
363 The following sections have tips for testing the UI of your Android application, specifically
364 to help you handle actions that run in the UI thread, touch screen and keyboard events, and home
365 screen unlock during testing.
367 <h3 id="RunOnUIThread">Testing on the UI thread</h3>
369 An application's activities run on the application's <strong>UI thread</strong>. Once the
370 UI is instantiated, for example in the activity's <code>onCreate()</code> method, then all
371 interactions with the UI must run in the UI thread. When you run the application normally, it
372 has access to the thread and does not have to do anything special.
375 This changes when you run tests against the application. With instrumentation-based classes,
376 you can invoke methods against the UI of the application under test. The other test classes don't allow this.
377 To run an entire test method on the UI thread, you can annotate the thread with <code>@UIThreadTest</code>.
378 Notice that this will run <em>all</em> of the method statements on the UI thread. Methods that do not interact with the UI
379 are not allowed; for example, you can't invoke <code>Instrumentation.waitForIdleSync()</code>.
382 To run a subset of a test method on the UI thread, create an anonymous class of type
383 <code>Runnable</code>, put the statements you want in the <code>run()</code> method, and instantiate a new
384 instance of the class as a parameter to the method <code><em>appActivity</em>.runOnUiThread()</code>, where
385 <code><em>appActivity</em></code> is the instance of the app you are testing.
388 For example, this code instantiates an activity to test, requests focus (a UI action) for the Spinner displayed
389 by the activity, and then sends a key to it. Notice that the calls to <code>waitForIdleSync</code> and <code>sendKeys</code>
390 aren't allowed to run on the UI thread:</p>
392 private MyActivity mActivity; // MyActivity is the class name of the app under test
393 private Spinner mSpinner;
397 protected void setUp() throws Exception {
399 mInstrumentation = getInstrumentation();
401 mActivity = getActivity(); // get a references to the app under test
404 * Get a reference to the main widget of the app under test, a Spinner
406 mSpinner = (Spinner) mActivity.findViewById(com.android.demo.myactivity.R.id.Spinner01);
410 public void aTest() {
412 * request focus for the Spinner, so that the test can send key events to it
413 * This request must be run on the UI thread. To do this, use the runOnUiThread method
414 * and pass it a Runnable that contains a call to requestFocus on the Spinner.
416 mActivity.runOnUiThread(new Runnable() {
418 mSpinner.requestFocus();
422 mInstrumentation.waitForIdleSync();
424 this.sendKeys(KeyEvent.KEYCODE_DPAD_CENTER);
427 <h3 id="NotouchMode">Turning off touch mode</h3>
429 To control the emulator or a device with key events you send from your tests, you must turn off
430 touch mode. If you do not do this, the key events are ignored.
433 To turn off touch mode, you invoke <code>ActivityInstrumentationTestCase2.setActivityTouchMode(false)</code>
434 <em>before</em> you call <code>getActivity()</code> to start the activity. You must invoke the method in a test method
435 that is <em>not</em> running on the UI thread. For this reason, you can't invoke the touch mode method
436 from a test method that is annotated with <code>@UIThread</code>. Instead, invoke the touch mode method from <code>setUp()</code>.
438 <h3 id="UnlockDevice">Unlocking the emulator or device</h3>
440 You may find that UI tests don't work if the emulator's or device's home screen is disabled with the keyguard pattern.
441 This is because the application under test can't receive key events sent by <code>sendKeys()</code>. The best
442 way to avoid this is to start your emulator or device first and then disable the keyguard for the home screen.
445 You can also explicitly disable the keyguard. To do this,
446 you need to add a permission in the manifest file (<code>AndroidManifest.xml</code>) and
447 then disable the keyguard in your application under test. Note, though, that you either have to remove this before
448 you publish your application, or you have to disable it programmatically in the published app.
451 To add the the permission, add the element <code><uses-permission android:name="android.permission.DISABLE_KEYGUARD"/></code>
452 as a child of the <code><manifest></code> element. To disable the KeyGuard, add the following code
453 to the <code>onCreate()</code> method of activities you intend to test:
456 mKeyGuardManager = (KeyguardManager) getSystemService(KEYGUARD_SERVICE);
457 mLock = mKeyGuardManager.newKeyguardLock("<em>activity_classname</em>");
458 mLock.disableKeyguard();
460 <p>where <code><em>activity_classname</em></code> is the class name of the activity.</p>
461 <h3 id="UITestTroubleshooting">Troubleshooting UI tests</h3>
463 This section lists some of the common test failures you may encounter in UI testing, and their causes:
466 <dt><code>WrongThreadException</code></dt>
468 <p><strong>Problem:</strong></p>
469 For a failed test, the Failure Trace contains the following error message:
471 android.view.ViewRoot$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
473 <p><strong>Probable Cause:</strong></p>
474 This error is common if you tried to send UI events to the UI thread from outside the UI thread. This commonly happens if you send UI events
475 from the test application, but you don't use the <code>@UIThread</code> annotation or the <code>runOnUiThread()</code> method. The test method tried to interact with the UI outside the UI thread.
476 <p><strong>Suggested Resolution:</strong></p>
477 Run the interaction on the UI thread. Use a test class that provides instrumentation. See the previous section <a href="#RunOnUIThread">Testing on the UI Thread</a>
480 <dt><code>java.lang.RuntimeException</code></dt>
482 <p><strong>Problem:</strong></p>
483 For a failed test, the Failure Trace contains the following error message:
485 java.lang.RuntimeException: This method can not be called from the main application thread
487 <p><strong>Probable Cause:</strong></p>
488 This error is common if your test method is annotated with <code>@UiThreadTest</code> but then tries to
489 do something outside the UI thread or tries to invoke <code>runOnUiThread()</code>.
490 <p><strong>Suggested Resolution:</strong></p>
491 Remove the <code>@UiThreadTest</code> annotation, remove the <code>runOnUiThread()</code> call, or re-factor your tests.