*

robo: runs a smart, automated exploration of the activities in your Android app which records any installation failures or crashes and builds an activity map with associated screenshots and video.

*

instrumentation: runs automated unit or integration tests written using a testing framework. Firebase Test Lab for Android currently supports the Espresso and UI Automator 2.0 testing frameworks.

*

game-loop: executes a special intent built into the game app (a "demo mode") that simulates the actions of a real player. This test type can include multiple game loops (also called "scenarios"), which can be logically organized using scenario labels so that you can run related loops together. Refer to firebase.google.com/docs/test-lab/android/game-loop for more information about how to build and run Game Loop tests.

The type of test to run can be specified with the --type flag, although the type can often be inferred from other flags. Specifically, if the --test flag is present, the test --type defaults to instrumentation. If --test is not present, then --type defaults to robo.
All arguments for gcloud beta firebase test android run may be specified on the command line and/or within an argument file. Run $ gcloud topic arg-files for more information about argument files.

POSITIONAL ARGUMENTS

[ARGSPEC]

An ARG_FILE:ARG_GROUP_NAME pair, where ARG_FILE is the path to a file containing groups of test arguments in yaml format, and ARG_GROUP_NAME is the particular yaml object holding a group of arg:value pairs to use. Run $ gcloud topic arg-files for more information and examples.

COMMONLY USED FLAGS

--app=APP

The path to the application binary file. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation. Android App Bundles are specified as .aab, all other files are assumed to be APKs.

--device=DIMENSION=VALUE,[...]

A list of DIMENSION=VALUE pairs which specify a target device to test against. This flag may be repeated to specify multiple devices. The four device dimensions are: model, version, locale, and orientation. If any dimensions are omitted, they will use a default value. The default value, and all possible values, for each dimension can be found with the list command for that dimension, such as $ gcloud beta firebase test android models list. --device is now the preferred way to specify test devices and may not be used in conjunction with --devices-ids, --os-version-ids, --locales, or --orientations. Omitting all of the preceding dimension-related flags will run tests against a single device using defaults for all four device dimensions.
Examples:

--device model=Nexus6 --device version=23,orientation=portrait --device model=shamu,version=22,locale=zh_CN,orientation=default

--test=TEST

The path to the binary file containing instrumentation tests. The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://.

--timeout=TIMEOUT

The max time this test execution can run before it is cancelled (default: 15m). It does not include any time necessary to prepare and clean up the target device. The maximum possible testing time is 30m on physical devices and 60m on virtual devices. The TIMEOUT units can be h, m, or s. If no unit is given, seconds are assumed. Examples:

---

--timeout 1h is 1 hour

---

--timeout 5m is 5 minutes

---

--timeout 200s is 200 seconds

---

--timeout 100 is 100 seconds

--type=TYPE

The type of test to run. TYPE must be one of: instrumentation, robo, game-loop.

FLAGS

--additional-apks=APK,[APK,...]

A list of up to 100 additional APKs to install, in addition to those being directly tested. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation.

--app-package=APP_PACKAGE

(DEPRECATED) The Java package of the application under test (default: extracted from the APK manifest).
The --app-package flag is deprecated and should no longer be used. By default, the correct application package name is parsed from the APK manifest.

--async

Invoke a test asynchronously without waiting for test results.

--auto-google-login

Automatically log into the test device using a preconfigured Google account before beginning the test. Enabled by default, use --no-auto-google-login to disable.

--directories-to-pull=[DIR_TO_PULL,...]

A list of paths that will be copied from the device's storage to the designated results bucket after the test is complete. These must be absolute paths under /sdcard or /data/local/tmp (for example, --directories-to-pull /sdcard/tempDir1,/data/local/tmp/tempDir2). Path names are restricted to the characters a-zA-Z0-9_-./+. The paths /sdcard and /data will be made available and treated as implicit path substitutions. E.g. if /sdcard on a particular device does not map to external storage, the system will replace it with the external storage path prefix for that device.

--environment-variables=[KEY=VALUE,...]

A comma-separated, key=value map of environment variables and their desired values. This flag is repeatable. The environment variables are mirrored as extra options to the am instrument -e KEY1 VALUE1 ... command and passed to your test runner (typically AndroidJUnitRunner). Examples:
Break test cases into four shards and run only the first shard:

--environment-variables numShards=4,shardIndex=0

Enable code coverage and provide a file path to store the coverage results when using Android Test Orchestrator (--use-orchestrator):

--environment-variables clearPackageData=true,coverage=true,coverageFile=/sdcard/coverage.ec

Enable code coverage and provide a file path to store the coverage results when not using Android Test Orchestrator (--no-use-orchestrator):

--environment-variables coverage=true,coverageFile=/sdcard/coverage.ec

Note: If you need to embed a comma into a VALUE string, please refer to gcloud topic escaping for ways to change the default list delimiter.

--network-profile=PROFILE_ID

The name of the network traffic profile, for example --network-profile=LTE, which consists of a set of parameters to emulate network conditions when running the test (default: no network shaping; see available profiles listed by the $ gcloud firebase test network-profiles list command). This feature only works on physical devices.

--obb-files=OBB_FILE,[OBB_FILE]

A list of one or two Android OBB file names which will be copied to each test device before the tests will run (default: None). Each OBB file name must conform to the format as specified by Android (e.g. [main|patch].0300110.com.example.android.obb) and will be installed into <shared-storage>/Android/obb/<package-name>/ on the test device.

--other-files=FILE=DEVICE_DIR,[...]

A list of file=device-directory pairs that indicate paths of files to push to the device before starting tests, and the device directory to push them to.
Source file paths may be in the local filesystem or in Google Cloud Storage (gs://...). Device directories must be absolute, whitelisted paths (${EXTERNAL_STORAGE}, or ${ANDROID_DATA}/local/tmp).
Examples:

--other-files local/file1=/sdcard/dir1/ --other-files gs://bucket/file2=/sdcard/dir2

This flag only copies files to the device. To install files, like OBB or APK files, see --obb-files and --additional-apks.

--performance-metrics

Monitor and record performance metrics: CPU, memory, network usage, and FPS (game-loop only). Enabled by default, use --no-performance-metrics to disable.

--record-video

Enable video recording during the test. Enabled by default, use --no-record-video to disable.

--results-bucket=RESULTS_BUCKET

The name of a Google Cloud Storage bucket where raw test results will be stored (default: "test-lab-<random-UUID>"). Note that the bucket must be owned by a billing-enabled project, and that using a non-default bucket will result in billing charges for the storage used.

--results-dir=RESULTS_DIR

The name of a unique Google Cloud Storage object within the results bucket where raw test results will be stored (default: a timestamp with a random suffix). Caution: if specified, this argument must be unique for each test matrix you create, otherwise results from multiple test matrices will be overwritten or intermingled.

--results-history-name=RESULTS_HISTORY_NAME

The history name for your test results (an arbitrary string label; default: the application's label from the APK manifest). All tests which use the same history name will have their results grouped together in the Firebase console in a time-ordered test history list.

ANDROID GAME-LOOP TEST FLAGS

--scenario-labels=LABEL,[LABEL,...]

A list of game-loop scenario labels (default: None). Each game-loop scenario may be labeled in the APK manifest file with one or more arbitrary strings, creating logical groupings (e.g. GPU_COMPATIBILITY_TESTS). If --scenario-numbers and --scenario-labels are specified together, Firebase Test Lab will first execute each scenario from --scenario-numbers. It will then expand each given scenario label into a list of scenario numbers marked with that label, and execute those scenarios.

--scenario-numbers=int,[int,...]

A list of game-loop scenario numbers which will be run as part of the test (default: all scenarios). A maximum of 1024 scenarios may be specified in one test matrix, but the maximum number may also be limited by the overall test --timeout setting.

ANDROID INSTRUMENTATION TEST FLAGS

--test-package=TEST_PACKAGE

(DEPRECATED) The Java package name of the instrumentation test (default: extracted from the APK manifest).
The --test-package flag is deprecated and should no longer be used. By default, the correct test package name is parsed from the APK manifest.

--test-runner-class=TEST_RUNNER_CLASS

The fully-qualified Java class name of the instrumentation test runner (default: the last name extracted from the APK manifest).

--test-targets=TEST_TARGET,[TEST_TARGET,...]

A list of one or more test target filters to apply (default: run all test targets). Each target filter must be fully qualified with the package name, class name, or test annotation desired. Any test filter supported by am instrument -e ... is supported. See developer.android.com/reference/android/support/test/runner/AndroidJUnitRunner for more information. Examples:

---

--test-targets "package com.my.package.name"

---

--test-targets "notPackage com.package.to.skip"

---

--test-targets "class com.foo.ClassName"

---

--test-targets "notClass com.foo.ClassName#testMethodToSkip"

---

--test-targets "annotation com.foo.AnnotationToRun"

---

--test-targets "size large notAnnotation com.foo.AnnotationToSkip"

--use-orchestrator

Whether each test runs in its own Instrumentation instance with the Android Test Orchestrator (default: Orchestrator is not used, same as specifying --no-use-orchestrator). Orchestrator is only compatible with AndroidJUnitRunner v1.0 or higher. See developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator for more information about Android Test Orchestrator.

ANDROID ROBO TEST FLAGS

--robo-directives=[TYPE:RESOURCE_NAME=INPUT,...]

A comma-separated (<type>:<key>=<value>) map of robo_directives that you can use to customize the behavior of Robo test. The type specifies the action type of the directive, which may take on values click or text. If no type is provided, text will be used by default. Each key should be the Android resource name of a target UI element and each value should be the text input for that element. Values are only permitted for text type elements, so no value should be specified for click type elements. For example, use

--robo-directives text:username_resource=username,text:password_resource=password

to provide custom login credentials for your app, or

--robo-directives click:sign_in_button=

to instruct Robo to click on the sign in button. To learn more about Robo test and robo_directives, see firebase.google.com/docs/test-lab/android/command-line#custom_login_and_text_input_with_robo_test
Caution: You should only use credentials for test accounts that are not associated with real users.

--robo-script=ROBO_SCRIPT

The path to a Robo Script JSON file. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation. You can guide the Robo test to perform specific actions by recording a Robo Script in Android Studio and then specifying this argument. Learn more at firebase.google.com/docs/test-lab/robo-ux-test#scripting

DEPRECATED DEVICE DIMENSIONS FLAGS

--device-ids=MODEL_ID,[MODEL_ID,...], -d MODEL_ID,[MODEL_ID,...]

The list of MODEL_IDs to test against (default: one device model determined by the Firebase Test Lab device catalog; see TAGS listed by the $ gcloud beta firebase test android devices list command).

--locales=LOCALE,[LOCALE,...], -l LOCALE,[LOCALE,...]

The list of LOCALEs to test against (default: a single locale determined by the Firebase Test Lab device catalog).

--orientations=ORIENTATION,[ORIENTATION], -o ORIENTATION,[ORIENTATION]

The device orientation(s) to test against (default: portrait). Specifying 'default' will pick the preferred orientation for the app. ORIENTATION must be one of: portrait, landscape, default.

--os-version-ids=OS_VERSION_ID,[...], -v OS_VERSION_ID,[...]

The list of OS_VERSION_IDs to test against (default: a version ID determined by the Firebase Test Lab device catalog).

LIST COMMAND FLAGS

--filter=EXPRESSION

Apply a Boolean filter EXPRESSION to each resource item to be listed. If the expression evaluates True, then that item is listed. For more details and examples of filter expressions, run $ gcloud topic filters. This flag interacts with other flags that are applied in this order: --flatten, --sort-by, --filter, --limit.

--limit=LIMIT

Maximum number of resources to list. The default is unlimited. This flag interacts with other flags that are applied in this order: --flatten, --sort-by, --filter, --limit.

--page-size=PAGE_SIZE

Some services group resource list output into pages. This flag specifies the maximum number of resources per page. The default is determined by the service if it supports paging, otherwise it is unlimited (no paging). Paging may be applied before or after --filter and --limit depending on the service.

--sort-by=[FIELD,...]

Comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with ``~'' for descending order on that field. This flag interacts with other flags that are applied in this order: --flatten, --sort-by, --filter, --limit.

GCLOUD WIDE FLAGS

EXAMPLES

$ gcloud beta firebase test android run --app APP_APK --timeout 100s

$ gcloud beta firebase test android run --app APP_APK \
    --device model=NexusLowRes,orientation=landscape

$ gcloud beta firebase test android run --app APP_APK \
    --test TEST_APK --device model=shamu,version=21,locale=fr

$ gcloud beta firebase test android run --app APP_APK \
    --test TEST_APK --device model=Nexus4,version=19 \
    --device model=Nexus4,version=21 \
    --device model=NexusLowRes,version=25

$ gcloud beta firebase test android run --app bundle.aab

$ gcloud beta firebase test android run --app APP_APK --timeout 5m \
    --device-ids=shamu,NexusLowRes,Nexus5,g3,zeroflte \
    --os-version-ids=19,21,22,23,24,25 --locales=en_GB,es,fr,ru,zh \
    --orientations=portrait,landscape

$ gcloud beta firebase test android run --app APP_APK \
    --results-bucket=gs://my-bucket \
    --results-dir=my/test/results/<unique-value>

$ gcloud beta firebase test android run --app APP_APK \
    --test TEST_APK \
    --results-history-name='Excelsior App Test History'

$ gcloud beta firebase test android run \
    path/to/excelsior_args:robo-args

NOTES

$ gcloud firebase test android run $ gcloud alpha firebase test android run