Mn Python tests¶
Marionette is the codename of a remote protocol built in to Firefox as well as the name of a functional test framework for automating user interface tests.
The in-tree test framework supports tests written in Python, using
Python’s unittest library. Test cases are written as a subclass
of MarionetteTestCase
, with child tests belonging to instance
methods that have a name starting with test_
.
You can additionally define setUp
and tearDown
instance
methods to execute code before and after child tests, and
setUpClass
/tearDownClass
for the parent test. When you use
these, it is important to remember calling the MarionetteTestCase
superclass’ own setUp
/tearDown
methods since they handle
setup/cleanup of the session.
The test structure is illustrated here:
from marionette_harness import MarionetteTestCase
class TestSomething(MarionetteTestCase):
def setUp(self):
# code to execute before any tests are run
MarionetteTestCase.setUp(self)
def test_foo(self):
# run test for 'foo'
def test_bar(self):
# run test for 'bar'
def tearDown(self):
# code to execute after all tests are run
MarionetteTestCase.tearDown(self)
Test assertions¶
Assertions are provided courtesy of unittest. For example:
from marionette_harness import MarionetteTestCase
class TestSomething(MarionetteTestCase):
def test_foo(self):
self.assertEqual(9, 3 * 3, '3 x 3 should be 9')
self.assertTrue(type(2) == int, '2 should be an integer')
The API¶
The full API documentation is found here, but the key objects are:
MarionetteTestCase
: a subclass forunittest.TestCase
used as a base class for all tests to run.Marionette
: client that speaks to Firefox
Registering Test Manifests¶
To run Marionette Python tests locally via mach
or as part of the Mn
tests jobs
in CI they need to be registered. This happens by adding a manifest file to the tree,
which includes a reference to the test files and expectations for results.
Such a manifest file can look like the following and is stored with the extension .toml
:
[DEFAULT]
["test_expected_fail.py"]
expected = "fail"
["test_not_on_windows.py"]
skip-if = ["os == 'win'"]
The registration of such a manifest file is done in two different ways:
To run the tests locally via
./mach test
or./mach marionette-test
the created Marionette manifest file needs to be referenced in the folder’s relatedmoz.build
file by adding it to theMARIONETTE_MANIFESTS
variable like:MARIONETTE_MANIFESTS += [“test/marionette/manifest.toml”]
To run the tests in CI the manifest file also needs to be included in the Marionette’s own master manifest file. This ensures that the test packaging step will find the tests and include them as well in the test package.