Copyright © 2008-2009 W3C® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use rules apply.
As support for Web technologies grows, it is important that tests writers develop test suites that will work as well as possible across devices. This document offers guidance in the form of simple guidelines to follow to create device-independent tests.
These guidelines were produced by members of the Mobile Web Test Suite Working Group which is part of the Mobile Web Initiative (see About).
Comments on, and discussions of this document can be sent on the (archived) public mailing list public-mwts@w3.org (see instructions). W3C Members can also send comments directly to the Mobile Web Test Suites working group.
These guidelines represent the current thinking of the working group and as such may be updated, replaced or rendered obsolete by other W3C documents at any time. Its publication does not imply endorsement by the W3C membership or the Mobile Web Test Working Group.
Patent disclosures relevant to the Mobile Web Test Suites Working Group may be found on the Working Group's public patent disclosure page.
This document provide a set of guidelines for writing test cases that can be run effectively across devices, in particular on mobile devices.
The Authoring Challenges for Device Independence [ACDI] explore these different limitations in the general context of writing device independent content, and the Mobile Web Best Practices 1.0 [MWBP] give specific guidance on writing content targeted at mobile devices.
The CSS2.1 Test Case Authoring Guidelines [CSSTCAG] provides guidance on how to write test cases, and sets device-independence as a goal. The SVG Test Suite Manual [SVGTS] also offers advices on writing test cases.
Inspired by this existing work, and based on the experience gathered by the Mobile Web Test Suites Working Group while reviewing test cases and their fitness to mobile devices [MWTSSURVEY], this document explores the specific aspects to take into account when writing test cases to ensure greater device independence.
Consider recording the browser identifier with the widely implemented
window.navigator.userAgent, as there might be several browsers
on one device.
When designing device-independent test cases, it is important to acknowledge the limitations of most devices:
For tests that require interaction (either for running the test or for submitting results), consider:
<body onload="beginTest();">Screen size matters when designing visual test cases - e.g. where the tester needs to assess whether the rendering of a test cases matches a reference rendering.
Across devices, the following screen parameters vary widely:
In general, to avoid problems when running test across devices:
Bad test:
Good test:
To avoid hitting memory limitations of the devices on which the technology under test run:
Less is more.
The characteristics of network access across devices vary greatly, in particular in terms of bandwidth available, the latency induced by network requests, and the cost of transferring content over the network.
To cater for test environments where network is slow or costly:
CPU-intensive operations on mobile devices can drain the battery, and are likely to be much slower than on larger hardware. As a result:
Most mobile devices have limited capabilities when it comes to plugins, additional fonts, or software extensions in general. Therefore do not rely on non-standard features such as specific fonts to be installed.
Many mobile devices don't provide a full keyboard, and thus require several key presses to enter a given character.