We created this mobile connection test to gather heretofore unavailable information about the behavior of web browsers for mobile devices. The test is constructed to capture the number of concurrent HTTP requests generated by browsers (or in some cases, the HTTP requests generated by the network provider's proxies that sit between the mobile device and the web server).
Our interest in concurrent requests comes from a desire to build web services with improved usability for mobile devices. Concurrent requests generally improve page rendering speed, and speed is highly correlated to the perception of web usability. We could not find this information available elsewhere, so we decided to construct a test of our own.
We are interested in capturing both the total number of concurrent connections and the number of per-domain connections when using multiple domains. We want to do this programmatically on the server side of the connection, and we want to do this with minimal interaction by users generous enough to spend their time taking the test.
To test concurrent HTTP requests across multiple domains, we have created four separate subdomains to host images. Each of these domains lives on the same physical server as the main test page, and each domain contains the server-side scripting code necessary to generate the images. The concurrency test measures the ability of a device to simultaneously fetch images from one or more domains.
When the mobile device connects to the test page (test.php) the server completes the following steps:
After several experiments with sample devices, we set our overall test time to fifteen seconds. Our empirical data suggest that this is sufficient time for most devices to issue the initial set of image requests.
After receiving the test page HTML, the mobile device must begin fetching the 64 unique images from the image servers. This is where the image generation code comes into play.
Each image is rendered by a server-side script (spacer.php) that is uniquely parameterized for the specific device test, the image domain, and the image requested. The image script does the following steps:
The effect of the above sequence is to delay all image delivery until the timeout period elapses. While the timeout period is open, the server accepts and records connection requests for images from each of the four domains. Our assumption, since verified via log inspection, is that devices will issue as many requests as they can at any one time, and that a reasonable server environment will be able to respond in kind.
Because the test relies upon "reasonably good" behavior by everyone involved, the results are subject to some variability. It will be necessary to collect a large sample size and check for test-induced outliers.
Some factors that could influence test behavior and corresponding results:
Preliminary test results show that it will be necessary to review the web server access logs to validate the auto-calculated connection counts. During our pre-release testing, we have already observed several potential issues:
With these test setup caveats in mind, we have decided to proceed with the test as currently constructed. Our initial survey of mobile devices has yielded interesting results and our inspection of the web server access logs leads us to believe these results are being correctly recorded. We will continue to monitor test data as the testing period proceeds and make adjustments as necessary.
And, of course, we'll publish everything we find, anomalies and all.
Concurrency test developed and provided by Cloud Four
Last updated: April 27, 2008