Five Things You Didn't Know About Data Testing
At 10:30 in the morning a few weeks ago I hopped into a plain white van, a mysterious briefcase carefully laid out on the passenger's seat abuzz with phones.
Into the afternoon, the driver answered my probing questions about long hours on the road, awkward run-ins with the cops, and flashed guns. This was no police surveillance ride along; my driver was merely sharing a day in the life of an independent cell phone network tester.
Meet RootMetrics A few miles away from Microsoft, a small company called RootMetrics (formerly Root Wireless) sends out a tiny squadron of testers -- called scouts -- to independently test carrier networks, and they do it using little more than a Garmin GPS unit and a briefcase full of phones.
Inside the briefcase, one phone per carrier runs through the battery of tests. Every 5 minutes, the phone tracks calls, upload and download speeds, and texting. After the weeklong-testing period (or longer), RootMetrics crunches the data. Twice a year, the company compiles a larger report on the health of each network around the country.
Together, the scouts who form the backbone of the operation clock 20,000 hours of testing and calculate information from 100,000,000 data points in a single year. The bottom line: network testing is serious stuff. Here are five things you probably didn't know about the voice and data testing that goes on in cities across America, maybe even your city, maybe even right now.
Editors' note: CNET partners with RootMetrics to provide coverage maps on our mobile reviews site.
1. The Gettysburg Address plays a role RootMetrics hasn't been around for four score and seven years, but President Abraham Lincoln's iconic speech is a bedrock element for RootMetrics' network tests, albeit indirectly.
Inside the scout's black portfolios are six or seven phones, each one fixed with Velcro and powered by a large, all-day battery. Each phone runs through the series of tests: outbound calls first, followed by inbound calls and then photo, video, and music uploads. Next come the downloads and finally the text message. A master phone connected through Bluetooth to the other five or six runs the whole batch.
Tom Ingarson, RootMetrics' field operations manager, holds up a typical testing kit outfitted with a master phone and five test phones. (Credit: Jessica Dolcourt/CNET)
Now back to Lincoln. RootMetrics uses Twilio, a San Francisco-based company that lets developers integrate voice and text to their apps, as the server that these test phones ping. After Twilio's server answers the phone call, it plays back the Gettysburg Address for a total of 1 minute and 40 seconds.
The entire testing cycle is over in 4 minutes and starts fresh every 5 minutes, which means that within the testing briefcase of five phones, the Gettysburg Address plays at least five times every 5 minutes, 60 times every hour, and 480 times every 8-hour shift.
2. Android forms the backbone I mentioned that a RootMetrics scout carries a total of six or seven phones in the briefcase. These aren't just any phones. Each of the company's 12 portfolios are identical, and RootMetrics chooses the testing handsets with care.
Right now the devices are all Android phones, because the app runs on Google's mobile operating system. RootMetrics recently performed a sample test on the iPhone, but it required five scouts in one car to manually run through the test every 5 minutes -- one scout per network in this outlier case.
In San Francisco, RootMetrics tests AT&T, MetroPCS, Sprint, T-Mobile, and Verizon, but in other markets, the company will add or replace carriers based on the provider's strength in a given market. For example, scouts might test Cricket in Tucson and San Diego in addition to the standard Big Four carriers, and add Virgin Mobile and U.S. Cellular in Chicago and Milwaukee.
(Credit: Jessica Dolcourt/CNET)
In contrast, New York is a 35-day market, requiring five scouts scouring the greater metropolitan area around the clock. The smallest markets warrant five days of testing. During even a five-day scouting session, each hour of the clock is covered at least once, with an emphasis on the daylight hours that draw the majority of call, texts, and data activity. (London, which RootMetrics plans to test, is a 50-day market, and collecting data in Berlin will take a whopping 80 days.)
For instance, a scout's shift could run from 9:00 a.m. to 5:00 p.m. one day, and from 10:00 p.m. to 6:00 a.m. the next. In larger markets like New York, the smaller hours of the morning would see proportionally more testing time than in San Francisco, but fewer than the peak times of calling, testing, and data activity.
RootMetrics keeps its scouts busy. They cover 75 U.S. cities in the first half of the year, and tackle 78 cities in the second half. "They're our front line in data collection," Senior Marketing Manager Doehne said.
After the day is done, scouts power down devices and recharge them, then upload notes on the drive -- how was traffic, what was the weather like, and did stores seem crowded? Along the route, they also snap photos inside and outside of an indoor location, and log the GPS coordinates of each stop. At the end of the shift, scouts also upload the GPS tracks to the company database.
5. It can get awkward Indoors, scouts stay out of the way as much as they can, but with the testing kit in tow, there are only so many places to hide.
"It can get a little awkward walking around a Safeway at 3 a.m. with a portfolio," RootMetrics' field operations manager Tom Ingarson said.
Furthermore, scouts are instructed to skedaddle or scrap the indoor plan if the environs seem unsafe. When prodded about any danger, Ingarson told me he once had a gun flashed at him in Kansas City, then quickly backed off the topic. After all, he doesn't want to make his work seem dangerous or unsavory.
(Credit: Jessica Dolcourt/CNET)