Measurement Lab Launches New Global Data Visualization Tools
The Measurement Lab (M-Lab) global measurement platform contains the largest collection of open internet performance data on the planet. Hundreds of thousands of people (and rising everyday!) test their internet connections on M-Lab and provide that data to the public domain for research, analysis and advocacy. As a consortium of research, industry, and public-interest partners, M-Lab is dedicated to providing an ecosystem for the open, verifiable measurement of global network performance. With an extensive library of data dating back to 2009, M-Lab has over 5 petabytes of information about quality of experience on the internet contributed by people like you.
BBR TCP and Measurement Lab
What is TCP?
Since the beginning of the its existence, the internet has expanded in scope, traffic, content, and a myriad of other ways. The protocols that make up the internet’s backbone have mostly remained the same since they were developed in the 1980s. The Transmission Control Protocol (TCP) was one of the first networking protocols defined during the internet’s development, and specifies how data should be transmitted and received. TCP implementations, initially developed in the 1980’s, attempted to discover the right rate at which to send data by constantly trying to send more until reaching the point that not all of the data arrived at its destination, and then backing off on the amount being sent.
Showing Missing Data in Line Charts
The M-Lab team has been working with Bocoup’s Data Visualization team to overhaul our visualizations and give all of you better support in exploring all of the Measurement Lab data. Look for more about that soon – and reach out via support@measurementlab.net if you are interested in helping with testing and user feedback!
M-Lab has lots of data, but depending on how you slice the data, you might end up with too small of a sample size on a given day in a given location. As part of designing the Observatory visualization, we came up with the idea of using dotted lines to show that we didn’t have a large enough sample size to assert the data value, but that leaving out the data would be incorrect as well. The Bocoup Team took that further and developed a new d3 plugin, d3-line-chunked, which allows you to easily visualize gaps in your data and has good animation support.
Traffic Microbursts and their Effect on Internet Measurement
In August 2015, M-Lab was notified of potential degradation of site performance by a measurement partner based on discrepancies compared to results for their own servers. After a full investigation these patterns were found to have been caused by the unique confluence of several specific conditions. Interim remediation measures were taken in early October 2015, and the resolution of the degradation was confirmed by the partner and others. Due to these administrative actions, the episode, which we are calling the “switch discard issue,” has not affected testing conducted in the United States (the region impacted by this problem) since October 11, 2015, and thus measurements after this period are not affected by the incident. M-Lab has also conducted an evaluation of data collected during the time period in which the issue occurred, and has taken steps to remove affected measurements from its dataset. This incident will not affect use of its dataset, past or present, as a result.
Making it Easier to Use M-Lab Data
In January, M-Lab launched a beta test of new BigQuery tables for M-Lab data. Today, M-Lab is pleased to announce that the beta test was successful. The new, faster-performing tables will be M-Lab’s new standard BigQuery tables.
Before we move on to specifics, when we say faster performing, we mean a lot faster. As in, certain queries that used to take over 2 hours now complete in 8 seconds. That means that playing with the data just became a lot more fun.
To help users dig in to this data as quickly and seamlessly as possible, M-Lab has consolidated all of its data documentation and updated it to show how to take advantage of the new tables.