Switch discard investigation for early 2018

On February 1st, 2018, during a regular data quality review, we identified an increase in switch discards at sites with 10Gbps equipment connected to 1Gbps uplinks. We used our switch telemetry data to assess whether there were any negative consequences for tests contained in our SideStream or NDT data sets, and then we used the same data sets to determine whether our remediation strategy had any negative effects. In both cases, we found no observable effects, indicating that everything was below the noise floor for Internet performance data.

Read More

M-Lab is Turning 10!

M-Lab 10 Year Anniversary Graphic

  • When: Aug. 7, 2018 - Aug. 8, 2018, 9AM - 5PM
  • Where: New America, 740 15th St NW #900, Washington, D.C. 20005

Measurement Lab is turning 10! On August 7 and 8, we look forward to gathering the Measurement Lab community to showcase how the platform has evolved, learn from you about how you are using M-Lab, and discuss how we plan for the next 10 years of measuring the Internet and providing public data to the world. So much has changed over the last 10 years (and that’s not just our expanding volume of longitudinal data!), come celebrate, brainstorm, analyze, and share with us.

Read More

New ETL Pipeline and Transition to New BigQuery Tables

Since May 2017, the M-Lab team has been working on an updated, open source pipeline, which pulls raw data from our servers, saves it to Google Cloud Storage, and then parses it into our BigQuery tables. The team is particularly excited about this update because it means that the pipeline no longer relies on closed source libraries.

Read More

Transitioning to a New Backend Pipeline and Data Availability

M-Lab data is collected from distributed experiments hosted on servers all over the world, processed in a pipeline, and published for free in both raw and parsed (structured) formats. The back end processing component for this has served us well for many years, but it’s been showing its age recently. As M-Lab collects an increasing amount of data thanks to new partnerships, we have been concerned that it will not be as reliable.

Read More

Making it Easier to Use M-Lab Data

In January, M-Lab launched a beta test of new BigQuery tables for M-Lab data. Today, M-Lab is pleased to announce that the beta test was successful. The new, faster-performing tables will be M-Lab’s new standard BigQuery tables.

Before we move on to specifics, when we say faster performing, we mean a lot faster. As in, certain queries that used to take over 2 hours now complete in 8 seconds. That means that playing with the data just became a lot more fun.

To help users dig in to this data as quickly and seamlessly as possible, M-Lab has consolidated all of its data documentation and updated it to show how to take advantage of the new tables.

Read More

Announcing improved performance for M-Lab BigQuery data

Today, M-Lab is happy to announce the public beta of new M-Lab BigQuery tables. These tables provide substantially improved performance and reduce the difficulty of writing BigQuery SQL.

Read More

Back to Top