Measurement data from many experiments hosted on M-Lab are processed via the ETL pipeline and published in two forms:
- Google Cloud Storage
- M-Lab publishes raw output from many measurement tests on Google Cloud Storage as file archives.
- See M-Lab Google Cloud Storage documentation for more information.
- Google BigQuery
- M-Lab parses data for a subset of tests and publishes the data on BigQuery so that users can run SQL queries on the data.
- See M-Lab BigQuery QuickStart for more information.
Some M-Lab hosted tests do not use our ETL pipeline. Data for these tests are published independently by the test developers.
There is typically at least a 24-hour delay between data collection and data publication. Below we provide links to data for our Active tests and archival data from Inactive tests.
M-Lab also publishes public data sets about the M-Lab Platform, listed below.
Measurement Data (Active Tests)
- Paris Traceroute
- Reverse Traceroute
- Reverse traceroute measures the network path back to a user from selected network endpoints, and provides a rich source of information on network routing and topology. Reverse Traceroute data is not processed by the M-Lab ETL Pipeline.
- More information is available at Reverse Traceroute
- Reverse Traceroute Raw Data
- The SamKnows performance testing platform is used by the USA’s Federal Communications Commission (FCC), European Commission, UK government (Ofcom), Brazilian government (Anatel), Singapore’s IDA and other government-backed studies worldwide.
- SamKnows infrastructure includes off-net test servers hosted by M-Lab, and the M-Lab and SamKnows teams coordinate regularly to support the various regulatory reporting periods of data collection conducted by SamKnows.
- More information is available at the SamKnows website
Measurement Data (Platform Data)
- M-Lab Collectd
- M-Lab DISCO Switch Telemetry Data
- Since June 2016, M-Lab has collected high resolution switch telemetry for each M-Lab server and site uplink and published it as the DIScard COllection (a.k.a. DISCO) dataset.
- More information is available in the blog post announcing this dataset provides more information about the DISCO dataset.
- M-Lab DISCO Raw Data - M-Lab DISCO BigQuery Dataset
Historical Data Sets (e.g. Retired Tests)
- ShaperProbe detected prioritization of network traffic.
- Shaperprobe Raw Data (archived)
- WindRider attempted to detect whether your mobile provider was performing application- or service-specific differentiation.
- More information is available at Windrider.
Data License and Citing M-Lab Data
All data collected by M-Lab tests are available to the public without restriction under a No Rights Reserved Creative Commons Zero Waiver.
Please cite M-Lab data sets as follows:
The M-Lab test name Data Set, date range used. M-Lab test URL
The M-Lab NDT Data Set 2009-02-11–2015-12-21. https://measurementlab.net/tests/ndt