Utilization BigQuery Schema

Since June 2016, M-Lab has collected high resolution switch telemetry for each M-Lab server and site uplink and published it as the DIScard COllection (a.k.a. DISCO) dataset. We publish this data in the utilization dataset in the switch BigQuery View(s).

Switch BigQuery Views

Switch Schema - measurement-lab.utilization.switch

Switch - measurement-lab.base_tables.switch

The switch table schema (also known as “DISCO”, named after the “DIScard COllection” service that records the switch data).

Field name Type Description
partition_date date The day (in UTC) in which the data was loaded.
test_id string ID of the test. It represents the Google Cloud Storage path, M-Lab server, and file name of log that contains the data generated during the test (e.g. 2019/02/09/mlab1.yul02.measurement-lab.org/20190209T23:00:00-to-20190210T00:00:00-switch.json.gz).
task_filename string The raw data file in Google Cloud Storage from which the test row was parsed.
parse_time timestamp Timestamp of when test data was parsed into BigQuery from Google Cloud Storage.
parser_version string A link to the tagged version of the M-Lab ETL parser which processed the test row.
log_time timestamp Never set for the switch data. The sample.timestamp should be used instead for the sample collection time.
sample record A repeated record with the value and timestamp of each 10 second observation. Typically, there will be 360 samples per hour. Due to system maintenance, or machine restarts, some intervals may contain more or less samples.
sample.timestamp timestamp Timestamp of the beginning of the 10 second time bin.
sample.value float Delta value of the metric during this 10 second time bin.
metric string The canonical metric name for samples, e.g. switch.discards.uplink.tx
hostname string The fully qualified domain name of the machine that collected the data, e.g. mlab2.abc01.measurement-lab.org.
experiment string The fully qualified domain name of the switch that produced the data, e.g. s1.abc01.measurement-lab.org.
Back to Top