More of a good thing: Analyze and Access M-Lab data using Google BigQuery and Google Storage
We are pleased to announce that Google, an M-Lab partner, will be hosting M-Lab datasets (currently featuring tens of terabytes, and counting, from the NDT, NPAD, and Glasnost tools) on Google Storage for Developers. This service is currently accessible on an invitation-only basis (you can request an account here). Google (along with Amazon Web Services) has made the commitment to make this data available to all of you. (Note that some the data currently in Google Storage for Developers isn’t yet available through Amazon. We’re working to make this happen, and we’ll update you when it does.)
In addition to downloading the M-Lab datasets on Google Storage, it will be possible to analyze the data using Google BigQuery (more information here and here). BigQuery enables you to interactively analyze massive datasets using a SQL-like query language. Like Google Storage for Developers, BigQuery is in preview stage (you can sign up here), and is being made available to a limited number of people before the open public roll-out.
In the meantime, a number of researchers have been granted preview access, and will be working with BigQuery to extract and share meaningful and currently unexamined information in M-Lab’s raw data set. It is these potential revelations that underscore the goal of M-Lab’s open data efforts, and we will certainly announce any new stories that the data and analysis may tell us.
M-Lab is a community-based collaborative effort, and we are extremely grateful to partners like Google and Amazon for providing the infrastructure to make M-Lab data available to the public. We are always looking for others who would like to support our goal of providing better data on broadband performance measurement to users, researchers and policymakers. If you would like to get involved with M-Lab by providing additional data repositories, analyzing the collected data, deploying tools, developing applications or providing servers, please let us know.
