I hope this finds you well. Today I’m writing with the first of many newsletters that you’ll receive on this mailing list. In each monthly edition, we’ll provide updates about M-Lab such as changes to our platform and pipeline, data schemas, community tools, best practice analyses, community case studies, documentation and guides, and more. Ultimately, our goal is to communicate with you on a more regular basis, so we encourage you to reply with any questions or comments that come to mind. Additionally, as part of our efforts to communicate more effectively, we’ve established a lightweight set of community guidelines that we ask participants to keep in mind as you engage here and on any other M-Lab hosted forum. If there is anything you’d like to see in these newsletters, please do let me know. And without further ado!
In 2021, Measurement Lab will facilitate a series of community calls focusing on 3 core topics: General M-Lab updates, Internet Measurement Research and Broadband Advocacy/Policy. We will rotate through these topics on the fourth Wednesday of every month and every meeting will take place at 11am EST for 60-90 mins.
In response to increased reliance on the Internet during the COVID-19 pandemic Measurement Lab, with the support of Internet Society, is dedicated to making our NDT data more accessible so everyone with an interest in Internet Performance can use our data in their research. We started with a visualization of test count and median download speeds in New York, India, and Sao Paulo as an example of what data was available to researchers through BigQuery.
Measurement Lab (M-Lab) has been a supportive partner of the FCC’s Measuring Broadband America program since its beginning due to the program’s commitment to openness, open data and transparency which aligns closely with M-Lab’s mission around open data and open source internet measurement. Measurement Lab began hosting the SamKnows server-side measurement tools in 2009.
As of February 14, 2020, Measurement Lab is unable to host the SamKnows experiment due to SamKnows’ decision to not comply with M-Lab’s long-standing open source requirements. To ensure and encourage transparency for the MBA program, we are publishing this post to document the timeline leading up to that decision.
Measurement Lab’s dataset is free, open, and full of data for Internet researchers to study how the Internet is performing under the increased COVID-19 related Internet usage. The DataStudio dashboards below correspond to the internal monitoring metrics we posted last week. They show data about test rate and median download speeds as COVID-19 increased its impact globally.
As COVID-19’s impact increases globally, more people are working, learning and living online. Internet infrastructure is critical in these moments. The user-initiated measurements collected by M-Lab provide an opportunity to understand whether our shared infrastructure can handle the unprecedented shift of load to the network’s edges. The following is a snapshot analysis of how our platform’s traffic has shifted in the last three weeks. We will follow up with DataStudio dashboards using our public data set soon.
As these dashboards will show, the benefit of open data is that anyone can access it, use it, and reproduce research done with it. If you are using or would like to use M-Lab data in your response to the Internet’s performance during COVID-19, please let us know!
As COVID-19’s impact increases globally, more people are working, learning and living online. In partnership with NDIA, X-Lab, ILSR and Marconi Society, Measurement Lab has set up a speed test portal to collect geolocated data with an accompanying survey focused on Digital Inclusion @ Home.
M-Lab has always committed to provide traceroute data for the tests running on M-Lab platform. Since we launched new platform in 2019 Q4, the new traceroute binary, Scamper, replaced Paris Traceroute on all our servers. Our BigQuery table for traceroute now has unified schema to cover both the legacy Paris Traceroute test data and new Scamper data. This blog is to analyze the difference between the legacy data and new data.
After years of planning and steady development, the new M-Lab 2.0 platform has landed. We want to express special thanks to those who have supported the project and helped us get here.
Last week M-Lab was honored to attend and contribute at the 2019 Indigenous Connectivity Summit (ICS), presented by Internet Society, in Hilo, Hawai’i; and a community network training and building event in the soverign lands of indigenous Hawaiian people, Pu’uhonua o Waimanalo, on the island of Oahu.
The first annual Michigan Broadband Summit was held on Sept. 24, 2019, sponsored by the MERIT research and education network, and it was my pleasure to attend on behalf of M-Lab. The gathering was an opportunity to learn more about the amazing work that MERIT, industry, municipalities, cooperatives, school districts and others are doing to improve internet access and service quality in Michigan. I came away inspired by every conversation and presentation, with the keynote from Francella Ochillo from Next Century Cities, and the MERIT team’s presentation of the Moonshot framework being particularly moving.
To make our traceroute data in BigQuery more useful, researchers have sought an easy way to reconstruct the path of hops for the same test. This task was particularly hard because the schema, which was designed many years ago, put the hops of the same test in different rows.
To address this need from many of our partners and researchers, M-Lab is delighted to announce that the
traceroute BigQuery table in the
aggregate dataset is now available to the public. The new
traceroute schema has one test per row, and all hops for a single test are inside the same row.
M-Lab was pleased to be invited to contribute to the Internet Measurements workshop at the 2019 African Internet Summit, June 15-16, 2019 in Kampala, Uganda. M-Lab tech lead, Peter Boothe, and advisor, Georgia Bullen, presented a hands-on tutorial on querying and visualizing performance and routing datasets.
M-Lab is working on replacing the current traceroute BigQuery table with new schema, which will put all hops of one test in one row of BigQuery table. The new table will have all the information in the current table but make the search of hops within one test much easier. To make this happen, we will stop the new data feed of current traceroute BigQuery table in early July, 2019. The details of new schema will be published once the conversion of all data to BigQuery tables with the new traceroute schema is completed and available to the public.
M-Lab is excited to highlight the launch of “Michigan Moonshot”, a pilot data collection project developed through a partnership with the MERIT Network, Michigan State University’s Quello Center, and M-Lab. We worked with MERIT developers who adapted the Piecewise codebase to build the Michigan Moonshot website.
Over the past few years, M-Lab has supported an increasing number of local communities, municipalities, regional coalitions, and others who are interested in understanding the state of broadband service in their area. In every case, communities come to M-Lab because of the openness of our data. Combined with M-Lab’s open source code and community examples, we’ve been able to further support communities in building and running their own broadband measurement test sites and broadband community surveys, to support local outreach, advocacy, and planning.
Hello M-Lab Community!
TLDR; Interested in working with M-Lab? We are hiring a Project Director and have opportunities for contracts! Check out the job description and more information about the contract, and apply by March 31st!
AFRICOMM 2018. Left to right: ￼Amreesh Phokeer (AFRINIC)￼, Josiah Chavula (University of Capetown), Georgia Bullen (M-Lab), Antoine Delvaux (perfSonar), Stephen Soltesz (M-Lab).
In late November 2018, M-Lab was invited to the Internet Measurement Workshop at AFRINIC-29 in Tunisia and to give a keynote about M-Lab and open internet measurement at AFRICOMM 2018 in Senegal. Both trips were a fantastic opportunity to deepen our relationship with researchers focused on the African Internet, learn more about how our platform is serving community needs, foster conversation around open Internet measurement, and identify opportunities for further collaboration, research and tool development to better support the Internet measurement, research and policy community in Africa.
After A Decade of Growth, Measurement Lab Spins Out of New America to Join Code for Science & Society
Code for Science & Society (CS&S) and New America’s Open Technology Institute are excited to announce that Measurement Lab (M-Lab) is joining CS&S’s Sponsored Projects Program on March 1 after a decade of growth at New America. Measurement Lab has been working in the public interest to measure Internet performance around the world and share data openly since it was launched at New America in 2008, work that will continue and expand at its new institutional home. Read on for more about M-Lab’s history and future plans.
M-Lab had the pleasure of attending the first ever SIGCOMM hackathon on August 25, 2018, at the Nokia Skypark headquarters in Budapest, Hungary. The hackathon, sponsored by Nokia, DECIX, and Netflix, invited network research faculty, students, and industry professionals from around the world to form teams and develop tools, new features or analyses during the Saturday following the SIGCOMM conference.
We’ve reached a point in human history where, for many of us, the Internet has become a standard presence in our daily lives. In the United States, the Internet is simply part of how many of us engage with the world. In other countries (and parts of this one), the Internet remains unaffordable, unreliable, and inaccessible. The Internet unites us in many ways, and at the center of work on the future of the Internet is a dedicated community of experts exploring the questions that will move the Internet to the next level of its evolution: What is an open Internet? What is a healthy Internet? What factors contribute to the Internet ecosystem’s health?
- When: Saturday, August 25, 2018
- Where: SIGCOMM, Budapest, Hungary
- When: Aug. 7, 2018 - Aug. 8, 2018, 9AM - 5PM
- Where: New America, 740 15th St NW #900, Washington, D.C. 20005
Measurement Lab is turning 10! On August 7 and 8, we look forward to gathering the Measurement Lab community to showcase how the platform has evolved, learn from you about how you are using M-Lab, and discuss how we plan for the next 10 years of measuring the Internet and providing public data to the world. So much has changed over the last 10 years (and that’s not just our expanding volume of longitudinal data!), come celebrate, brainstorm, analyze, and share with us.