Version 1.7 a new dawn

lecture: The Matter of Heartbleed

What went wrong, how the Internet reacted, what we can learn for the future

Heartbleed

The Heartbleed vulnerability took the Internet by surprise in April of this year. The vulnerability was one of the most consequential in the history of the Internet, since it allowed attackers to potentially steal login credentials, cryptographic keys, and other private data from up to half of all popular HTTPS sites. In this talk, we take a detailed look at Heartbleed and its aftermath, based on comprehensive measurements and analysis that our research team performed over the past six months. We began tracking Heartbleed's impact within hours of its disclosure using massive ZMap scans and large network telescopes. This allowed us to track which sites remained vulnerable, observe certificate revocations, and monitor for large scale attacks in close to real time. Based on this data, we also conducted one of the largest ever mass vulnerability notifications, informing the network administrators for all devices still susceptible to Heartbleed across the entire IPv4 address space. Finally, we investigated the question of whether attackers knew about and exploited Heartbleed prior to its public disclosure---and we will present new details about this question in the talk. We hope that by learning from the Heartbleed security disaster, our community can prepare to respond more effectively to such events in the future.

In March 2014, researchers found a catastrophic vulnerability in OpenSSL, the cryptographic library used to secure connections in popular servers including Apache and Nginx. The bug allowed attackers to extract cryptographic keys, login credentials, and other private data from an estimated 22-55% of HTTPS sites. Worsening its severity, the bug was both simple to understand and exploit.

We used ZMap to perform comprehensive scans of the IPv4 address space and popular web servers in the days and months following disclosure. We provide more extensive estimates on who was originally vulnerable, track who patched their sites, and replaced certificates. We will present exactly which server products and devices were vulnerable. We will further discuss how Heartbleed affected the HTTPS CA ecosystem. Worryingly, we find that only 10% of the known vulnerable sites replaced their certificates within the next month, and of those that did, 14% neglected to change the private key, gaining no protection from certificate replacement! We'll also present the shortcomings in the public key infrastructure that Heartbleed unearthed and problems our community needs to focus on moving forward.

We investigated widespread attempts to exploit Heartbleed post disclosure at four network sites. We will discuss the subsequent exploit attempts we observed from almost 700 sources and the Internet-wide scans that started post disclosure. We also investigated whether exploit attempts took place prior to Heartbleed's public disclosure, including examining suspicious network traces recorded months earlier. We will disclose new details of these traces and their implications in the talk.

Even with global publicity, Heartbleed patching plateaued after two weeks. To try to help, we notified network administrators responsible for more than 500,000 unpatched systems. While much of the security community (including us!) assumed that mass vulnerability notifications would be too difficult or ineffective, we found that it increased the Heartbleed patching rate by nearly 50%. We will discuss how we performed these notifications, the reactions of network operators, and prospects for performing automatic mass notifications based on Internet-wide scanning in future vulnerability events.

Throughout the talk, we will use real world data to frame what went well and what went poorly in the Internet's response to Heartbleed. The vulnerability's severe risks, widespread impact, and costly global cleanup qualify it as a security disaster. However, by understanding what went wrong and learning from it, the Internet security community can be better prepared to address major security failures in the future.


Archived page - Impressum/Datenschutz