Doing IT operations during these heartbleeding times isn’t easy. If you’ve been following even remotely the Heartbleed saga, you know it’s a critial design flaw in the most popular implementation of the most popular encryption protocol, OpenSSL. To be plain, most of the credentials you held in the recent past are potentially compromised, since even if eveyone are now updated and protected, your credentials might have been stolen beforehand.
If you indeed do operations, you know it gets painful logging into another web console of some service provider to see a warning banner with the latest security advisory about Heartbleed. It usually goes in the same spirit of “please reset all your passwords, please recycle all your keys, please reissue all your certificates”. And the more responsible of us do: We reset all our passwords, we annihilate the deployment keys we used for the last year, and certificates older than the buildings we work in are being reissued. We update and patch all our servers, and we have a lot of servers. We mass-email users to change their login credentials, and go on a frenzy of forcing everyone to enable 2-factor authentication and security questions. And we’re probably just getting started.
We press hard, and the users, for once as terrified as we are, are being cooperative. Hours are spent on memorizing, forgetting and resetting new passwords, patching servers, and fixing outages caused by revoked keys and certificates. Everyone gets a crash course in encryption. Security is on everyone’s agenda. Users who thought certificates were for gifts now ask about SSL. Naturally, educating, updating, revoking and reissuing is time consuming, and the regular workday goes down the drain. Yet another compromised vendor or service, yet another unpatched server.
And here you should stop and think about what you are going through: if you do value security with more than just your lips, you realize this is one of the best things that happened to your operation. You are basically forced to do the very same things you knew you should have been doing all along, but always left for later. Recycling my passwords? New ones are hard to remember! Reissuing certificates on a regular basis? What kind of a sadist would put that in OpenVPN’s Wiki! Changing deployment keys? Doesn’t look amazing on anyone’s resume! Just stop and try to remember how many times, before Heartbleed, have your users/coworkers/clients cared about the same thing you do? How many times have they came to you asking you about updating a server, instead of you suggesting to them in a friendly manner they should really apply that last year of updates on their corporate laptop?
Heartbleed is the best thing that happened to network security since firewalls: Very few credentials were likely compromised via Heartbleed, but everyone is rotating their credentials and protecting themselves from more common vulnerabilities and leaked passwords. It’s hard to appreciate it between all the password resets, but when you are done with it all, sit back and have a beer. To Heartbleed!