Login
Geez we've been getting hammered down here: Optus, MyDeal, Vinomofo, Medibank and now Australian Clinical Labs. It's crazy how much press interest there's been down here and whilst I think some of it is a bit hyperbolic, bringing the issue to the forefront and ensuring it's being discussed is certainly a good thing. Anyway, let's see what happens between now and next week's video, at this rate there'll be at least one more major Aussie breach to talk about!
Just over 3 years ago now, I sat down at a makeshift desk (ok, so it was a kitchen table) in an Airbnb in Olso and built the authenticated API for Have I Been Pwned (HIBP). As I explained at the time, the primary goal was to combat abuse of the service and by adding the need to supply a credit card, my theory was that the bad guys would be very reluctant to, well, be bad guys. The theory checked out, and now with the benefit of several years of data, I can confidently say abuse is near non-existent. I just don't see it. Which is awesome 😊
But there were other things I also didn't see, and it's taken a while for me to get around to addressing them. Some of them are fixed now (like right now, already in production), and some of them will be fixed very, very soon. I think it's all pretty cool, let me explain:
A little more background will help me explain this better: in the opening sentence of this blog post I mentioned building the original authenticated API out on a kitchen table at an Airbnb in Oslo. By that time, everyone knew I was going through an M&A process with HIBP I called Project Svalbard, which ultimately failed. What most people didn't know at the time was the other very stressful goings on in my life which combined, had me on a crazy rollercoaster ride I had little control over. It was in that environment that I created the authenticated API, complete with the Azure API Management (APIM) component and Stripe integration. It was rough, and I wish I'd done it better. Now, I have.
In the beginning, I pushed as much of the payment processing as possible to the HIBP website. This was due to a combination of me wanting to create a slick UX and frankly, not understanding Stripe's own UI paradigms. It looked like this:
Cards never ended up hitting HIBP directly, rather the site did a dance with Stripe that involved the card data going to them directly from the client side, a token coming back and then that being used for the processing. It worked, but it had numerous problems ranging from lack of support for things like 3D Secure payments, no support for other payments mechanisms such as Google Pay and Apple Pay and increasingly, large amounts of plumbing required to tie it all together. For example, there were hundreds of lines of code on my end to process payments, change the default card and show a list of previous receipts. The Stripe APIs are extraordinarily clever, but I couldn't escape writing large troves of my own code to make it work the way I originally designed it.
Two new things from Stripe since I originally wrote the code have opened up a whole new way of doing this:
Rolling to these services removed a huge amount of code from HIBP with the bulk of what's left being email address verification, API key management and handling callbacks from Stripe when a payment is successful. What all this means is that when you first create a subscription, after verifying your email address, you see these two screens:
That's the embeddable pricing table following by Stripe's own hosted payment page. I left the browser address bar in the latter to highlight that this is served by Stripe rather than HIBP. I love distancing myself from any sort of card processing and what's more, everything to do with actually taking the payment is now Stripe's problem 😊 If you're interested in the mechanics of this, a successful payment calls a webhook on HIBP with the customer's details which updates their account with a month of API key whilst the screen above redirects them over to the HIBP website where they can grab their key. Easy peasy.
I silently rolled this out a week ago, watched it being used, made a few little tweaks and then waited until now to write about it. The rollout coincided with a typical email I've received so many times before:
First of all I would like to thank you for the wonderful service that helps people to keep track of their email breaches. I was trying to build a product to provide your services via my website, something similar to Firefox, avast and 100's of other companies doing. We were trying to do it according to the guidelines mentioned in the website. However I am not able to renew my purchase due to payment gateway failures at stripe payment. Requesting you to kindly check the same and advise me on alternate methods for making the payment.
The old model often caused payments to be rejected, especially from subscribers in India. The painful thing for me when trying to help folks is that Stripe would simply report the failed payment as follows:
However, going back to the individual who raised the query above after rolling out this update, things changed very dramatically:
To the title of this section, I simply wasn't "Striping" right. I'm sure there's a way with enough plumbing that it's feasible, but why bother? I cut hundreds of lines of code out just by delegating more of the workload back to them. Further, with ever tightening PCI DSS standards (read Scott's piece, interesting stuff) the less I have to do with cards, the better.
This was a "penny drop" moment for me and it's already made a big difference in a positive way. But there's another penny that dropped for me at the same time: one-off keys were an unnecessary problem.
It was at the moment I was ripping out those hundreds of lines of code that I wondered: why do I have all the additional kludge to support the paradigm of a one-off key that only lasts a month? Why had I built (and was now maintaining) server side code to handle different types of purchases and UX paradigms to represent one-off versus recurring keys? My gut feel was that most payments formed part of an ongoing subscription but hey, who needs gut feels when you have real data?! So I pulled the numbers:
Only 7% of payments were one-offs, with 93% of payments forming part of ongoing subscriptions.
And so I killed the one-off keys. Kinda, because you can still have a key for only one month, you just purchase a monthly subscription then immediately cancel it via the Stripe Customer Portal:
That's linked into from the API key dashboard on HIBP and it'll take all of 5 seconds to do (also note the ability to change payment method directly on the Stripe site). I've added text to that effect on the HIBP website (you may have spotted that in the earlier screen cap) so in practice, the ability to purchase a one-off key is still there and the main upside of this is that I've just killed a trove of code I no longer have to worry about 🙂 Because this is the internet, I'm sure someone will still be upset, but if you only want a key for a month then that capability still well and truly exists.
All of this so far amounts to doing the same things that were always there but better. Now let's talk about the all new stuff!
The title is self-explanatory and "very soon" is in about 2 weeks from now 😎
Let me illustrate the first part of that title with a message I received recently:
Is there a way to procure a 10 year API key? Our client wants to use the Have I been Pwned plugin for [redacted service name]; however, the $3.50 monthly subscription is too small to go through procurement.
What's that saying about no good deed going unpunished? In my naivety, I made the pricing low with the thinking that was a good thing, yet here we are with that posing barriers! This was a recurring message over and over again with folks simply struggling to get their $3.50 reimbursed. I should have seen this coming after years of living the corporate life myself (I have vivid flashbacks of how hard it was to get small sums reimbursed), and filling out an untold number of expense reports. Speaking of which, this was another recurring theme:
Is there a way to pay yearly for HIBP API access vs monthly? Monthly adds overhead in paperwork.
And again, I get it, this is a painful process. It somehow feels even more painful due to the fact the sum is so low; how much time are people burning trying to justify $3.50 to their boss?! It's painful, and this likely explains why the request for annual payments is the second most requested idea on HIBP's UserVoice. The comments there speak for themselves, and I'm having corporate PTSD flashbacks just reading them again now!
Sticking with the UserVoice theme, the 5th most requested feature is for different pricing on different rate limits. This is mostly self-explanatory but what I wasn't aware of until I went and pulled the stats was just how many people were hacking around the rate limit problem. There are heaps of API accounts like this:
FreshRSS 
hibp+1@domain.com
hibp+2@domain.com
hibp+3@domain.com
...Because there can only be one key per email address, organisations are creating heaps of unique sub-addressed emails in order to buy multiple keys. This would have been a manual, laborious process; there's no automated way to do this, quite the contrary with anti-automation controls built into the process. Further, each key has it's own rate limit so I imagine they were also building a bunch of plumbing in the back end to then distribute requests across a collection of keys which, yeah, I get it, but man that seems like hard work! When I say "a collection of keys", I'm not just talking about a few of them either; the largest number of active in-use keys by a single organisation is 112. One hundred and twelve! The next largest is 110. I never expected that 🤯 (Incidentally, these orgs and the others obtaining multiple keys are all precisely the kinds I want using the API to do good things.)
Building the mechanics of annual billing and different rate limits is only part of the challenge and most of that is already done, the harder part is pricing it. I'm pulling troves of analytics from APIM at present to better understand the usage patterns, and it's quite interesting to see the data as it relates to requests for the API:
There's no persistent logging of the actual queries themselves, but APIM makes it easy to understand both the volume of queries and how many of them are successful versus failed, namely because they exceed the existing rate limit or were made with an invalid (likely expired) key. So, that's what I need to work out over the next couple of weeks when I'll launch everything and write it up, as always, in detail 🙂
The HIBP API has become an increasingly important part of all sorts of different tools and systems that use the data to help protect people impacted by data breaches. The changes I've pushed out over the last week help make the service more accessible and easier to manage, but it's the coming changes I'm most excited about. These are the ones that will make life so much easier on so many people integrating the service and, I sincerely hope, will enable them to do things that make a much more profound impact on all of us who've been pwned before.
Go and check out how the whole API key process works, I'd love to hear your feedback 😊
Aussie breachapalooza! That what it feels like this week between Optus (ok, it was weeks ago but it's still in the news), Vinomofo, My Deal and the mother of all of them (at least as far as media interest goes), Medibank. That last one totally smashed my week out with unprecedented press enquiries, so is it any wonder I totally missed the Microsoft one? I read through that last one live in this week's video and as you'll hear, a breach of any kind is never a good look but what stands out for me about this one isn't the breach itself, rather the marketing effort SOCRadar has made around it. As I say in the video, it just feels... icky. See if you agree.
I decided to do something a bit different this week and mostly just answer questions from my talk at GOTO Copenhagen last week. I wasn't actually in Denmark this time, but a heap of really good questions came through and as I started reading them, I thought "this would actually make for a really good weekly update". So here we are, and those questions then spurned on a whole heap more from the live audience too so this week's video became one large Q&A. I hope you enjoy this one, let me know if I should do more of these in the future.
Geez it's nice to be home 😊 It's nice to live in a home that makes you feel that way when returning from a place as beautiful as Bali 😊 This week's video is dominated by the whole discussion around this tweet:
I love that part of the Microsoft Security Score for Identity in Azure improves your score if you *don't* enforce password rotation, what a sign of the times! Who out there still works somewhere that forces rotation (because "reasons")? pic.twitter.com/a2yQQvNRpa
— Troy Hunt (@troyhunt) October 6, 2022
I love this for the way it throws traditional logic out the window, logic we all knew sucked and I suspect the massive engagement the tweet drove is due to precisely that: Microsoft giving us all a good reason to whinge about a sucky practice that still prevails so broadly. So... I hope you enjoy listening to just how bad enforced password rotation sucks 😊
How's this weeks video for a view?! It's a stunning location here in Bali and it's just been the absolute most perfect spot for a honeymoon, especially after weeks of guests and celebrations. But whoever hacked and ransom'd Optus didn't care about me taking time out and I've done more media in the last week than I have in a long time. I don't mind, it's a fascinating story the way this has unfolded and that's where most of the time in this week's video has gone, I hope you enjoy my analysis of what has become a pretty crazy story back home in Australia.
Wow, what a week! Of course there's lots of cyber / tech stuff in this week's update, but it was really only the embedded tweet below on my mind so I'm going to leave you with this then come to you from somewhere much more exotic than usual (and I reckon that's a pretty high bar for me!) next week 😎
Absolutely over the moon to formally make @Charlotte_Hunt_ a part of our family ❤️ 💍 pic.twitter.com/XfahXElboC
— Troy Hunt (@troyhunt) September 21, 2022
I came so close to skipping this week's video. I'm surrounded by family, friends and my amazing wife to be in only a couple of days. But... this video has been my constant companion through very difficult times, and I'm happy to still being doing it at the best of times 😊 So, with that, I'm signing out and heading off to do something much more important. See you next week.
Taking a bit of time off Twitter while @charlottelyng and I do more important things 💍 👰♀️ pic.twitter.com/9JJrPM9kWX
— Troy Hunt (@troyhunt) September 13, 2022
I'm so excited to see the book finally out and awesome feedback coming in, but I'm disappointed with this week's video. I frankly wasn't in the right frame of mind to do it justice (it's been a very hard road up until this point, for various reasons), then my connection dropped out halfway through and I had to roll to 5G, and now I'm hearing (both from other people and with my own ears), a constant background noise being picked up by the mic. Argh! But, that's the reality of scheduled live streams and for better or worse, you end up getting the "warts and all" version. It is what it is, and next week's will be better 😊
The first time I ever wrote publicly about a company's security vulnerabilities, my boss came to have a word with me after seeing my name in the news headlines.
One of the worst days I've ever had was right in the middle of the Have I Been Pwned sale process, and it left me an absolute emotional wreck.
When I wrote about how I deal with online abuse, it was off the back of some pretty nasty stuff... which I've now included in this book 😊
These are the stories behind the stories and finally, the book about it all is here:
I announced the book back in April last year after Rob, Charlotte and I had already invested a heap of effort before releasing a preview in October. I'd hoped to have it out by Christmas... but it wasn't perfect. Ok, so it'll never be perfect without faults of any kind, but it had to meet an extremely high bar for me to be happy with an end result we could charge people money for. I completely rewrote the intro, changed a bunch of the posts we decided to include, reordered them all and edited a heap of the personal stuff. Rob wrote an amazing intro (I genuinely felt emotional reading it, given the time of both our lives he refers to), and both Charlotte and I wrote pieces at the end of it all. That bit in particular is very personal, and it's what was the most natural to us at the time. With our wedding being only next week, it just felt... right. That's why we're publishing today; to get this story about the earlier phases of my life out before the next phase begins. There are things we both wanted to say, and I hope you enjoying reading them 😊
We actually pushed the book out to a preview audience a couple of weeks ago and requested feedback to make it even better before releasing to the masses. We got two kinds of feedback: typos, which we've now fixed and testimonials, which have been awesome:
Great read! Troy has the technical know-how, and is able to effectively communicate complex topics so that anyone understands them. The personal stories kept me hooked!
- Mikko Hyppönnen
I haven't been able to put the book down. The added intros and epilogue on each post in particular and the retrospectives from today's perspective are particularly interesting... Captivating stuff, apart from infosec, you really feel as though you’ve been taken on a journey with Troy through the years of living in paradise a.k.a. Gold Coast, craft beer, good coffee’s, travel and jet skis. Top of my list for resources I point anyone new to the space to. Simply Outstanding read - 10/10.
- Henk Brink
Love, love, LOVE the intros - i'm only familiar with Lars in terms of his online identity and videos, but the "commendations" from Richard Campbell, and in particular Rob Conery are fantastic and really anchor an emotional aspect to the book, that i was not expecting. Great to see a book deliver this authenticity - we're all only human after all!. I "cheated" and also skipped to read Charlotte's epilogue and again was blown away by the depth and genuine nature of the emotion on display. I honestly was not expecting there to be so much heart on display, but am very glad there is.
- C. Morgan
A famous American newsman used to call this "The rest of the story." This is the kind of insight that only comes with time, as the author can reflect and pick out important details that didn't get the coverage they deserved and these then find a life of their own. This book provides "the rest of the story" behind other stories which broke months or years ago. It gives these stories new life, and new significance.
- Pat Phelan
PWNED! Troy Hunt takes us on his life journey, ups and downs, explaining how haveIbeenpwned came to be, raising awareness of the world’s poor password and online security habits... This book has it all. Plenty of tech, data breaches, career hacks, IoT, Cloud, password management, application security, and more, delivered in a fun way. This info is gold, and has improved and complemented my career in IT, and also my digital life at home too. I highly recommend this book as it explains cyber security so well, and hope you have your eureka moment and improve your cyber hygiene too.
- George Ousak
I never imagined turning blog posts into a book, but here we are, and it's come out awesome 😎 This book is a labour of love the three of us have poured ourselves into over the last couple of years. It's a personal story I'm publishing at a very personal time and it's now live at book.troyhunt.com
