Why Did Netlify Charge A User $104,000 For Bandwidth Overages?
Since this issue has gone viral this week, I wanted to address the Netlify issue since it's what we always recommend and use ourselves.
The post in question:
Looks like a user had a 3MB mp3 on a website that was on the free tier Netlify plan and in over 4 years haven't had any issues. Then for some reason their site was DDOS'd and incurred 60TB of overages from the file being loaded so many times from the requests resulting in $104k of fees.
They reached out to support and support said that they recognized that this was traffic from user agents and definitely a DDOS and in cases like this they generally lower the bill to 20% the original overage fee and since his was so much they can reduce it to 5%.
This was picked up by a Hacker New post in which the CEO of Netlify responded:
"Netlify CEO here. Our support team has reached out to the user from the thread to let them know they're not getting charged for this. It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact. Apologies that this didn't come through in the initial support reply."
When asked if Netlify has forgiven bills before that didn't go viral they had this to say
"Yes. We've forgiven lots and lots of bills over the last 9 years and they haven't gone viral"
And when asked How do you plan to address this issue so that it never happens again, they said:
"While I've always favored erring towards keeping people's sites up we are currently working on changing the default behavior to never let free sites incur overages"
It appears the CEO has waived all charges for the individual. So that's good. But the fact this has apparently been happening over the last 9 years and only now they are addressing the issue and changing the policy of free tier accounts not incurring overage fees is not a good look. People are understandably upset, weary, and angry.
If you want to move your sites to a different provider, there's always AWS, Cloudfare, or GitHub pages. I have talked with the CodeStitch team about how we can provide the same integrations and abilities with DecapCMS, Netlify forms, and Identity logins with Cloudfare or GitHub pages so that you can still have your blogs and CMS functions on different providers. It's going to be a lot of work and research, but we are going to be working on new kits and CMS integrations for different hosting platforms to give our users options and not have to rely on Netlify given the situtation over the last 24 hours. Our main focus right now is to finish pumping out our stitches for this next design pack coming up and the ones after to get back on schedule after our 3 month break while I was on paternity leave. If anyone comes up with any good solutions they want to share with us we're always open to hear your ideas or integrate your solutions into our resources for everyone to use and benefit from.
If you're on Netlify free tiers, I think you're safe for now since the heat is on them to prevent MORE screw ups and bad press, and they are going to fix the core of the issue. Otherwise if you want to keep using them and still sleep at night, as far as I am concerned and have read this only affected free tier sites. So you can jump on the Pro plan and should be safe. I am only recommending that if you use it currently and are worried about your current set up and workflows and need more time to find other solutions. You don't need to panic or put stress on yourself.
If you want to move to a different provider our users have been jumping on Cloudfare since they have a truely free unlimited tier with no potential for overages. Hopefully they do turn things around and this kind of thing won't happen anymore and they issue a big apology. I think given the circumstances, without any big changes, and soon, I might have to stop recommending Netlify to devs as a default option. Hopefully in the coming months we can build some new website starter kits to work with alternative hosting providers to allow our users the most flexibility in where they host their sites with our tools.
Ryan Postell
Founder/CEO CodeStitch
