Jonty WareingUser: you charge me when people make unauthorised requests to an S3 bucket?
AWS: yes of course
User: but
AWS: working as intended
User: but
AWS: thank you for your money
IBBoard@jonty On the one hand, this is bad. People wouldn't expect those bills.
On the other hand, is this really materially different to a VPS or anything else with "included amount then overage charges" for bandwidth?
If someone keeps hitting your server with large request payloads then you're still going to get charged bandwidth whether you 403 it or not.
VPS firewall would limit it, but not prevent it.
Main difference is that this can be millions of 0 byte requests.
Lunaphied@jonty @ibboard most/all hosting we've seen doesn't charge ingress bandwidth. So you can at least theoretically block outbound transfers once you've used up your bandwidth (I say theoretically because there's no tools for strongly enforcing this so you're still gambling if you get hit by lots of request traffic).
The reason this is different is because AWS (and many others but AWS in particular is terrible about this) charges per-request too, and that is charged regardless of validity. The cost is tiny subtractions of a cent but because they're effectively free inbound, you just send a lot and boom. Massive bill.
@Lunaphied @jonty I think most hosts work on the assumption that a web server is receiving small requests and returning much larger payloads. But if you're accepting a constant stream of gigs of data then they're not going to be happy. Some companies explicitly list this as "your bandwidth is the higher of ingress or egress". Others will keep quiet until you become a problem.
Bandwidth still costs per request. It's just not recorded that way.
@ibboard @jonty yeah I think ingress is a little different though because they're more likely to consider sudden high traffic ingress as a DDOS and protect you if they offer that service or just cut you off.
Outbound bandwidth is expected though so it's just like "hope you don't fuck up your configs!"
@Lunaphied @jonty If I run an archive-only server where I upload terabytes of data per week but hardly download anything, would that be blocked as a DDOS? Because I'm well within the outbound bandwidth limits, but you won't get far claiming that your account should be fine because you're not downloading much.
AWS is just in the "accepts both inbound-heavy and outbound-heavy workloads and charges appropriately" camp. But here it's unexpected.
@Lunaphied @jonty But my point is, is it really that different a scenario?
Someone makes lots of requests into your service on any platform where the billing is by volume. You have a high volume, you get billed a lot, whether the service at the end of the bill-by-volume bit accepted the request or not.
I have no way of preventing people sending lots of data to my physical server with bandwidth limits. Just as there's no way to stop requests to S3 buckets.
@Lunaphied @jonty Is it RIGHT that AWS charges for denied requests as well as authorised ones? Generally not.
I expect they can tell the difference here and could choose not to charge (whereas a network provider can count bandwidth without working out if I accept it decline the traffic).
It's only reasonable if it came from your own account and was denied. And even then, I can't see a use case where you would intentionally leave something making millions of rejected S3 requests.
@jonty @ibboard yeah but this is flat out stupid. If their systems don't have anti-abuse code to handle enough failed requests that it's costing thousands then their systems are poorly designed. Absolutely they would not accept such a thing hitting their own usages if it were incurring real and actual costs like that.
AWS is a scam of a provider. They literally structure their entire service not just around making money (as is typical of a normal business) but especially around creating thousands of hidden fees which have only gotten worse and worse with time.