When a Get request to Amazon S3 returns 304 Not Modified, does it also count against request quota?
I save user profile pictures on Amazon S3 with some random path/filenames for each user. The database knows the path for each user image. My question is that when user navigates within the app, a get request is issued to the profile picture several times. After the first time, S3 returns http 304 Not Modified status. Does every such get request count against the GET request quota for billing?
Solution 1:
I understand that you're looking for firm proof as to whether GET requests that return a 304 are counted as a GET request for billing purposes, in the same way that a GET request returning a 200 is.
The documentation does not call out any billing difference based on the returned HTTP Status Code, but this does not offer any proof one way or another. To provide you with a firm answer, I setup a Cost and Usage Report for my account with the aggregation frequency set to hourly and resource ids included, and then tested this and looked at what appeared in the report.
TLDR; The billing report confirms what everyone expected, a GET is billed as a GET whether it returns a status code of 200 or 304.
Test Setup
I selected an unused publically readable S3 bucket of mine and uploaded an image file and set an HTTP Header of Cache-Control: public, max-age=10 on the file. This means if the page is reloaded after 10 seconds then the browser must check with the server to see if the file has changed before using its locally cached copy.
To ensure the test would be repeatable and not have any unexpected results (e.g. from a typo etc), I created a local html file that would display the image.
<html>
<body>
<img src="http://mybucket.s3.amazonaws.com/5DS05736-2.jpg" width="868" height="488" />
</body>
</html>
The reason for keeping this on my local machine was to avoid any requests to S3 for things like favicon (which would potentially pollute the results).
Confirming the test would work
I performed an initial test in chrome by opening developer tools, looking in the network tab and loading my local html file. On the first request the image was loaded with an HTTP Status Code of 200. Waiting and then refreshing the page showed the image being loaded a second time with an HTTP Status Code of 304.
I cleared the cache of entries from the last hour.
Actual Test
The next morning I repeated the experiment. To make sure I could distinguish between tests from the previous evening and the real test, I noted the time and the status codes I got. I did one load without the image in the cache resulting in a 200, and then two refreshes ~20 seconds apart resulting in 304s.
This would give the following possibilities in the billing report:
- 1 GET Request = Only GET requests returning 200 are counted
- 2 GET Requests = Only GET requests returning 304 are counted (unlikely)
- 3 GET Requests = All GET requests are counted
The Billing Report
I then waited until the billing report was available and loaded it into Athena. If you're repeating this, the Athena CREATE TABLE command is:
CREATE EXTERNAL TABLE `cost_and_usage`(
`identity_lineitemid` string,
`identity_timeinterval` string,
`bill_invoiceid` string,
`bill_billingentity` string,
`bill_billtype` string,
`bill_payeraccountid` string,
`bill_billingperiodstartdate` string,
`bill_billingperiodenddate` string,
`lineitem_usageaccountid` string,
`lineitem_lineitemtype` string,
`lineitem_usagestartdate` string,
`lineitem_usageenddate` string,
`lineitem_productcode` string,
`lineitem_usagetype` string,
`lineitem_operation` string,
`lineitem_availabilityzone` string,
`lineitem_resourceid` string,
`lineitem_usageamount` string,
`lineitem_normalizationfactor` string,
`lineitem_normalizedusageamount` string,
`lineitem_currencycode` string,
`lineitem_unblendedrate` string,
`lineitem_unblendedcost` string,
`lineitem_blendedrate` string,
`lineitem_blendedcost` string,
`lineitem_lineitemdescription` string,
`lineitem_taxtype` string,
`product_productname` string,
`product_accountassistance` string,
`product_architecturalreview` string,
`product_architecturesupport` string,
`product_availability` string,
`product_bestpractices` string,
`product_cacheengine` string,
`product_caseseverityresponsetimes` string,
`product_clockspeed` string,
`product_currentgeneration` string,
`product_customerserviceandcommunities` string,
`product_databaseedition` string,
`product_databaseengine` string,
`product_dedicatedebsthroughput` string,
`product_deploymentoption` string,
`product_description` string,
`product_durability` string,
`product_ebsoptimized` string,
`product_ecu` string,
`product_endpointtype` string,
`product_enginecode` string,
`product_enhancednetworkingsupported` string,
`product_executionfrequency` string,
`product_executionlocation` string,
`product_feecode` string,
`product_feedescription` string,
`product_freequerytypes` string,
`product_freetrial` string,
`product_frequencymode` string,
`product_fromlocation` string,
`product_fromlocationtype` string,
`product_group` string,
`product_groupdescription` string,
`product_includedservices` string,
`product_instancefamily` string,
`product_instancetype` string,
`product_io` string,
`product_launchsupport` string,
`product_licensemodel` string,
`product_location` string,
`product_locationtype` string,
`product_maxiopsburstperformance` string,
`product_maxiopsvolume` string,
`product_maxthroughputvolume` string,
`product_maxvolumesize` string,
`product_maximumstoragevolume` string,
`product_memory` string,
`product_messagedeliveryfrequency` string,
`product_messagedeliveryorder` string,
`product_minvolumesize` string,
`product_minimumstoragevolume` string,
`product_networkperformance` string,
`product_operatingsystem` string,
`product_operation` string,
`product_operationssupport` string,
`product_physicalprocessor` string,
`product_preinstalledsw` string,
`product_proactiveguidance` string,
`product_processorarchitecture` string,
`product_processorfeatures` string,
`product_productfamily` string,
`product_programmaticcasemanagement` string,
`product_provisioned` string,
`product_queuetype` string,
`product_requestdescription` string,
`product_requesttype` string,
`product_routingtarget` string,
`product_routingtype` string,
`product_servicecode` string,
`product_sku` string,
`product_softwaretype` string,
`product_storage` string,
`product_storageclass` string,
`product_storagemedia` string,
`product_technicalsupport` string,
`product_tenancy` string,
`product_thirdpartysoftwaresupport` string,
`product_tolocation` string,
`product_tolocationtype` string,
`product_training` string,
`product_transfertype` string,
`product_usagefamily` string,
`product_usagetype` string,
`product_vcpu` string,
`product_version` string,
`product_volumetype` string,
`product_whocanopencases` string,
`pricing_leasecontractlength` string,
`pricing_offeringclass` string,
`pricing_purchaseoption` string,
`pricing_publicondemandcost` string,
`pricing_publicondemandrate` string,
`pricing_term` string,
`pricing_unit` string,
`reservation_availabilityzone` string,
`reservation_normalizedunitsperreservation` string,
`reservation_numberofreservations` string,
`reservation_reservationarn` string,
`reservation_totalreservednormalizedunits` string,
`reservation_totalreservedunits` string,
`reservation_unitsperreservation` string,
`resourcetags_username` string,
`resourcetags_usercostcategory` string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
WITH SERDEPROPERTIES (
'escape.delim'='\\')
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
's3://my-billing-bucket/reports/hourly/20180501-20180601/dcd20d15-fffd-4a40-bc5d-555f32fc64b2'
TBLPROPERTIES (
'skip.header.line.count'='1')
And the query to narrow it down to the usage you're looking for is:
select * from cost_and_usage where lineitem_resourceid = 'my-bucket' and lineitem_operation = 'GetObject' and lineitem_usagetype = 'USW2-Requests-Tier2' order by lineitem_usagestartdate desc
You will need to change my-bucket and USW2-Requests-Tier2 to the appropriate values for your bucket and bucket location etc.
For the hour in which I performed my test, the lineitem_usageamount column lists 3 requests. This means that all of my requests were billed.
If you're wondering how I can be sure no other use of the bucket happened, I have CloudTrail with S3 logging enabled and pointing at CloudWatch Logs. Using this I was able to verify that no other requests were made to the bucket at that time. :)
Solution 2:
Does every such get request count against the GET request quota for billing?
Yes. "GET, SELECT and all other Requests" are in the exact same price category (source: Amazon S3 Pricing). The 302 requests are part of GET category (talking about "Request Pricing", your question is not about "Data Transfer Pricing").