What constitutes "idle" CPU during T2 CPU credit calculation?
Solution 1:
Good question, the Amazon EC2 documentation for CPU Credits in T2 Instances could be more precise here indeed - the introductory blog post New Low Cost EC2 Instances with Burstable Performance provides a better explanation, see the table in the middle:
The column labeled "Baseline Performance" indicates the percentage of single core performance of the underlying physical CPU allocated to the instance. For example, a t2.small instance has access to 20% of a single core of an Intel Xeon processor running at 2.5 GHz (up to 3.3 GHz in Turbo mode). A t2.medium has access to 40% of the performance of a single core, which you (or your operating system, to be a bit more precise) can use on one or both cores as dictated by demand.
The column labeled "CPU Credits / Hour" indicates the rate of CPU Credits that the T2 instance receives each hour. CPU Credits accumulate when the instance doesn't use its baseline allocation of CPU, and are spent when the instance is active. Unused CPU Credits are stored for up to 24 hours. [emphasis mine]
Solution 2:
"idle" is notoriously hard to define in general for Linux servers. On general level, idle should be any time not spent on user-mode, system-mode, low priority mode, i/o waiting or interrupts -- with the special provision that given the right kind of workload, i/o wait time could be converted to user-mode time (for another process).
However, on the question of idle of AWS t2, this doesn't hold. It didn't hold for t1 either (due to "stolen" time), but even that doesn't explain t2 behavior. I've run into scenarios where idle time (as reported both by Linux top and by AWS CloudWatch) is way above the documented threshold levels, yet the instance is losing CPU credits instead of gaining them. I believe this to be an accounting error on Amazon's side, and until it is fixed, trying too hard to understand how CPU utilization and idle are defined will be a waste of time..
I've posted some illustrative charts on AWS's support forums, where hopefully they will respond. https://forums.aws.amazon.com/thread.jspa?messageID=568026#568026