How is graphics RAM different from system RAM?

But what I don't get is why standard system RAM has always been a generation behind the RAM used on video cards.

The GDDR specification, while based on the DDR standard, has its own hardware specification. The DDR specification is technically ahead of the GDDR specification, since GDDR is based on the previous DDR specification (most of the time, except when it's based on the previous GDDR specification).

One of the reasons there is a false belief that GDDR is ahead of DDR is that there have been multiple iterations of the GDDR standard that were based on DDR3. This was also the case with GDDR2, being that its specification has design elements from both DDR and DDR2.

However, it is important to note that this GDDR2 memory used on graphics cards is not DDR2 per se, but rather an early midpoint between DDR and DDR2 technologies. Using "DDR2" to refer to GDDR2 is a colloquial misnomer.

Source: DDR2 SDRAM

Likewise, GDDR4 and GDDR5 both took design elements from DDR3. GDDR5 obviously is an improved GDDR design, when compared to GDDR4.

Like its predecessor, GDDR4, GDDR5 is based on DDR3 SDRAM memory, which has double the data lines compared to DDR2 SDRAM. GDDR5 also uses 8-bit wide prefetch buffers similar to GDDR4 and DDR3 SDRAM.

Source: GDDR5 SDRAM

As I understand it, they're both just different types of SDRAM, but it seems to me that the differences could be abstracted away by the memory controller baked into CPU and GPU silicon.

The two standards are actually vastly different. The difference in the number of bits that can be transferred over a data line is one of those differences. The GDDR specification is not compatible with Intel and AMD x86 processors. The GDDR specification is able to transfer more bits, due to it being connected to an entirely different connection, mainly PCI-e (within the specification of the various revisions of this standard).

The current standard for system RAM is DDR4, but video cards were using GDDR4 for years before DDR4 became a thing for desktops.

This is due to the fact that GDDR4 is based off the DDR3 specification, not the DDR2 specification. The DDR3 standard wasn't ratified until 2005. We didn't see products until 2007 due to entirely different market needs. GDDR4 was announced in 2005 and didn't see products until 2007. So you can see that while they have different names, the actual products were released together.

• GDDR4 SDRAM
• DDR3 SDRAM

Video cards are now shipping with HBM RAM (GDDR5?), which is faster than DDR4 system memory.

The current GDDR standard(s) are actually GDDR5X and GDDR6. HBM (High Bandwidth Memory) is a Hynix and Samsung DDR manufacturing process.

Why aren't we using the same kind of RAM for both?

The two standards are not compatible with one another.

What makes them different?

What makes them different is their manufacturing process and their specifications. While GDDR is based off the DDR specification, GDDR is not actually ahead of DDR, although there huge performance gaps between the two standards at this point due to the available bandwidth that GDDR has access to.

The underlying tech is more or less the same, GPUs just leverage a much wider memory bus.

GPUs are easier to design this way as a single unit where many memory modules can be directly connected to the processing unit through a custom circuit board. This allows for a very wide memory bus, often exceeding 256-bits. HBM takes this further with with a 1024-bit bus.

CPUs rely on a much more generalized architecture of sockets and motherboard specifications, so more than the standard two 64-bit channels is typically reserved to the high-end and server market.

It should also be mentioned that GPU memory is tuned to trade latency performance for high bandwidth - lots of shoveling and not a lot of seeking. This is not the case with CPU memory where low latency is desired for good random access speeds.

One special feature of some types of graphics RAM is that they can be accessed by two independent or mostly independent bus systems - which makes using them as either framebuffers (the portion of video ram where the pixels sent to the screen every 1/60th or so second are kept) or texture buffers easier and doable with less access conflicts and overhead....