How can a linux kernel be so small?

Solution 1:

Yes.

If you strip them down (compile only modules for the things that you actually need.)
Optimize the kernel for size (ie.: -Os)
Compress the kernel (such as XZ.)

These "options" are all available in the standard kernel and you can do this at the configuration step. You don't need any kind of special tool.

Using those three things will result in such a small kernel. You can make even smaller ones, for a targeted system. Like I know my PC, I know my stuff. So I'll just compile what I need. Like my own SATA drivers, a driver for USB, etc. Nothing else. No webcam, no old IDE drives, nothing.

The bigger ones you encounter are the WHOLE kernel. They are compiled with the most generic options to be able to run on most hardware without issues and they contain almost every driver available as modules. These modules don't make the kernel slower speed-wise as the modules are loaded on-demand.

(There are 3 states for modules at the configuration stage:
[ ] - Won't get compiled, won't be part of your kernel.
[M] - Will be compiled as a module.
(You can modprobe it, or your system will auto-load it when it's needed.
Depends on the system, userland.)
[X] - It'll be compiled into the kernel AND will be present - always.

Distros with small but still generic kernel:

Tiny Core Linux, Micro Core Linux, 12MB Linux GUI Desktop, Live, Frugal, Extendable
Puppy Linux Community Home - Getting Started

Solution 2:

Early Linux distributions such as Tom's Root-Boot, MuLinux, Diet-router, (the now defunct) LOAF and many others now departed, all fit a working Linux system on one or two 1.44 MB diskettes.

The Linux kernel has grown but it is modular: kernel modules are loaded as needed. Thus it is still possible to produce a Linux distribution with a very small footprint.

See: Kernel Size Tuning Guide - eLinux.org

If you read Linux_tiny.pdf you will see this