10 year old Mac won't install OS updates?

I'm trying to update a 10 year old iMac currently running OS 10.10.5. The hardware appears to be in great shape.

When I go to Software Updates, it say no updates available.

When I manually download 10.11 or 10.12, the installer PKG will launch, go through some prompts, then say it's installed a few megabytes (MB) and stop without asking for a reboot. After a manual reboot, it's still running 10.10.5

When I go to App Store and try to install 10.13, it says there's a problem with the installer file.

The .pkg installers don't directly upgrade the OS, they produce installer applications, and those can update the OS. As the support article with the download links says:

Open the disk image, then open the .pkg installer inside the disk image. It installs an app named Install [Version Name]. Open that app from your Applications folder to begin installing the operating system."

Also, your Mac may be too old to upgrade. You can look your Mac model up with the Mactracker app or on EveryMac.com. Or just run the installer app will tell you if it's too old.

MacOS installers are date stamped such that even if the hardware is compatible with the version you are trying to install, the installer maybe deemed “expired” and not work. To get around this you need to set your system clock to a date when the installer would still be considered good. See https://www.macworld.com/article/228426/installing-os-x-what-to-do-when-this-copy-of-the-install-os-x-application-cannot-be-verified.html for more information.