Does a server need a GPU?
Solution 1:
You do not need one, but you will be very hard pressed to get a proper server WITHOUT one, even if it is a low end one. This is simply a matter of what people OFFER — not what you want. Server boards mostly are prepared for attaching a screen, and if anything you will value that the next time you must upgrade the BIOS. SSH will not work for that. So, you get a server that not only has a GPU, but also has IPMI functionality allowing you to see both a "virtual screen" as well as using this screen over HTML (mostly) to allow you to see and configure the BIOS. You may say "you use Linux" — but somehow you must install it, and sometimes you must update the BIOS, and things being as they are, those operations are not happening via Linux over SSH. Also at times of failure you may realize "using SSH" means "not having any idea how to get to the server as the SSH interface is failing for whatever reason". At these points a non-networked access (via a browser on a separate management system) is a lifesaver.
It has been many years since I have seen a proper server that does not have IPMI and a GPU. You literally have to look into the garbage bin these days — if anyone has something without that is proper please inform me, but I am mostly looking at SuperMicro and am not aware of any board that would fit the OP's demands.
Solution 2:
You do not need a GPU but you need some means of access in situations where the OS isn't running and hence SSH access isn't available, for example to install the OS in the first place, for configuring the hardware or in the event of an OS crash. On Intel platforms (aka industry standard architecture aka PCs) this is typically an onboard graphics controller and keyboard controller, often supplemented by a pointing device. On other platforms you may find a serial I/O interface for the same purpose to which you'll have to connect a serial terminal. Historically the latter was standard until PC architectures took over the server arena. It is still rather common in appliances and embedded systems.
Solution 3:
There is actually two questions here :
Do I need a dedicated GPU in my server ?
It depends on your needs. Dedicated GPU on servers are used for highly parallelized application (number crunching, physics simulations, signal processing, AI, cryptomining, etc.). If your applications relies on CPU only you don't need any dedicated GPU.
Do I need a video output on my headless server ?
Yes, yes you do. While it's technically possible to have a server running without it, it's very convenient to be able to plug a monitor for maintenance purpose (OS install, BIOS update, diagnostic, etc.). This video output on a server is provided by a very light integrated GPU or an management interface. I don't recommend relying on a graphics card lying around to get video output from the server, the last thing you want to do when you server is unresponsive is to have to shut it off and switch hardware to diagnose the issue. You should have a readily available way of getting video & keyboard access, IPMI is great for that but an integrated video output can also be a life saver in last resort.
Most recent server motherboard comes with a dedicated chip for platform management and basic video output, you should use it or at least get used to it for the day you'll need it.