Remote server security: handling compiler tools
Solution 1:
Lot of people remove compilers, and compiler tools because they can theoretically be used for exploits. I think that's dated. My experience lately has been that, if they can get in, they can get their tools in as well, so removing things like compilers, and emacs, and junk like that doesn't add much to your overall security.
I'd be extremely careful what services I ran on the machine: everything that connects to the outside is a potential vulnerability. Otherwise, I'd only remove the compilers to keep my programmers honest. I don't think it adds much to security these days.
Solution 2:
Common wisdom of "proper way to do things" is to put the minimum necessary on servers. If you don't need the compilers, don't put them on there. It's one more thing that, if cracked, a system cracker could potentially use against your system, and it introduces more binaries that could have bugs in them that can be taken advantage of.
It also frees space on the server.
That said, I think there are many small offices and small businesses (and people who don't analyze security or best practices) who leave compilers and extra tools on their servers and don't consider it a huge deal.
If you want to play it safe, remove them. Install binaries you either pre-build or test yourself on another machine or install from trusted repos. It'll also save some space and streamline your backups a little bit.
If it's a critical production server, you probably shouldn't be playing with unknown test kernel configurations on it. You'd be building it on other hardware then migrating it over. We usually just use the "good enough" kernels from our distro's repos of choice for our platform, so there's no recompilation in our case.