Bad byte in one RAM module / Block area of RAM from being used
I have recently found out that there is a kernel parameter (to provide at boot, in grub config for example)
memmap=[ammount]M$[startlocation]M
You can supply it many times to lock out parts of the memory.
eg.
memmap=3M$217M
locks out megabytes from 217 to 219
There used to be the "BadMEM" patch for Linux, however it would take a bit of effort to upgrade it to a current keernel.
http://badmem.sourceforge.net/docu/BadMEM-HOWTO.html
That sounds like a possible bad slot on the motherboard as your testing showed. I have never seen a way around this besides just not using that slot. To my knowledge, there is no easy answer. I suppose some software could be created, but like nik said, that would be more of an SO question.
I don't think you can lock-out
specific parts of a RAM address range.
It is owned by the virtual memory management of your OS.
However, this is an interesting application -- to lock out dynamic memory address ranges from the OS. If there is such a technique available (for any OS), I am interested in knowing it.
If it is not available, that sounds like an interesting project to try.
But, that would be a stackoverflow question.