Why can I fill my black star by soul trapping a mammoth?

According to the UESP wiki, the Black Star should only get filled with black (i.e. humanoid) souls, but due to a bug/glitch (as you said) it also accepts 'white' (i.e. non-humanoid) souls.