Delay when using the volume buttons

When I change the volume using the buttons on an Apple Wireless keyboard (with a Mac Mini) I almost have a huge delay between the key press and the action (up to several seconds and from time to time even several minutes).

• using the slider on the menu works perfectly (without delay)
• resetting the PRAM does not bring any improvement
• I do not have any sound-related third party utility as boom (see an Apple Support discussion)

Any idea what the cause could be?

Edit

• Brightness is affected as well
• I also noticed that the process loginwindow is not responding and uses 100% of one core
• The console shows no errors

Typing sudo killall coreaudiod in the terminal fixed the issue straight away for me.

It also removed the delay on the keyboard brightness buttons. Issue started after going back to internal speakers after streaming audio through airport.

In your sound preferences go on the 'Sound Effects' tab and turn off 'Play feedback when volume is changed'.

This solved the delay for me. Seems it was taking a second to read the sound effect from disk each time.