Solution 1:

The command window uses those key combination for varies applications within the dos emulating window. If Alt-F4 closed the window, it may be impossible to use certain utilities. If you want a keyboard only fix just type exit to quit.

Solution 2:

On the other hand, if you are looking to have Alt+F4 close the console window, you can use the following AutoHotkey script:

#IfWinActive, ahk_class ConsoleWindowClass
    !F4::WinClose, A

#IfWinActive