I can't reply to the comments of the other answer, but have you tried setting both auth lines to required instead of having radius be set as sufficient?
configure pam with ssh to do 2 factor authentication
I can't reply to the comments of the other answer, but have you tried setting both auth lines to required instead of having radius be set as sufficient?