Word that means "annoyingly sheepish"

You could describe that kind of person as spineless.

A spineless person lacks determination and is not willing to take risks. So is the case in your example:

B: You're right...but I don't feel confident in my abilities.

You could say B is spineless.

Lily-livered could also be used for such kind of person but it's a bit old-fashioned.

Informal words such as yellow-bellied, mousy, or chicken could also be used to describe such kind of person.

Milquetoast is also a good word for this.

Cambridge English dictionary defines milquetoast as:

a shy, nervous person with a weak character, usually a man.