Tag question with 'used to be'
I cannot find any resources online for tag question with the phrase used to be. I'm looking for something that can give the rules for this example, He used to be handsome, didn't/wasn't he?
The correct ending would be "didn't he?" Used to pairs with didn't (as in "didn't used to be").