to begin with vs in the first place

I was wondering if it would be grammatically and idiomatically correct to use to begin with in the sense of

used at the end of a sentence to talk about why something was done or whether it should have been done or not

For example :

If you don’t like her, why invite her in the first place?

I should never have taken that job in the first place.

Is it possible to say :

If you don’t like her, why invite her to begin with?

I should never have taken that job to begin with.

Yes, you can use it. You can google it and check for references; here's one that you can bear out, from Invincible By Dustin Humphreys.

"Listen, we should never have taken that job to begin with," he said.

"In the first place" and "To begin with" are both ok to be used at the end of a sentence (in the examples you provided). However, I hear "to begin with" used more in the beginning of a statement, rather than the end. For example:

I should never have taken that job in the first place. To begin with, it didn't even meet the minimum wage requirement.

or whatever reason. Basically:

I should not have __________ in the first place. To begin with, **** reason for previous sentence ***** .