Contact back to somebody?

I ran a search at CoCA, BNC and Google ngrams for "contact you back". The only hit I got was at BNC, which was for speech in 1992.

In conclusion, "contact you back" is not usual English. It is fine in informal, conversational contexts, but should not be used in formal circumstances.

"Contacting you back" resulted in zero hits on any of the corpuses.

It is not unusual to say you will do something back to someone, meaning you'll do something in response at a later time or date, e.g. call you back or email you back.

I'm not sure why contact does not seem to work in the same way. It could perhaps be to do with its general nature, i.e. it doesn't specify how the contacting will take place.

Contact back to someone indeed is not common English usage.

Instead, consider using any of the following:

Sorry for my tardy/late reply/response...

Sorry for not getting back to you earlier/sooner...

Sorry for getting back to you (that) late...

Sorry for not replying/responding to you earlier/sooner...