We're finished vs We're done [duplicate]
According to the Online Etymology Dictionary, "done" has had "finished" as a meaning since the early 13th century, so I think you're on pretty solid ground with it.
I'm afraid your husband appears to be a linguistic snob. "I'm done [X-ing]" is a normal colloquial expression, at least in the USA.
Incidentally, there is a distinct transatlantic divide in the prevalence of "I'm done [X-ing]". It is far more common in the USA than it is in Britain. (Your profile doesn't say where you live, but my starting assumption is that it is somewhere in America.)