Left-dislocation of a content clause, or just a comma splice?
Solution 1:
I don't think your examples involve left dislocation. "That's what I say" is a separate, independent clause. The written forms have some odd punctuation, but there is nothing very interesting going on there.
On the other hand, there are other constructions where sentences act like noun phrases. "Joe sneezed, which surprised me." There, the "coreferential" antecedent for the noun phrase "which" is the sentence "Joe sneezed".