Using "this" as an indefinite article
Solution 1:
While the demonstrative "this" is not semantically the same as an indefinite article, it has some similarities in the sense that both can be used to introduce a new topic/object to the discourse.
For example, if I say "I want this shirt" and point to a shirt, it generally means I haven't mentioned the shirt before, whereas the statement "I want the shirt" implies that before I made my statement, you should have already been able to know which shirt I had in mind.
Perhaps this is what led to the use of "this" in examples like the one you mention, to introduce something that is not physically present and can't be pointed to, but that is being introduced in the course of the narrative.
I can't think of any very obvious restrictions on its use. I think that generally, it would imply that you are going to continue to talk about the introduced thing, or at least something related to it, more strongly than the indefinite article would: if I say "there was a dog walking down the street" it might just be a description of an area, but if I say "there was this dog walking down the street" it is likely that I am about to say something else about the dog, or at least related to it.
I guess "this" can't replace non-specific "a": "I want a hot dog" is not generally equivalent to "I want this hot dog." "I want this hot dog" would mean "There's a particular hot dog (or a particular type of hot dog) that I want" while "I want a hot dog" would generally mean "I want any hot dog."