Name for this grammatical construction in literary English? [duplicate]

In your first example sentence, the preposition "with" is followed by a nominal phrase ("a hint of apology in his voice") functioning as the object of the preposition. Deleting "with" means that it can no longer function as an OOTP; instead, it becomes an absolute nominal phrase, which is perfectly grammatical.

In your next example sentence, the subordinating conjunction "while" introduces an adverbial clause ("while [[she was]] setting down her silverware").* Deleting "while" ruins that clause's structure; however, unlike the situation in the previous example, the remaining words do not seem to make sense as an absolute nominal phrase**; after all, they did not constitute a nominal phrase to begin with. It therefore seems best to me to construe "setting" as a present participle (that's actually how I'd also have construed it in the original sentence) and the phrase "setting down her silverware" as functioning adverbially.

As far as readability/flavor, I think that all four sentences are perfectly fine.

*I expect that others may analyze this differently.

**An absolute nominal phrase typically contains a nominal with an adjective or appositive dependent. We could consider "setting down her silverware" to be a nominal (if "setting" were a gerund), but then we'd still have to add such a dependent, e.g.:

"I've had better," Jane decreed, setting down her silverware the only action she could undertake.

Now, "the only action she could undertake" is in apposition to "setting down her silverware". However, this clearly adds some meaning that wasn't in the original sentence.