Blanket term for things we often buy at grocery store that are not groceries, e.g., toilet paper, laundry detergent, window cleaner, saran wrap

Solution 1:

You might go with sundries

Various items not important enough to be mentioned individually (here)

While this meaning is a bit broader than what you describe, there is a convention of using sundries for exactly the types of items you list (toiletries, etc.). This is evinced by the sample phrase Oxford Dictionary Online uses in the link, "a drugstore selling newspapers, magazines, and sundries."

Further, Vocabulary.com writes:

Most people associate the word sundry with the old-fashioned drugstore in their neighborhood that used to sell all sorts of odds and ends, from magazines to hairbrushes (here).

Lastly, here is a sign from an old general store contrasting groceries with sundries.

Solution 2:

I would definitely go with household items.

The apartment we rented was completely empty. We had to buy a whole bunch of household items ourselves.

Household items differ from furniture etc, in that they are expendables, things you use up -- like all of your examples, but unlike a vase, decorations, furniture, curtains, rugs etc. A cloth is a grey area, depending on its durability.