Are generators with context managers an anti-pattern?

Solution 1:

While it does indeed extend the lifetime of the object until the generator exits or is destroyed, it also can make the generators clearer to work with.

Consider creating the generators under an outer with and passing the file as an argument instead of them opening it. Now the file is invalid for use after the context manager is exited, even though the generators can still be seen as usable.

If limiting the time for how long the handles are held is important, you can explicitly close the generators using the close method after you are done with them.

This is a similar problem to what trio tries to solve with its nurseries for asynchronous tasks, where the nursery context manager waits for every task spawned from that nursery to exit before proceeding, the tutorial example illustrates this. This blog post by the author can provide some reasoning for the way it's done in trio which can be an interesting read that's somewhat related to the problem.

Solution 2:

There are two answers to your question :

• the absolutist : indeed, the context managers will not serve their role, the GC will have to clean the mess that should not have happened
• the pragmatic : true, but is it actually a problem ? Your file handle will get released a few milliseconds later, what's the bother ? Does it have a measurable impact on production, or is it just bikeshedding ?

I'm not an expert to Python alt implementations' differences (see this page for PyPy's example), but I posit that this lifetime problem will not occur in 99% of cases. If you happen to hit in prod, then yes, you should address it (either with your proposal, or a mix of generator with context manager) otherwise, why bother ? I mean it in a kind way : your point is strictly valid, but irrelevant to most cases.