How to solve "OSError: telling position disabled by next() call"
I don't know if this was the original error but you can get the same error if you try to call f.tell() inside of a line-by-line iteration of a file like so:
with open(path, "r+") as f:
for line in f:
f.tell() #OSError
which can be easily substituted by the following:
with open(path, mode) as f:
line = f.readline()
while line:
f.tell() #returns the location of the next line
line = f.readline()
I have an older version of Python 3, and I'm on Linux instead of a Mac, but I was able to recreate something very close to your error:
IOError: telling position disabled by next() call
An IO error, not an OS error, but otherwise the same. Bizarrely enough, I couldn't cause it using your open('a+', ...)
, but only when opening the file in read mode: open('r+', ...)
.
Further muddling things is that the error comes from _io.TextIOWrapper
, a class that appears to be defined in Python's _pyio.py
file... I stress "appears", because:
The
TextIOWrapper
in that file has attributes like_telling
that I can't access on the whatever-it-is object calling itself_io.TextIOWrapper
.The
TextIOWrapper
class in_pyio.py
doesn't make any distinction between readable, writable, or random-access files. Either both should work, or both should raise the sameIOError
.
Regardless, the TextIOWrapper
class as described in the _pyio.py
file disables the tell
method while the iteration is in progress. This seems to be what you're running into (comments are mine):
def __next__(self):
# Disable the tell method.
self._telling = False
line = self.readline()
if not line:
# We've reached the end of the file...
self._snapshot = None
# ...so restore _telling to whatever it was.
self._telling = self._seekable
raise StopIteration
return line
In your tell
method, you almost always break
out of the iteration before it reaches the end of the file, leaving _telling
disabled (False
):
One other way to reset _telling
is the flush
method, but it also failed if called while the iteration was in progress:
IOError: can't reconstruct logical file position
The way around this, at least on my system, is to call seek(0)
on the TextIOWrapper
, which restores everything to a known state (and successfully calls flush
in the bargain):
def tell(self, char=False):
t, lc = self.f.tell(), 0
self.f.seek(0)
for line in self.f:
if t >= len(line):
t -= len(line)
lc += 1
else:
break
# Reset the file iterator, or later calls to f.tell will
# raise an IOError or OSError:
f.seek(0)
if char:
return lc, t
return lc
If that's not the solution for your system, it might at least tell you where to start looking.
PS: You should consider always returning both the line number and the character offset. Functions that can return completely different types are hard to deal with --- it's a lot easier for the caller to just throw away the value her or she doesn't need.
Just a quick workaround for this issue:
As you are iterating over the file from the beginning anyways, just keep track of where you are with a dedicated variable:
file_pos = 0
with open('file.txt', 'rb') as f:
for line in f:
# process line
file_pos += len(line)
Now file_pos
will always be, what file.tell()
would tell you. Note that this only works for ASCII files as tell and seek work with byte positions. Working on a line-basis it's easy though to convert strings from byte to unicode-strings.