Python: for loop not looping over all the files

Solution 1:

Largely untested, but something along the following lines should work.

This code first processes a file ending in 'aa.gz' (note: not all files ending in 'aa.gz' are processed first, as this is not stated in the question), then processes the remaining files. There is no particular ordering for the remaining files: this will depend on how Python has been built on the system, and what the (file)system does by default, and is simply not guaranteed.

# Obtain an unordered list of compressed files
filenames = glob.glob("*.gz")

# Now find a filename ending with 'aa.gz'
for i, filename in enumerate(filenames):
    if filename.endswith('aa.gz'):
        firstfile = filenames.pop(i)
        # We immediately break out of the loop, 
        # so we're safe to have altered `filenames`
        break
else:  
    # the sometimes useful and sometimes confusing else part 
    # of a for-loop: what happens if `break` was not called:
    raise ValueError("no file ending in 'aa.gz' found!")

# Ignoring the `full_path` part
df = pd.read_csv(firstfile, compression='gzip', header=0, sep='|', encoding="ISO-8859-1")
# do something
print(f"1 rule: The file processed is {firstfile}")
          
# Process the remaining files
for filename in filenames:
    df = pd.read_csv(filename, compression='gzip', header=0, sep='|', encoding="ISO-8859-1")
    if filename.endswith('aa.gz'):
        # do something
        print(f"2 rule: The file processed is {filename}")
    else:
        # do something else
        print(f"3 rule: The file processed is {filename}")