Python: Adding element to list while iterating

I know that it is not allowed to remove elements while iterating a list, but is it allowed to add elements to a python list while iterating. Here is an example:

    for a in myarr:
      if somecond(a):
          myarr.append(newObj())

I have tried this in my code and it seems to work fine, however I don't know if it's because I am just lucky and that it will break at some point in the future?

EDIT: I prefer not to copy the list since "myarr" is huge, and therefore it would be too slow. Also I need to check the appended objects with "somecond()".

EDIT: At some point "somecond(a)" will be false, so there can not be an infinite loop.

EDIT: Someone asked about the "somecond()" function. Each object in myarr has a size, and each time "somecond(a)" is true and a new object is appended to the list, the new object will have a size smaller than a. "somecond()" has an epsilon for how small objects can be and if they are too small it will return "false"

Solution 1:

Why don't you just do it the idiomatic C way? This ought to be bullet-proof, but it won't be fast. I'm pretty sure indexing into a list in Python walks the linked list, so this is a "Shlemiel the Painter" algorithm. But I tend not to worry about optimization until it becomes clear that a particular section of code is really a problem. First make it work; then worry about making it fast, if necessary.

If you want to iterate over all the elements:

i = 0  
while i < len(some_list):  
  more_elements = do_something_with(some_list[i])  
  some_list.extend(more_elements)  
  i += 1  

If you only want to iterate over the elements that were originally in the list:

i = 0  
original_len = len(some_list)  
while i < original_len:  
  more_elements = do_something_with(some_list[i])  
  some_list.extend(more_elements)  
  i += 1

Solution 2:

well, according to http://docs.python.org/tutorial/controlflow.html

It is not safe to modify the sequence being iterated over in the loop (this can only happen for mutable sequence types, such as lists). If you need to modify the list you are iterating over (for example, to duplicate selected items) you must iterate over a copy.

Solution 3:

You could use the islice from itertools to create an iterator over a smaller portion of the list. Then you can append entries to the list without impacting the items you're iterating over:

islice(myarr, 0, len(myarr)-1)

Even better, you don't even have to iterate over all the elements. You can increment a step size.

Solution 4:

In short: If you'are absolutely sure all new objects fail somecond() check, then your code works fine, it just wastes some time iterating the newly added objects.

Before giving a proper answer, you have to understand why it considers a bad idea to change list/dict while iterating. When using for statement, Python tries to be clever, and returns a dynamically calculated item each time. Take list as example, python remembers a index, and each time it returns l[index] to you. If you are changing l, the result l[index] can be messy.

NOTE: Here is a stackoverflow question to demonstrate this.

The worst case for adding element while iterating is infinite loop, try(or not if you can read a bug) the following in a python REPL:

import random

l = [0]
for item in l:
    l.append(random.randint(1, 1000))
    print item

It will print numbers non-stop until memory is used up, or killed by system/user.

Understand the internal reason, let's discuss the solutions. Here are a few:

1. make a copy of origin list

Iterating the origin list, and modify the copied one.

result = l[:]
for item in l:
    if somecond(item):
        result.append(Obj())

2. control when the loop ends

Instead of handling control to python, you decides how to iterate the list:

length = len(l)
for index in range(length):
    if somecond(l[index]):
        l.append(Obj())

Before iterating, calculate the list length, and only loop length times.

3. store added objects in a new list

Instead of modifying the origin list, store new object in a new list and concatenate them afterward.

added = [Obj() for item in l if somecond(item)]
l.extend(added)