Iterating over a dictionary in python and stripping white space
I am working with the web scraping framework Scrapy and I am wondering how do I iterate over all of the scraped items which seem to be in a dictionary and strip the white space from each one.
Here is the code I have been playing with in my item pipeline:
for info in item:
info[info].lstrip()
But this code does not work, because I cannot select items individually. So I tried to do this:
for key, value item.items():
value[1].lstrip()
This second method works to a degree, but the problem is that I have no idea how then to loop over all of the values.
I know this is probably such an easy fix, but I cannot seem to find it.
Solution 1:
In a dictionary comprehension (available in Python >=2.7):
clean_d = { k:v.strip() for k, v in d.iteritems()}
Python 3.X:
clean_d = { k:v.strip() for k, v in d.items()}
Solution 2:
Not a direct answer to the question, but I would suggest you look at Item Loaders and input/output processors. A lot of your cleanup can be take care of here.
An example which strips each entry would be:
class ItemLoader(ItemLoader):
default_output_processor = MapCompose(unicode.strip)