Storing a List into Python Sqlite3

Solution 1:

There is a normalization problem in your table.

Using 2 tables with

TABLE domains
int id primary key
text name

TABLE field_ids
int id primary key
int domain_id foreign key ref domains
text value

is a better solution.

Solution 2:

Proper database design would suggest you have a table of URLs, and a table of fields, each referenced to a URL record. But depending on what you want to do with them, you could pack lists into a single column. See the docs for how to go about that.

Is sqlite a requirement? It might not be the best way to store the data. E.g. if you need random-access lookups by URL, the shelve module might be a better bet. If you just need to record them and iterate over the sites, it might be simpler to store as CSV.