run python script and flask app at the same time

I am trying to create project that

1. Scrape data from the website automatically in every 5 minutes and save into db
2. Flask app interacts with user. User requests data from db

python script to scrape data would be like:

import os
import time

while True:
  os.system(scrape.py)
  time.sleep(5*60)

I originally tried to scrape data from the website every time the user requests data. But I fount out that it works so slowly.

So now I am trying to make another python script that scrapes data from website and save them into db automatically. And flask app just needs to get data from db.

My question is, how can I run both python script and flask app at the same time? Or is there any better way to handle the problem?

Solution 1:

There is no reason for these two components to run in the same process, or even in the same application. They can share some modules (e.g. object definitions), but you need one batch process that builds your database, and one flask app that serves user requests. Keep them loosely coupled and you will have many benefits, including simplicity and the possibility of replacing one component with something else in the future.

So set up your project over several files, and create one execution point (file) for the flask app and one execution point for the scraper.