How can I send a GET request from my flask app to another site?

Originally, I tried to post an ajax request from my client side to a third party url, but it seems that the browser have security issues with that. I thought about sending an ajax to the server side, from there to send a GET request to the third party, get the response and send it back to the client side. How can I do that with flask?

Solution 1:

Install the requests module (much nicer than using urllib2) and then define a route which makes the necessary request - something like:

import requests
from flask import Flask
app = Flask(__name__)

@app.route('/some-url')
def get_data():
    return requests.get('http://example.com').content

Depending on your set up though, it'd be better to configure your webserver to reverse proxy to the target site under a certain URL.

Solution 2:

Flask alone does not have this capability, but it is a simple matter to write a request handler that makes a request to another server using an HTTP client library and then return that response.

# third-party HTTP client library
import requests

# assume that "app" below is your flask app, and that
# "Response" is imported from flask.

@app.route("/proxy-example")
def proxy_example():
    r = requests.get("http://example.com/other-endpoint")
    return Response(
        r.text
        status=r.status_code,
        content_type=r.headers['content-type'],
    )

However, this will not achieve exactly the same result as you might expect from a client-side request. Since your server cannot "see" any cookies that the client browser has stored for the target site, your proxied request will be effectively anonymous and so, depending on the target site, may fail or give you a different response than you'd get requesting that resource in the browser.

If you have a relationship with the third-party URL (that is, if you control it or are able to work with the people who do) they can give access for cross-domain requests in the browser using CORS (which is only supported in modern browsers) or JSON-P (an older workaround that predates CORS).

The third-party provider could also give you access to the data you want at an endpoint that is designed to accept requests from other servers and that provides a mechanism for you to authenticate your app. The most popular protocol for this is OAuth.

Solution 3:

As the other answers have stated using the requests module for python would be the best way to tackle this from the coding perspective. However as the comments mentioned (and the reason I came to this question) this can give an error that the request was denied. This error is likely cause by SELinux.

To check if this is the issue first make sure SELinux is enabled with this command:

sestatus

If 'Current Mode' is 'enforcing' then SELinux is enabled.

Next get the current bool values that apply directly to apache with this command:

getsebool -a | grep httpd

Look for the setting 'httpd_can_network_connect' this determines if apache is allowed to make TCP requests out to the network. If it is on then all apache TCP requests will be allowed. To turn it on run the following as root:

setsebool -P httpd_can_network_connect 1

If you only need database access (I had this problem before which is why I suspected SELinux here) then it would probably be better to only turn on 'httpd_cna_network_connect'.

The purpose of this policy is that if a hacker was to hijack your apache server they would not be able to get out through the server to the rest of your internal network.

This probably would've been better as a comment but I don't have enough rep..

Sources: https://tag1consulting.com/blog/stop-disabling-selinux https://wiki.centos.org/TipsAndTricks/SelinuxBooleans