Python requests get doesn't return anything

Solution 1:

Found the issue:

You need to have a User-Agent header:

import requests

if __name__ == '__main__':
    req = requests.Session()
    link12month = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"
    response = req.get(link12month, headers={'Accept': 'application/xml; charset=utf-8','User-Agent':'foo'})
    print(response.status_code, response.text)

You can also simplify your code a bit here as well:

import requests

if __name__ == '__main__':
    link12month = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"
    response = requests.get(link12month, headers={'Accept': 'application/xml; charset=utf-8','User-Agent':'foo'})
    print(response.status_code, response.text)

Make sure you verify if you can as well. If you want JSON, remove the 'Accept' header.

You can also simplify the URL a lot by adding the parameters as a dictionary, check out the docs here for the params argument https://www.w3schools.com/python/ref_requests_get.asp

Solution 2:

1.tried to get the url with firefox, got a valid response

2.tried to directly use requests library

import requests
uri = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"
session = requests.Session()
x = session.get(uri, timeout=2)

result: timeout exeption => server does not respond

3.tried to set header used by firefox

import requests
uri = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"

session = requests.Session()
session.headers.update({"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0"})
response = session.get(uri, timeout=2)

print(response.content)

result looks like a valid xml response:

b'<SearchReportListDto xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Codal.Services.Model.Dto.Search"><Letters><CodalLetterHeaderDto><AttachmentUrl>/Reports/Attachment.aspx?LetterSerial=cmjQJt%2bh5QxBpUMcT8rfPA%3d%3d</AttachmentUrl><CompanyName>\xd9\xbe\xd8\xa7\xd9\x84\xd8\xa7\xdb\x8c\xd8\xb4 \xd9\x86\xd9\x81\xd8\xaa \xd8\xa7\xd8\xb5\xd9\x81\xd9\x87\xd8\xa7\xd9\x86</CompanyName><ExcelUrl>https://excel.codal.ir/service/Excel/GetAll/cmjQJt%2bh5QxBpUMcT8rfPA%3d%3d/0</ExcelUrl><HasAttachment>true</HasAttachment><HasHtml>true</HasHtml><HasXbrl>false</HasXbrl><IsEstimate>false</IsEstimate><LetterCode>\xd9\x86-\xdb\xb1\xdb\xb0</LetterCode><PdfUrl>DownloadFile.aspx?hs=cmjQJt%2bh5QxBpUMcT8rfPA%3d%3d&amp;ft=1005&amp;let=6</PdfUrl><PublishDateTime>\xdb\xb1\xdb\xb4\xdb\xb0\xdb\xb0/\xdb\x...