Python requests POST file with multipart form and cookies

I need to upload a file with some information about the file using cookies. A command line as below works fine:

curl -v -X -POST -F file=@"path_to_file/my_file.xml; type=text/xml" -F rowData='{"fileVersion":"1.0", "owner":"xyz", "fileName":"my_file.xml"}' https://x.x.x.x:xxxx/path_to_receiving_point -H 'Cookie: my_cookie_goes_here'

I want to convert that command to Python using requests.

import requests

#First I get the cookies with a POST
#please assume this part works and I get a <RequestsCookieJar> output below
req = requests.POST(...)
cookies = req.cookies

#Then I construct the actual request to upload file
address = "https://x.x.x.x:xxxx/path_to_receiving_point"
rowData = {"fileVersion":"1.0", "owner":"xyz", "fileName":"my_file.xml"}
#files = {'file': ("path_to_file/my_file.xml", open("path_to_file/my_file.xml", 'rb'), 'type=text/xml', rowData)}
files = {'file': ('my_file.xml', open("path_to_file/my_file.xml", 'rb'), 'type=text/xml', rowData)}

upload_req = requests.post(address,  files=files, cookies=cookies)

print(upload_req.json())

Is this the correct way? Because I receive this error:

raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

For the cookies, this is how it looks like:

<RequestsCookieJar[<Cookie userData=%7B%22access_token%22%3A%2288fcdfb8-451f-4607-9517-9ecae16c9285%22%2C%22role%22%3A%5B1%2C5%2C32%5D%2C%22authType%22%3A%22LOCAL%22%2C%22user%22%3A%22admin%22%2C%22password_regex%22%3A%22%28%28%3F%3D.*%5BA-Z%5D%29%28%3F%3D.*%5Ba-z%5D%29%28%3F%3D.*%5C%5Cd%29%28%3F%3D.*%5B__special%5D%29.%7B15%2C%7D%29%22%2C%22special_chars%22%3A%22%21%5C%22%23%24%25%26%27%28%29*%2B%2C-.%2F%3A%3B%3C%3D%3E%3F%40%5B%5D%5E_%60+%7B%5C%5C%7C%7D%7E%22%7D for x.x.x.x/path_to_login/login>]>

Update: just to confirm I am able to verify the cookies part is correct. My suspicion now is that the files dict is not in the right format. If someone could advise on that?

Update 2: I edited the files dict to exclude the first file name as above. Now the error I got is:

return request('post', url, data=data, json=json, **kwargs)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/requests/sessions.py", line 528, in request
    prep = self.prepare_request(req)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/requests/sessions.py", line 456, in prepare_request
    p.prepare(
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/requests/models.py", line 319, in prepare
    self.prepare_body(data, files, json)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/requests/models.py", line 512, in prepare_body
    (body, content_type) = self._encode_files(files, data)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/requests/models.py", line 166, in _encode_files
    rf.make_multipart(content_type=ft)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/urllib3/fields.py", line 268, in make_multipart
    self._render_parts(
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/urllib3/fields.py", line 226, in _render_parts
    parts.append(self._render_part(name, value))
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/urllib3/fields.py", line 206, in _render_part
    return self.header_formatter(name, value)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/urllib3/fields.py", line 117, in format_header_param_html5
    value = _replace_multiple(value, _HTML5_REPLACEMENTS)
  File "/home/username/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/urllib3/fields.py", line 90, in _replace_multiple
    result = pattern.sub(replacer, value)
TypeError: expected string or bytes-like object

This error seems to say the XML file is not read into string or ```bytes-like```` object.

Update 3:

So I digged further into this file

/home/usernme/.local/share/virtualenvs/proj-zrz-666L/lib/python3.9/site-packages/urllib3/fields.py

particularly, function:

def _replace_multiple(value, needles_and_replacements):
    def replacer(match):
        return needles_and_replacements[match.group(0)]

    pattern = re.compile(
        r"|".join([re.escape(needle) for needle in needles_and_replacements.keys()])
    )
    result = pattern.sub(replacer, value)
    return result

Apparently, this function takes an argument value which is supposed to be string or byte-type. But in this POST, it is a tuple. Why such an error?

Solution 1:

You can deserialize the response with upload_req.json() only when your service is serializing the data with some JSON serializer. clearly, your response Content-type is not JSON

Before directly writing print(upload_req.json()) you can simply check the response with print(upload_req.text) or print(upload_req.__dict__) and this way you can see everything present in the response including response headers and then you can make a decision on parsing it properly.

Python requests POST file with multipart form and cookies

Solution 1:

Related

Recent Posts