Sending files using POST with HttpURLConnection
Since the Android developers recommend to use the HttpURLConnection class, I was wondering if anyone can provide me with a good example on how to send a bitmap "file" (actually an in-memory stream) via POST to an Apache HTTP server. I'm not interested in cookies or authentication or anything complicated, but I just want to have a reliable and logic implementation. All the examples that I've seen around here look more like "let's try this and maybe it works".
Right now, I have this code:
URL url;
HttpURLConnection urlConnection = null;
try {
url = new URL("http://example.com/server.cgi");
urlConnection = (HttpURLConnection) url.openConnection();
} catch (Exception e) {
this.showDialog(getApplicationContext(), e.getMessage());
}
finally {
if (urlConnection != null)
{
urlConnection.disconnect();
}
}
where showDialog should just display an AlertDialog (in case of an invalid URL?).
Now, let's say that I generate a bitmap like so: Bitmap image = this.getBitmap() inside a control derived from View and I want to send it via POST. What would be the proper procedure to achieve such a thing? What classes do I need to use? Can I use HttpPost like in this example? If so, how would I construct the InputStreamEntity for my bitmap? I would find it revolting to be required to first store the bitmap in a file on the device.
I should also mention that I really need to send every unaltered pixel of the original bitmap to the server, so I can't convert it to JPEG.
I have no idea why the HttpURLConnection class does not provide any means to send files without having to compose the file wrapper manually. Here's what I ended up doing, but if someone knows a better solution, please let me know.
Input data:
Bitmap bitmap = myView.getBitmap();
Static stuff:
String attachmentName = "bitmap";
String attachmentFileName = "bitmap.bmp";
String crlf = "\r\n";
String twoHyphens = "--";
String boundary = "*****";
Setup the request:
HttpURLConnection httpUrlConnection = null;
URL url = new URL("http://example.com/server.cgi");
httpUrlConnection = (HttpURLConnection) url.openConnection();
httpUrlConnection.setUseCaches(false);
httpUrlConnection.setDoOutput(true);
httpUrlConnection.setRequestMethod("POST");
httpUrlConnection.setRequestProperty("Connection", "Keep-Alive");
httpUrlConnection.setRequestProperty("Cache-Control", "no-cache");
httpUrlConnection.setRequestProperty(
"Content-Type", "multipart/form-data;boundary=" + this.boundary);
Start content wrapper:
DataOutputStream request = new DataOutputStream(
httpUrlConnection.getOutputStream());
request.writeBytes(this.twoHyphens + this.boundary + this.crlf);
request.writeBytes("Content-Disposition: form-data; name=\"" +
this.attachmentName + "\";filename=\"" +
this.attachmentFileName + "\"" + this.crlf);
request.writeBytes(this.crlf);
Convert Bitmap to ByteBuffer:
//I want to send only 8 bit black & white bitmaps
byte[] pixels = new byte[bitmap.getWidth() * bitmap.getHeight()];
for (int i = 0; i < bitmap.getWidth(); ++i) {
for (int j = 0; j < bitmap.getHeight(); ++j) {
//we're interested only in the MSB of the first byte,
//since the other 3 bytes are identical for B&W images
pixels[i + j] = (byte) ((bitmap.getPixel(i, j) & 0x80) >> 7);
}
}
request.write(pixels);
End content wrapper:
request.writeBytes(this.crlf);
request.writeBytes(this.twoHyphens + this.boundary +
this.twoHyphens + this.crlf);
Flush output buffer:
request.flush();
request.close();
Get response:
InputStream responseStream = new
BufferedInputStream(httpUrlConnection.getInputStream());
BufferedReader responseStreamReader =
new BufferedReader(new InputStreamReader(responseStream));
String line = "";
StringBuilder stringBuilder = new StringBuilder();
while ((line = responseStreamReader.readLine()) != null) {
stringBuilder.append(line).append("\n");
}
responseStreamReader.close();
String response = stringBuilder.toString();
Close response stream:
responseStream.close();
Close the connection:
httpUrlConnection.disconnect();
PS: Of course I had to wrap the request in private class AsyncUploadBitmaps extends AsyncTask<Bitmap, Void, String>, in order to make the Android platform happy, because it doesn't like to have network requests on the main thread.
I actually found a better way to send files using HttpURLConnection using MultipartEntity
private static String multipost(String urlString, MultipartEntity reqEntity) {
try {
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setReadTimeout(10000);
conn.setConnectTimeout(15000);
conn.setRequestMethod("POST");
conn.setUseCaches(false);
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setRequestProperty("Connection", "Keep-Alive");
conn.addRequestProperty("Content-length", reqEntity.getContentLength()+"");
conn.addRequestProperty(reqEntity.getContentType().getName(), reqEntity.getContentType().getValue());
OutputStream os = conn.getOutputStream();
reqEntity.writeTo(conn.getOutputStream());
os.close();
conn.connect();
if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {
return readStream(conn.getInputStream());
}
} catch (Exception e) {
Log.e(TAG, "multipart post error " + e + "(" + urlString + ")");
}
return null;
}
private static String readStream(InputStream in) {
BufferedReader reader = null;
StringBuilder builder = new StringBuilder();
try {
reader = new BufferedReader(new InputStreamReader(in));
String line = "";
while ((line = reader.readLine()) != null) {
builder.append(line);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return builder.toString();
}
Assuming you are uploading an image with bitmap data:
Bitmap bitmap = ...;
String filename = "filename.png";
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, bos);
ContentBody contentPart = new ByteArrayBody(bos.toByteArray(), filename);
MultipartEntity reqEntity = new MultipartEntity(HttpMultipartMode.BROWSER_COMPATIBLE);
reqEntity.addPart("picture", contentPart);
String response = multipost("http://server.com", reqEntity);
And Voila! Your post data will contain picture field along with the filename and path on your server.
To upload file on server with some parameter using MultipartUtility in simple way.
MultipartUtility.java
public class MultipartUtility {
private final String boundary;
private static final String LINE_FEED = "\r\n";
private HttpURLConnection httpConn;
private String charset;
private OutputStream outputStream;
private PrintWriter writer;
/**
* This constructor initializes a new HTTP POST request with content type
* is set to multipart/form-data
*
* @param requestURL
* @param charset
* @throws IOException
*/
public MultipartUtility(String requestURL, String charset)
throws IOException {
this.charset = charset;
// creates a unique boundary based on time stamp
boundary = "===" + System.currentTimeMillis() + "===";
URL url = new URL(requestURL);
Log.e("URL", "URL : " + requestURL.toString());
httpConn = (HttpURLConnection) url.openConnection();
httpConn.setUseCaches(false);
httpConn.setDoOutput(true); // indicates POST method
httpConn.setDoInput(true);
httpConn.setRequestProperty("Content-Type",
"multipart/form-data; boundary=" + boundary);
httpConn.setRequestProperty("User-Agent", "CodeJava Agent");
httpConn.setRequestProperty("Test", "Bonjour");
outputStream = httpConn.getOutputStream();
writer = new PrintWriter(new OutputStreamWriter(outputStream, charset),
true);
}
/**
* Adds a form field to the request
*
* @param name field name
* @param value field value
*/
public void addFormField(String name, String value) {
writer.append("--" + boundary).append(LINE_FEED);
writer.append("Content-Disposition: form-data; name=\"" + name + "\"")
.append(LINE_FEED);
writer.append("Content-Type: text/plain; charset=" + charset).append(
LINE_FEED);
writer.append(LINE_FEED);
writer.append(value).append(LINE_FEED);
writer.flush();
}
/**
* Adds a upload file section to the request
*
* @param fieldName name attribute in <input type="file" name="..." />
* @param uploadFile a File to be uploaded
* @throws IOException
*/
public void addFilePart(String fieldName, File uploadFile)
throws IOException {
String fileName = uploadFile.getName();
writer.append("--" + boundary).append(LINE_FEED);
writer.append(
"Content-Disposition: form-data; name=\"" + fieldName
+ "\"; filename=\"" + fileName + "\"")
.append(LINE_FEED);
writer.append(
"Content-Type: "
+ URLConnection.guessContentTypeFromName(fileName))
.append(LINE_FEED);
writer.append("Content-Transfer-Encoding: binary").append(LINE_FEED);
writer.append(LINE_FEED);
writer.flush();
FileInputStream inputStream = new FileInputStream(uploadFile);
byte[] buffer = new byte[4096];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, bytesRead);
}
outputStream.flush();
inputStream.close();
writer.append(LINE_FEED);
writer.flush();
}
/**
* Adds a header field to the request.
*
* @param name - name of the header field
* @param value - value of the header field
*/
public void addHeaderField(String name, String value) {
writer.append(name + ": " + value).append(LINE_FEED);
writer.flush();
}
/**
* Completes the request and receives response from the server.
*
* @return a list of Strings as response in case the server returned
* status OK, otherwise an exception is thrown.
* @throws IOException
*/
public String finish() throws IOException {
StringBuffer response = new StringBuffer();
writer.append(LINE_FEED).flush();
writer.append("--" + boundary + "--").append(LINE_FEED);
writer.close();
// checks server's status code first
int status = httpConn.getResponseCode();
if (status == HttpURLConnection.HTTP_OK) {
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
response.append(line);
}
reader.close();
httpConn.disconnect();
} else {
throw new IOException("Server returned non-OK status: " + status);
}
return response.toString();
}
}
To upload you file along with parameters.
NOTE : put this code below in non-ui-thread to get response.
String charset = "UTF-8";
String requestURL = "YOUR_URL";
MultipartUtility multipart = new MultipartUtility(requestURL, charset);
multipart.addFormField("param_name_1", "param_value");
multipart.addFormField("param_name_2", "param_value");
multipart.addFormField("param_name_3", "param_value");
multipart.addFilePart("file_param_1", new File(file_path));
String response = multipart.finish(); // response from server.
The solution of Jaydipsinh Zala didn't work for me, I don't know why but it seems to be close to the solution.
So merging this one with the great solution and explanation of Mihai Todor, the result is this class that currently works for me. If it helps someone:
MultipartUtility2V.java
import java.io.*;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.file.Files;
public class MultipartUtilityV2 {
private HttpURLConnection httpConn;
private DataOutputStream request;
private final String boundary = "*****";
private final String crlf = "\r\n";
private final String twoHyphens = "--";
/**
* This constructor initializes a new HTTP POST request with content type
* is set to multipart/form-data
*
* @param requestURL
* @throws IOException
*/
public MultipartUtilityV2(String requestURL)
throws IOException {
// creates a unique boundary based on time stamp
URL url = new URL(requestURL);
httpConn = (HttpURLConnection) url.openConnection();
httpConn.setUseCaches(false);
httpConn.setDoOutput(true); // indicates POST method
httpConn.setDoInput(true);
httpConn.setRequestMethod("POST");
httpConn.setRequestProperty("Connection", "Keep-Alive");
httpConn.setRequestProperty("Cache-Control", "no-cache");
httpConn.setRequestProperty(
"Content-Type", "multipart/form-data;boundary=" + this.boundary);
request = new DataOutputStream(httpConn.getOutputStream());
}
/**
* Adds a form field to the request
*
* @param name field name
* @param value field value
*/
public void addFormField(String name, String value)throws IOException {
request.writeBytes(this.twoHyphens + this.boundary + this.crlf);
request.writeBytes("Content-Disposition: form-data; name=\"" + name + "\""+ this.crlf);
request.writeBytes("Content-Type: text/plain; charset=UTF-8" + this.crlf);
request.writeBytes(this.crlf);
request.writeBytes(value+ this.crlf);
request.flush();
}
/**
* Adds a upload file section to the request
*
* @param fieldName name attribute in <input type="file" name="..." />
* @param uploadFile a File to be uploaded
* @throws IOException
*/
public void addFilePart(String fieldName, File uploadFile)
throws IOException {
String fileName = uploadFile.getName();
request.writeBytes(this.twoHyphens + this.boundary + this.crlf);
request.writeBytes("Content-Disposition: form-data; name=\"" +
fieldName + "\";filename=\"" +
fileName + "\"" + this.crlf);
request.writeBytes(this.crlf);
byte[] bytes = Files.readAllBytes(uploadFile.toPath());
request.write(bytes);
}
/**
* Completes the request and receives response from the server.
*
* @return a list of Strings as response in case the server returned
* status OK, otherwise an exception is thrown.
* @throws IOException
*/
public String finish() throws IOException {
String response ="";
request.writeBytes(this.crlf);
request.writeBytes(this.twoHyphens + this.boundary +
this.twoHyphens + this.crlf);
request.flush();
request.close();
// checks server's status code first
int status = httpConn.getResponseCode();
if (status == HttpURLConnection.HTTP_OK) {
InputStream responseStream = new
BufferedInputStream(httpConn.getInputStream());
BufferedReader responseStreamReader =
new BufferedReader(new InputStreamReader(responseStream));
String line = "";
StringBuilder stringBuilder = new StringBuilder();
while ((line = responseStreamReader.readLine()) != null) {
stringBuilder.append(line).append("\n");
}
responseStreamReader.close();
response = stringBuilder.toString();
httpConn.disconnect();
} else {
throw new IOException("Server returned non-OK status: " + status);
}
return response;
}
}
This answer https://stackoverflow.com/a/33149413/6481542 got me 90% of the way with uploading large files to a development Django server, but I had to use setFixedLengthStreamingMode to make it worked. That requires setting the Content-Length before writing the content, thus requiring a fairly significant rewrite of the above answer. Here's my end result
public class MultipartLargeUtility {
private final String boundary;
private static final String LINE_FEED = "\r\n";
private HttpURLConnection httpConn;
private String charset;
private OutputStream outputStream;
private PrintWriter writer;
private final int maxBufferSize = 4096;
private long contentLength = 0;
private URL url;
private List<FormField> fields;
private List<FilePart> files;
private class FormField {
public String name;
public String value;
public FormField(String name, String value) {
this.name = name;
this.value = value;
}
}
private class FilePart {
public String fieldName;
public File uploadFile;
public FilePart(String fieldName, File uploadFile) {
this.fieldName = fieldName;
this.uploadFile = uploadFile;
}
}
/**
* This constructor initializes a new HTTP POST request with content type
* is set to multipart/form-data
*
* @param requestURL
* @param charset
* @throws IOException
*/
public MultipartLargeUtility(String requestURL, String charset, boolean requireCSRF)
throws IOException {
this.charset = charset;
// creates a unique boundary based on time stamp
boundary = "===" + System.currentTimeMillis() + "===";
url = new URL(requestURL);
fields = new ArrayList<>();
files = new ArrayList<>();
if (requireCSRF) {
getCSRF();
}
}
/**
* Adds a form field to the request
*
* @param name field name
* @param value field value
*/
public void addFormField(String name, String value)
throws UnsupportedEncodingException {
String fieldContent = "--" + boundary + LINE_FEED;
fieldContent += "Content-Disposition: form-data; name=\"" + name + "\"" + LINE_FEED;
fieldContent += "Content-Type: text/plain; charset=" + charset + LINE_FEED;
fieldContent += LINE_FEED;
fieldContent += value + LINE_FEED;
contentLength += fieldContent.getBytes(charset).length;
fields.add(new FormField(name, value));
}
/**
* Adds a upload file section to the request
*
* @param fieldName name attribute in <input type="file" name="..." />
* @param uploadFile a File to be uploaded
* @throws IOException
*/
public void addFilePart(String fieldName, File uploadFile)
throws IOException {
String fileName = uploadFile.getName();
String fieldContent = "--" + boundary + LINE_FEED;
fieldContent += "Content-Disposition: form-data; name=\"" + fieldName
+ "\"; filename=\"" + fileName + "\"" + LINE_FEED;
fieldContent += "Content-Type: "
+ URLConnection.guessContentTypeFromName(fileName) + LINE_FEED;
fieldContent += "Content-Transfer-Encoding: binary" + LINE_FEED;
fieldContent += LINE_FEED;
// file content would go here
fieldContent += LINE_FEED;
contentLength += fieldContent.getBytes(charset).length;
contentLength += uploadFile.length();
files.add(new FilePart(fieldName, uploadFile));
}
/**
* Adds a header field to the request.
*
* @param name - name of the header field
* @param value - value of the header field
*/
//public void addHeaderField(String name, String value) {
// writer.append(name + ": " + value).append(LINE_FEED);
// writer.flush();
//}
/**
* Completes the request and receives response from the server.
*
* @return a list of Strings as response in case the server returned
* status OK, otherwise an exception is thrown.
* @throws IOException
*/
public List<String> finish() throws IOException {
List<String> response = new ArrayList<String>();
String content = "--" + boundary + "--" + LINE_FEED;
contentLength += content.getBytes(charset).length;
if (!openConnection()) {
return response;
}
writeContent();
// checks server's status code first
int status = httpConn.getResponseCode();
if (status == HttpURLConnection.HTTP_OK) {
BufferedReader reader = new BufferedReader(new InputStreamReader(
httpConn.getInputStream()));
String line = null;
while ((line = reader.readLine()) != null) {
response.add(line);
}
reader.close();
httpConn.disconnect();
} else {
throw new IOException("Server returned non-OK status: " + status);
}
return response;
}
private boolean getCSRF()
throws IOException {
/// First, need to get CSRF token from server
/// Use GET request to get the token
CookieManager cookieManager = new CookieManager();
CookieHandler.setDefault(cookieManager);
HttpURLConnection conn = null;
conn = (HttpURLConnection) url.openConnection();
conn.setUseCaches(false); // Don't use a Cached Copy
conn.setRequestMethod("GET");
conn.setRequestProperty("Connection", "Keep-Alive");
conn.getContent();
conn.disconnect();
/// parse the returned object for the CSRF token
CookieStore cookieJar = cookieManager.getCookieStore();
List<HttpCookie> cookies = cookieJar.getCookies();
String csrf = null;
for (HttpCookie cookie : cookies) {
Log.d("cookie", "" + cookie);
if (cookie.getName().equals("csrftoken")) {
csrf = cookie.getValue();
break;
}
}
if (csrf == null) {
Log.d(TAG, "Unable to get CSRF");
return false;
}
Log.d(TAG, "Received cookie: " + csrf);
addFormField("csrfmiddlewaretoken", csrf);
return true;
}
private boolean openConnection()
throws IOException {
httpConn = (HttpURLConnection) url.openConnection();
httpConn.setUseCaches(false);
httpConn.setDoOutput(true); // indicates POST method
httpConn.setDoInput(true);
//httpConn.setRequestProperty("Accept-Encoding", "identity");
httpConn.setFixedLengthStreamingMode(contentLength);
httpConn.setRequestProperty("Connection", "Keep-Alive");
httpConn.setRequestProperty("Content-Type",
"multipart/form-data; boundary=" + boundary);
outputStream = new BufferedOutputStream(httpConn.getOutputStream());
writer = new PrintWriter(new OutputStreamWriter(outputStream, charset),
true);
return true;
}
private void writeContent()
throws IOException {
for (FormField field : fields) {
writer.append("--" + boundary).append(LINE_FEED);
writer.append("Content-Disposition: form-data; name=\"" + field.name + "\"")
.append(LINE_FEED);
writer.append("Content-Type: text/plain; charset=" + charset).append(
LINE_FEED);
writer.append(LINE_FEED);
writer.append(field.value).append(LINE_FEED);
writer.flush();
}
for (FilePart filePart : files) {
String fileName = filePart.uploadFile.getName();
writer.append("--" + boundary).append(LINE_FEED);
writer.append(
"Content-Disposition: form-data; name=\"" + filePart.fieldName
+ "\"; filename=\"" + fileName + "\"")
.append(LINE_FEED);
writer.append(
"Content-Type: "
+ URLConnection.guessContentTypeFromName(fileName))
.append(LINE_FEED);
writer.append("Content-Transfer-Encoding: binary").append(LINE_FEED);
writer.append(LINE_FEED);
writer.flush();
FileInputStream inputStream = new FileInputStream(filePart.uploadFile);
int bufferSize = Math.min(inputStream.available(), maxBufferSize);
byte[] buffer = new byte[bufferSize];
int bytesRead = -1;
while ((bytesRead = inputStream.read(buffer, 0, bufferSize)) != -1) {
outputStream.write(buffer, 0, bytesRead);
}
outputStream.flush();
inputStream.close();
writer.append(LINE_FEED);
writer.flush();
}
writer.append("--" + boundary + "--").append(LINE_FEED);
writer.close();
}
}
Usage is largely the same as in the above answer, but I've included CSRF support that Django uses by default with forms
boolean useCSRF = true;
MultipartLargeUtility multipart = new MultipartLargeUtility(url, "UTF-8",useCSRF);
multipart.addFormField("param1","value");
multipart.addFilePart("filefield",new File("/path/to/file"));
List<String> response = multipart.finish();
Log.w(TAG,"SERVER REPLIED:");
for(String line : response) {
Log.w(TAG, "Upload Files Response:::" + line);
}