Using curl POST with variables defined in bash script functions
When I echo I get this, which runs when I enter it into the terminal
curl -i \
-H "Accept: application/json" \
-H "Content-Type:application/json" \
-X POST --data '{"account":{"email":"[email protected]","screenName":"akdgdtk","type":"NIKE","passwordSettings":{"password":"Starwars1","passwordConfirm":"Starwars1"}},"firstName":"Test","lastName":"User","middleName":"ObiWan","locale":"en_US","registrationSiteId":"520","receiveEmail":"false","dateOfBirth":"1984-12-25","mobileNumber":"9175555555","gender":"male","fuelActivationDate":"2010-10-22","postalCode":"10022","country":"US","city":"Beverton","state":"OR","bio":"This is a test user","jpFirstNameKana":"unsure","jpLastNameKana":"ofthis","height":"80","weight":"175","distanceUnit":"MILES","weightUnit":"POUNDS","heightUnit":"FT/INCHES"}' https://xxx:[email protected]/xxxxx/xxxx/xxxx
But when run in the bash script file, I get this error
curl: (6) Could not resolve host: application; nodename nor servname provided, or not known
curl: (6) Could not resolve host: is; nodename nor servname provided, or not known
curl: (6) Could not resolve host: a; nodename nor servname provided, or not known
curl: (6) Could not resolve host: test; nodename nor servname provided, or not known
curl: (3) [globbing] unmatched close brace/bracket at pos 158
this is the code in the file
curl -i \
-H '"'Accept: application/json'"' \
-H '"'Content-Type:application/json'"' \
-X POST --data "'"'{"account":{"email":"'$email'","screenName":"'$screenName'","type":"'$theType'","passwordSettings":{"password":"'$password'","passwordConfirm":"'$password'"}},"firstName":"'$firstName'","lastName":"'$lastName'","middleName":"'$middleName'","locale":"'$locale'","registrationSiteId":"'$registrationSiteId'","receiveEmail":"'$receiveEmail'","dateOfBirth":"'$dob'","mobileNumber":"'$mobileNumber'","gender":"'$gender'","fuelActivationDate":"'$fuelActivationDate'","postalCode":"'$postalCode'","country":"'$country'","city":"'$city'","state":"'$state'","bio":"'$bio'","jpFirstNameKana":"'$jpFirstNameKana'","jpLastNameKana":"'$jpLastNameKana'","height":"'$height'","weight":"'$weight'","distanceUnit":"MILES","weightUnit":"POUNDS","heightUnit":"FT/INCHES"}'"'" "https://xxx:[email protected]/xxxxx/xxxx/xxxx"
I assume there's an issue with my quotation marks, but I've played with them a lot and I've gotten similar errors. All the variables are defined with different functions in the actual script
Solution 1:
You don't need to pass the quotes enclosing the custom headers to curl. Also, your variables in the middle of the data argument should be quoted.
First, write a function that generates the post data of your script. This saves you from all sort of headaches concerning shell quoting and makes it easier to read an maintain the script than feeding the post data on curl's invocation line as in your attempt:
generate_post_data()
{
cat <<EOF
{
"account": {
"email": "$email",
"screenName": "$screenName",
"type": "$theType",
"passwordSettings": {
"password": "$password",
"passwordConfirm": "$password"
}
},
"firstName": "$firstName",
"lastName": "$lastName",
"middleName": "$middleName",
"locale": "$locale",
"registrationSiteId": "$registrationSiteId",
"receiveEmail": "$receiveEmail",
"dateOfBirth": "$dob",
"mobileNumber": "$mobileNumber",
"gender": "$gender",
"fuelActivationDate": "$fuelActivationDate",
"postalCode": "$postalCode",
"country": "$country",
"city": "$city",
"state": "$state",
"bio": "$bio",
"jpFirstNameKana": "$jpFirstNameKana",
"jpLastNameKana": "$jpLastNameKana",
"height": "$height",
"weight": "$weight",
"distanceUnit": "MILES",
"weightUnit": "POUNDS",
"heightUnit": "FT/INCHES"
}
EOF
}
It is then easy to use that function in the invocation of curl:
curl -i \
-H "Accept: application/json" \
-H "Content-Type:application/json" \
-X POST --data "$(generate_post_data)" "https://xxx:[email protected]/xxxxx/xxxx/xxxx"
This said, here are a few clarifications about shell quoting rules:
The double quotes in the -H arguments (as in -H "foo bar") tell bash to keep what's inside as a single argument (even if it contains spaces).
The single quotes in the --data argument (as in --data 'foo bar') do the same, except they pass all text verbatim (including double quote characters and the dollar sign).
To insert a variable in the middle of a single quoted text, you have to end the single quote, then concatenate with the double quoted variable, and re-open the single quote to continue the text: 'foo bar'"$variable"'more foo'.
Solution 2:
Solution tested with https://httpbin.org/ and inline bash script
1. For variables without spaces in it i.e. 1:
Simply add ' before and after $variable when replacing desired
string
for i in {1..3}; do \
curl -X POST -H "Content-Type: application/json" -d \
'{"number":"'$i'"}' "https://httpbin.org/post"; \
done
2. For input with spaces:
Wrap variable with additional " i.e. "el a":
declare -a arr=("el a" "el b" "el c"); for i in "${arr[@]}"; do \
curl -X POST -H "Content-Type: application/json" -d \
'{"elem":"'"$i"'"}' "https://httpbin.org/post"; \
done
Wow works :)
Solution 3:
Curl can post binary data from a file so I have been using process substitution and taking advantage of file descriptors whenever I need to post something nasty with curl and still want access to the vars in the current shell. Something like:
curl "http://localhost:8080" \
-H "Accept: application/json" \
-H "Content-Type:application/json" \
--data @<(cat <<EOF
{
"me": "$USER",
"something": $(date +%s)
}
EOF
)
This winds up looking like --data @/dev/fd/<some number> which just gets processed like a normal file. Anyway if you wanna see it work locally just run nc -l 8080 first and in a different shell fire off the above command. You will see something like:
POST / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.43.0
Accept: application/json
Content-Type:application/json
Content-Length: 43
{ "me": "username", "something": 1465057519 }
As you can see you can call subshells and whatnot as well as reference vars in the heredoc. Happy hacking hope this helps with the '"'"'""""'''""''.
Solution 4:
We can assign a variable for curl using single quote ' and wrap some other variables in double-single-double quote "'" for substitution inside curl-variable. Then easily we can use that curl-variable which here is MERGE.
Example:
# other variables ...
REF_NAME="new-branch";
# variable for curl using single quote => ' not double "
MERGE='{
"repository": "tmp",
"command": "git",
"args": [
"pull",
"origin",
"'"$REF_NAME"'"
],
"options": {
"cwd": "/home/git/tmp"
}
}';
notice this line:
"'"$REF_NAME"'"
so we can use this bash variable $MERGE and calling curl as usual:
curl -s -X POST localhost:1365/M -H 'Content-Type: application/json' --data "$MERGE"
Solution 5:
A few years late but this might help someone if you are using eval or backtick substitution:
postDataJson="{\"guid\":\"$guid\",\"auth_token\":\"$token\"}"
Using sed to strip quotes from beginning and end of response
$(curl --silent -H "Content-Type: application/json" https://${target_host}/runs/get-work -d ${postDataJson} | sed -e 's/^"//' -e 's/"$//')