Unexpected error on UrlFetchApp.fetch in Google Apps Script using basic authentication

I have the following code in Google Apps Script which retrieves CSV data from a webpage via HTTP using basic authentication and places it into a spreadsheet:

CSVImport.gs

function parseCSVtoSheet(sheetName, url)
{
  // Credentials
  var username = "myusername";
  var password = "mypassword";
  var header = "Basic " + Utilities.base64Encode(username + ":" + password);
  
  // Setting the authorization header for basic HTTP authentication
  var options = {
    "headers": {
      "Authorization": header
    }
  };
  
  // Getting the ID of the sheet with the name passed as parameter
  var spreadsheet = SpreadsheetApp.getActive();
  var sheet = spreadsheet.getSheetByName(sheetName);
  var sheetId = sheet.getSheetId();
  
  // Getting the CSV data and placing it into the spreadsheet
  var csvContent = UrlFetchApp.fetch(url, options).getContentText();
  var resource = {requests: [{pasteData: {data: csvContent, coordinate: {sheetId: sheetId}, delimiter: ","}}]};
  Sheets.Spreadsheets.batchUpdate(resource, spreadsheet.getId());
}

This has been working up until recently where randomly I get the following error on the UrlFetchApp.fetch line:

Exception: Unexpected error: http://www.myurl.com/data/myfile.csv (line 21, file "CSVImport")

I have tried:

• Putting the credentials directly in the URL instead of in an Authorization header (I received a different error saying "Login information disallowed").
• Encoding the credentials to base64 right when I pass it into the headers object (didn't work, same error).
• Removing authentication altogether (predictably I received a 401 response from the HTTP page).

I'm not sure what else to try and why this randomly broke down all of a sudden. Any advice?

Solution 1:

This is related to a new bug, see here

Many users are affected, I recommend you to "star" the issue to increase visibility and hopefully accelerate the process.

Solution 2:

I had the same situation. At that time, I could noticed that when the built-in function of Google Spreadsheet is used for the URL, the values can be retrieved. In that case, as the current workaround, I used the following flow.

1. Put a formula of =IMPORTDATA(URL).
2. Retrieve the values from the sheet.

When above flow is reflected to your URL of http://www.myurl.com/data/myfile.csv, it becomes as follows.

About basic authorization for URL:

When I saw your script, I confirmed that you are using the basic authorization. In this case, the user name and password can be used for the URL like http://username:[email protected]/data/myfile.csv.

From your script, when the values of username and password are myusername and mypassword, respectively, you can use the URL as http://myusername:[email protected]/data/myfile.csv.

Here, there is an important point. If the specific characters are included in username and password, please do the url encode for them.

Sample script:

function myFunction() {
  const url = "http://myusername:[email protected]/data/myfile.csv";  // This is your URL.

  // Retrieve the values from URL.
  var spreadsheet = SpreadsheetApp.getActive();
  var sheet = spreadsheet.getSheetByName(sheetName);
  sheet.clear();
  var range = sheet.getRange("A1");
  range.setFormula(`=IMPORTDATA("${url}")`);
  
  // Retrieve the values from sheet to an array.
  SpreadsheetApp.flush();
  var values = sheet.getDataRange().getValues();
  range.clear();
  console.log(values)
}

• When above script is run, the values from the URL are put to the sheet, and the values are retrieved as 2 dimensional array for values. If you want to leave only values without the formula, I think that you can copy and paste the values.
• In this answer, I used IMPORTDATA. But for each situation, other functions might be suitable. In that case, please check them.

Note:

• This is the current workaround. So when this issue was removed, I think that you can use your original script.

References:

• IMPORTDATA
• setFormula()

Solution 3:

Disable Chrome V8 Runtime Engine until Google fix this.

To disable: From Menu click on Run > Disable new Apps Script runtime powered by Chrome V8