Android - accessing files in native C/C++ code with Google Scoped Storage API

Update May 19, 2020: just discovered: the READ_EXTERNAL_STORAGE permission lets you read files, but not list directory contents, when running on Android 11 and targeting API 30 (or higher in the future). I submitted it as a bug and just got "This is working as intended" reply (https://issuetracker.google.com/issues/156660903). If anyone cares, please comment there, and also at their "survey": https://google.qualtrics.com/jfe/form/SV_9HOzzyeCIEw0ij3?Source=scoped-storage I have no idea how to proceed developing apps on Android with all these limitations.

Update May 17 2020: Google has finally conceded and is permitting READ_EXTERNAL_STORAGE permission in Android 11 and beyond. After spending months in converting my apps to Storage Access Framework (SAF), I now take a week or two to convert it back, at least the reading files part, to regular file access... Thank you, Google! Thank you sarcastically, for the lost time and effort, and thank you sincerely, for at least partially understanding our point!

So, my previous answer listed below won't be needed anymore, sigh of relief!

After wasting another good day of my life on the Android "Scoped Storage" B.S., I found a solution that works, without modifying the source of 3rd party native libraries, provided that one has the source of these libraries and can build them.

My (very unsatisfactory) solution is: add the following option to C/C++ compile command:

-include "[some/path/]idiocy_fopen_fd.h"

and the idiocy_fopen_fd.h is as follows, as you can see, every call to regular fopen() is replaced by the idiocy_fopen_fd() code, which checks if the file name starts with "/proc/self/fd/", and if so, extracts the file descriptor number and calls fdopen() instead of fopen()... If someone has a better solution, preferably which would work also when you don't have the source code of the 3rd party libs, please share.

#ifndef fopen_fd

#include <stdio.h>
#include <string.h>
#include <unistd.h> // for dup()

#ifdef __cplusplus
extern "C" {
#endif

inline FILE* idiocy_fopen_fd(const char* fname, const char * mode) {
  if (strstr(fname, "/proc/self/fd/") == fname) {
    int fd = atoi(fname + 14);
    if (fd != 0) {
      // Why dup(fd) below: if we called fdopen() on the
      // original fd value, and the native code closes
      // and tries re-open that file, the second fdopen(fd)
      // would fail, return NULL - after closing the
      // original fd received from Android, it's no longer valid.
      FILE *fp = fdopen(dup(fd), mode);
      // Why rewind(fp): if the native code closes and 
      // opens again the file, the file read/write position
      // would not change, because with dup(fd) it's still
      // the same file...
      rewind(fp);
      return fp;
    }
  }
  return fopen(fname, mode);
}
// Note that the above leaves the original file descriptor
// opened when finished - close parcelFileDescriptor in
// Java/Kotlin when your native code returns!

#ifdef __cplusplus
}
#endif

#define fopen idiocy_fopen_fd

#endif

Тo anyone contemplating switching their app to SAF, I post my app's monthly average ratings chart from Google Play developer console. In January 2020 I released the first version of my app using SAF instead of regular file access, see attached image. As more and more users update, the ratings go down, a lot of comments that I destroyed a good app. The app even now has well over 1 million active installs.

enter image description here

Update May 17 2020: Google has finally conceded and is permitting READ_EXTERNAL_STORAGE permission in Android 11 and beyond. After spending months in converting my apps to Storage Access Framework (SAF), I now take a week or two to convert it back, at least the reading files part, to regular file access... Thank you, Google! Thank you sarcastically, for the lost time and effort, and thank you sincerely, for at least partially understanding our point!

This is not the end of the "scoped storage" nightmare. Google is "improving" it again in Android 11 (see https://developer.android.com/preview/privacy/storage) - all we did so far to make it work may soon be thrown out to garbage.

Please consider commenting on and supporting my request to not limit Scoped Storage access to Download directory in Android 11, and/or improve the system "Files" interface: https://issuetracker.google.com/issues/151759806

I'm seriously considering giving up further development for the Android platform... It seems to me that arrogant, controlling "parents" took over Android system evolution, who think the rest of us are little children, and they need to install safety gates all around at our houses, seriously limiting our freedom of movement. So sad.

Greg

Android - accessing files in native C/C++ code with Google Scoped Storage API

Related

Recent Posts