Installing compiler on Mac M1 Monterey for Rcpp and other tools

I'm trying to use packages that require Rcpp in R on my M1 Mac, which I was never able to get up and running after purchasing this computer. I updated it to Monterey in the hope that this would fix some installation issues but it hasn't. I tried running the Rcpp check from this page but I get the following error:

> Rcpp::sourceCpp("~/github/helloworld.cpp")

ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
ld: library not found for -lgfortran
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [sourceCpp_4.so] Error 1
clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
  Error 1 occurred building shared library.

I get that it can't "find" gfortran. I installed this release of gfortran for Monterey. When I type which gfortran into Terminal, it returns /opt/homebrew/bin/gfortran. (Maybe this version of gfortran requires Xcode tools that are too new—it says something about 13.2 and when I run clang --version it says 13.0—but I don't see another release of gfortran for Monterey?)

I also appended /opt/homebrew/bin: to PATH in R so it looks like this now:

> Sys.getenv("PATH")

[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"

Other things I checked:

• Xcode command line tools is installed (which clang returns /usr/bin/clang).
• Files ~/.R/Makevars and ~/.Renviron don't exist.

Here's my session info:

R version 4.1.1 (2021-08-10)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Monterey 12.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
[4] Rcpp_1.0.7        

Solution 1:

There are some obstacles on ARM-based Macs (and macOS in general) to compiling R packages containing C/C++ code from their sources, and likewise to using Rcpp. The main issues and their fixes are documented in the R-admin manual, but they are a bit scattered.

Your proximal problem is perhaps easier to solve than the problem you haven't mentioned, which is that the clang toolchain provided with Apple's Command Line Tools doesn't support OpenMP. Not having OpenMP support is an issue if you want to compile a C/C++ program that makes use of multithreading. I'll discuss your proximal problem first, then provide a complete set of instructions that should address everything at once, so that you can obtain a fully enabled toolchain.

LDFLAGS, not PATH

When R (and Rcpp, etc.) compiles C/C++ programs, it determines what compilers, compiler flags, preprocessor flags, etc. to use from make variables set in Makefiles. The correct way to link to a specific installation of gfortran—a library external to R—during compilation is to set the make variable LDFLAGS in a Makefile that R knows about. System-level default values of all make variables used by R are stored in $(R_HOME)/etc/Makeconf. You can query them with R CMD config, like so:

$ R CMD config LDFLAGS --no-user-files

A user can override any Makeconf setting by creating and modifying $(HOME)/.R/Makevars. Since you do not have a Makevars, R is finding LDFLAGS in your Makeconf, and its value there (I'm guessing) contains -L/opt/R/arm64/gfortran/lib, because that is how CRAN configured the build of R that you have installed.

Thus, you might resolve your compilation error by setting LDFLAGS in Makevars or Makeconf so that it includes -L$(F_DIR)/lib and excludes paths that don't exist. (You would replace $(F_DIR) with the path to your gfortran installation.) If you are the only user on your system, then I would recommend modifying Makevars and keeping a pristine Makeconf.

You might also resolve the error by moving your gfortran installation to /opt/R/arm64 and leaving Makevars and Makeconf alone. You should consider trying this first, especially if you are unfamiliar with Makefile syntax.

If neither of these approaches work, then read the next section.

Note: I should emphasize here that the environment variable PATH doesn't really affect how R compiles C/C++ programs apart from telling R where to find make. That is, the paths printed by

$ which clang clang++ gnufortran

aren't necessarily the paths to the C, C++, and Fortran compilers used by R.

Instructions for obtaining a working toolchain on ARM-based Macs

For the benefit of anyone else reading this, I'm going to assume that you are starting from nothing. Feel free to skip steps you've already taken, though you might find a fresh start helpful.

1. Download R from CRAN here and install. Be sure to select the binary built for Apple silicon.

2. Run

   $ xcode-select --install
   

   in Terminal to download and install the latest version of Xcode, which includes Apple's Command Line Tools. You can also download Xcode from your browser here. Earlier versions of Xcode are available here, but I would start with the latest. (There might be a good reason to install the exact version of Xcode than CRAN used to build your R binary, and that version might not be the newest. Not sure.)

3. Install the LLVM clang toolchain with Homebrew. Unlike Apple's clang, it supports OpenMP.

   $ brew update
   $ brew install llvm 
   

   It should unpack into /opt/homebrew/opt.

4. Download a gfortran binary built for your macOS version and architecture here and unpack into /opt/R/arm64.

   $ sudo mkdir -p /opt/R/arm64
   $ sudo tar xvf path/to/gfortran/tarball -C /opt/R/arm64
   

5. Modify a symbolic link in your gfortran installation so that it points to your Command Line Tools SDK. If the link doesn't exist in your installation, then ignore this step.

   $ ln -sfn $(eval xcrun -show-sdk-path) /opt/R/arm64/gfortran/SDK
   

6. Add the following lines to $(HOME)/.R/Makevars, creating the file if necessary:

   LLVM_DIR=/opt/homebrew/opt/llvm
   LIBS_DIR=/opt/R/arm64
   F_DIR=$(LIBS_DIR)/gfortran
   SDK_DIR=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
   
   CC=$(LLVM_DIR)/bin/clang -isysroot $(SDK_DIR) -target arm64-apple-macos11
   CXX=$(LLVM_DIR)/bin/clang++ -isysroot $(SDK_DIR) -target arm64-apple-macos11
   FC=$(F_DIR)/bin/gfortran -mtune=native
   
   CFLAGS=-falign-functions=8 -g -O2 -Wall -pedantic -Wno-implicit-function-declaration
   CXXFLAGS=-g -O2 -Wall -pedantic
   FFLAGS=-g -O2 -Wall -pedantic
   
   SHLIB_OPENMP_CFLAGS=-fopenmp
   SHLIB_OPENMP_CXXFLAGS=-fopenmp
   SHLIB_OPENMP_FFLAGS=-fopenmp
   
   CPPFLAGS=-I$(LLVM_DIR)/include -I$(LIBS_DIR)/include
   LDFLAGS=-L$(LLVM_DIR)/lib -L$(LIBS_DIR)/lib
   

7. Run R and test that you can compile a C/C++ program with OpenMP support. For example:

   if (!requireNamespace("Rcpp", quietly = TRUE)) {
     install.packages("Rcpp")
   }
   Rcpp::sourceCpp(code = '
   #include <Rcpp.h>
   #ifdef _OPENMP
   # include <omp.h>
   #endif
   
   // [[Rcpp::plugins(openmp)]]
   // [[Rcpp::export]]
   int omp_test()
   {
   #ifdef _OPENMP
       return omp_get_max_threads();
   #else
       return 0;
   #endif
   }
   ')
   omp_test()
   

   [1] 8
   

   If you get a compilation error or if omp_test returns 0 after compiling successfully, then something is wrong. Let me know if this happens...

FWIW, these steps do work for me. I am running Big Sur, not Monterey, but it shouldn't matter much. You might need -target arm64-apple-macos12 instead of -target arm64-apple-macos11 in CC and CXX, but I'm not sure. Let me know what works...