rJava load error in RStudio/R after "upgrading" to OSX Yosemite

I had the same problem and went through the same steps as you. The final step to allow starting RStudio through Finder/Spotlight was to link libjvm.dylib to /usr/local/lib:

sudo ln -f -s $(/usr/libexec/java_home)/jre/lib/server/libjvm.dylib /usr/local/lib

-f flag is added to force overwriting existing file/link

On OSX El Capitan 10.11, the user doesn't have permission to write to /usr/lib. So instead, use the following command:

sudo ln -s $(/usr/libexec/java_home)/jre/lib/server/libjvm.dylib /usr/local/lib

You can load the libjvm.dylib explictly, by run like

dyn.load('/Library/Java/JavaVirtualMachines/jdk1.8.0_66.jdk/Contents/Home/jre/lib/server/libjvm.dylib')
library(rJava)

On macOS High Sierra (10.13.1) and Java Version 9 you have to use a slightly different JVM path (notice the missing jre folder in the path compared to the instructions for earlier Java versions):

sudo ln -f -s $(/usr/libexec/java_home)/lib/server/libjvm.dylib /usr/local/lib

You also have to notify R about the JVM:

MY_R_VERSION=$(Rscript -e "cat(with(R.version, sprintf('%s.%s', major, substring(minor, 1, 1))))")
ln -s /usr/local/lib/libjvm.dylib /Library/Frameworks/R.framework/Versions/$MY_R_VERSION/Resources/lib/