How to access a Databricks database with sparklyr

New in Azure Databricks environment, I discover the packages SparkR and sparklyr.

From my notebooks with SparkR, I manage to connect to a database :

library(SparkR)
DW <- sql("select * from mydb.sometable")

It runs well, but SparkR syntax seems too far removed from classic R syntax (according to me).

So I wanted to try sparklyr, but I can't access this same database :

library(sparklyr)
sc <- spark_connect(method="databricks")
test <- spark_read_table(sc, "mydb.sometable")

Error : org.apache.spark.sql.AnalysisException: It is not allowed to add database prefix ...

Is something wrong ?

Thanks !!


Solution 1:

If it helps anyone, here's what I found that seems to work.

  1. Setting the default data base
  2. Read table in default data base
library(sparklyr)
library(dplyr)
sc <- spark_connect(method="databricks")
tbl_change_db(sc, "mydb")
foo <- spark_read_table(sc,"sometable")