How to use SQL query to define table in dbtable?

In JDBC To Other Databases I found the following explanation of dbtable parameter:

The JDBC table that should be read. Note that anything that is valid in a FROM clause of a SQL query can be used. For example, instead of a full table you could also use a subquery in parentheses.

When I use the code:

CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
  url "jdbc:postgresql:dbserver",
  dbtable "mytable"
)

everything works great, but the following:

 dbtable "SELECT * FROM mytable"

leads to the error:

enter image description here

What is wrong?

Since dbtable is used as a source for the SELECT statement it has be in a form which would be valid for normal SQL query. If you want to use subquery you should pass a query in parentheses and provide an alias:

CREATE TEMPORARY TABLE jdbcTable
USING org.apache.spark.sql.jdbc
OPTIONS (
    url "jdbc:postgresql:dbserver",
    dbtable "(SELECT * FROM mytable) tmp"
);

It will be passed to the database as:

SELECT * FROM (SELECT * FROM mytable) tmp WHERE 1=0

How to use SQL query to define table in dbtable?

Related

Recent Posts