How to check the Spark version

as titled, how do I know which version of spark has been installed in the CentOS?

The current system has installed cdh5.1.0.

Solution 1:

If you use Spark-Shell, it appears in the banner at the start.

Programatically, SparkContext.version can be used.

Solution 2:

Open Spark shell Terminal, run sc.version

Solution 3:

You can use spark-submit command: spark-submit --version

Solution 4:

In Spark 2.x program/shell,

use the

spark.version   

Where spark variable is of SparkSession object

Using the console logs at the start of spark-shell

[root@bdhost001 ~]$ spark-shell
Setting the default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel).
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.2.0
      /_/

Without entering into code/shell

spark-shell --version

[root@bdhost001 ~]$ spark-shell --version
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.2.0
      /_/
                        
Type --help for more information.

spark-submit --version

[root@bdhost001 ~]$ spark-submit --version
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.2.0
      /_/
                        
Type --help for more information.

Solution 5:

If you are using Databricks and talking to a notebook, just run :

spark.version