Property NameDefaultMeaningSince Version
spark.sql.event.truncate.length 2147483647

Threshold of SQL length beyond which it will be truncated before adding to event. Defaults to no truncation. If set to 0, callsite will be logged instead.

 3.0.0
spark.sql.extensions (none)

A comma-separated list of classes that implement Function1[SparkSessionExtensions, Unit] used to configure Spark Session extensions. The classes must have a no-args constructor. If multiple extensions are specified, they are applied in the specified order. For the case of rules and planner strategies, they are applied in the specified order. For the case of parsers, the last parser is used and each parser can delegate to its predecessor. For the case of function name conflicts, the last registered function name is used.

 2.2.0
spark.sql.hive.metastore.barrierPrefixes

A comma separated list of class prefixes that should explicitly be reloaded for each version of Hive that Spark SQL is communicating with. For example, Hive UDFs that are declared in a prefix that typically would be shared (i.e. org.apache.spark.*).

 1.4.0
spark.sql.hive.metastore.jars builtin

Location of the jars that should be used to instantiate the HiveMetastoreClient. This property can be one of three options: " 1. "builtin" Use Hive 2.3.7, which is bundled with the Spark assembly when -Phive is enabled. When this option is chosen, spark.sql.hive.metastore.version must be either 2.3.7 or not defined. 2. "maven" Use Hive jars of specified version downloaded from Maven repositories. 3. A classpath in the standard format for both Hive and Hadoop.

 1.4.0
spark.sql.hive.metastore.sharedPrefixes com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc

A comma separated list of class prefixes that should be loaded using the classloader that is shared between Spark SQL and a specific version of Hive. An example of classes that should be shared is JDBC drivers that are needed to talk to the metastore. Other classes that need to be shared are those that interact with classes that are already shared. For example, custom appenders that are used by log4j.

 1.4.0
spark.sql.hive.metastore.version 2.3.7

Version of the Hive metastore. Available options are 0.12.0 through 2.3.7 and 3.0.0 through 3.1.2.

 1.4.0
spark.sql.hive.thriftServer.singleSession false

When set to true, Hive Thrift server is running in a single session mode. All the JDBC/ODBC connections share the temporary views, function registries, SQL configuration and the current database.

 1.6.0
spark.sql.legacy.sessionInitWithConfigDefaults false

Flag to revert to legacy behavior where a cloned SparkSession receives SparkConf defaults, dropping any overrides in its parent SparkSession.

 3.0.0
spark.sql.queryExecutionListeners (none)

List of class names implementing QueryExecutionListener that will be automatically added to newly created sessions. The classes should have either a no-arg constructor, or a constructor that expects a SparkConf argument.

 2.3.0
spark.sql.streaming.streamingQueryListeners (none)

List of class names implementing StreamingQueryListener that will be automatically added to newly created sessions. The classes should have either a no-arg constructor, or a constructor that expects a SparkConf argument.

 2.4.0
spark.sql.streaming.ui.enabled true

Whether to run the Structured Streaming Web UI for the Spark application when the Spark Web UI is enabled.

 3.0.0
spark.sql.streaming.ui.retainedProgressUpdates 100

The number of progress updates to retain for a streaming query for Structured Streaming UI.

 3.0.0
spark.sql.streaming.ui.retainedQueries 100

The number of inactive queries to retain for Structured Streaming UI.

 3.0.0
spark.sql.ui.retainedExecutions 1000

Number of executions to retain in the Spark UI.

 1.5.0
spark.sql.warehouse.dir (value of $PWD/spark-warehouse)

The default location for managed databases and tables.

 2.0.0