| add_months(start_date, num_months) |
Returns the date that is `num_months` after `start_date`. |
| convert_timezone([sourceTz, ]targetTz, sourceTs) |
Converts the timestamp without time zone `sourceTs` from the `sourceTz` time zone to `targetTz`. |
| curdate() |
Returns the current date at the start of query evaluation. All calls of curdate within the same query return the same value. |
| current_date() |
Returns the current date at the start of query evaluation. All calls of current_date within the same query return the same value. |
| current_date |
Returns the current date at the start of query evaluation. |
| current_timestamp() |
Returns the current timestamp at the start of query evaluation. All calls of current_timestamp within the same query return the same value. |
| current_timestamp |
Returns the current timestamp at the start of query evaluation. |
| current_timezone() |
Returns the current session local timezone. |
| date_add(start_date, num_days) |
Returns the date that is `num_days` after `start_date`. |
| date_diff(endDate, startDate) |
Returns the number of days from `startDate` to `endDate`. |
| date_format(timestamp, fmt) |
Converts `timestamp` to a value of string in the format specified by the date format `fmt`. |
| date_from_unix_date(days) |
Create date from the number of days since 1970-01-01. |
| date_part(field, source) |
Extracts a part of the date/timestamp or interval source. |
| date_sub(start_date, num_days) |
Returns the date that is `num_days` before `start_date`. |
| date_trunc(fmt, ts) |
Returns timestamp `ts` truncated to the unit specified by the format model `fmt`. |
| dateadd(start_date, num_days) |
Returns the date that is `num_days` after `start_date`. |
| datediff(endDate, startDate) |
Returns the number of days from `startDate` to `endDate`. |
| datepart(field, source) |
Extracts a part of the date/timestamp or interval source. |
| day(date) |
Returns the day of month of the date/timestamp. |
| dayname(date) |
Returns the three-letter abbreviated day name from the given date. |
| dayofmonth(date) |
Returns the day of month of the date/timestamp. |
| dayofweek(date) |
Returns the day of the week for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday). |
| dayofyear(date) |
Returns the day of year of the date/timestamp. |
| extract(field FROM source) |
Extracts a part of the date/timestamp or interval source. |
| from_unixtime(unix_time[, fmt]) |
Returns `unix_time` in the specified `fmt`. |
| from_utc_timestamp(timestamp, timezone) |
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone. For example, 'GMT+1' would yield '2017-07-14 03:40:00.0'. |
| hour(timestamp) |
Returns the hour component of the string/timestamp. |
| last_day(date) |
Returns the last day of the month which the date belongs to. |
| localtimestamp() |
Returns the current timestamp without time zone at the start of query evaluation. All calls of localtimestamp within the same query return the same value. |
| localtimestamp |
Returns the current local date-time at the session time zone at the start of query evaluation. |
| make_date(year, month, day) |
Create date from year, month and day fields. If the configuration `spark.sql.ansi.enabled` is false, the function returns NULL on invalid inputs. Otherwise, it will throw an error instead. |
| make_dt_interval([days[, hours[, mins[, secs]]]]) |
Make DayTimeIntervalType duration from days, hours, mins and secs. |
| make_interval([years[, months[, weeks[, days[, hours[, mins[, secs]]]]]]]) |
Make interval from years, months, weeks, days, hours, mins and secs. |
| make_timestamp(year, month, day, hour, min, sec[, timezone]) |
Create timestamp from year, month, day, hour, min, sec and timezone fields. The result data type is consistent with the value of configuration `spark.sql.timestampType`. If the configuration `spark.sql.ansi.enabled` is false, the function returns NULL on invalid inputs. Otherwise, it will throw an error instead. |
| make_timestamp_ltz(year, month, day, hour, min, sec[, timezone]) |
Create the current timestamp with local time zone from year, month, day, hour, min, sec and timezone fields. If the configuration `spark.sql.ansi.enabled` is false, the function returns NULL on invalid inputs. Otherwise, it will throw an error instead. |
| make_timestamp_ntz(year, month, day, hour, min, sec) |
Create local date-time from year, month, day, hour, min, sec fields. If the configuration `spark.sql.ansi.enabled` is false, the function returns NULL on invalid inputs. Otherwise, it will throw an error instead. |
| make_ym_interval([years[, months]]) |
Make year-month interval from years, months. |
| minute(timestamp) |
Returns the minute component of the string/timestamp. |
| month(date) |
Returns the month component of the date/timestamp. |
| monthname(date) |
Returns the three-letter abbreviated month name from the given date. |
| months_between(timestamp1, timestamp2[, roundOff]) |
If `timestamp1` is later than `timestamp2`, then the result
is positive. If `timestamp1` and `timestamp2` are on the same day of month, or both
are the last day of month, time of day will be ignored. Otherwise, the difference is
calculated based on 31 days per month, and rounded to 8 digits unless roundOff=false. |
| next_day(start_date, day_of_week) |
Returns the first date which is later than `start_date` and named as indicated.
The function returns NULL if at least one of the input parameters is NULL.
When both of the input parameters are not NULL and day_of_week is an invalid input,
the function throws SparkIllegalArgumentException if `spark.sql.ansi.enabled` is set to true, otherwise NULL. |
| now() |
Returns the current timestamp at the start of query evaluation. |
| quarter(date) |
Returns the quarter of the year for date, in the range 1 to 4. |
| second(timestamp) |
Returns the second component of the string/timestamp. |
| session_window(time_column, gap_duration) |
Generates session window given a timestamp specifying column and gap duration.
See 'Types of time windows' in Structured Streaming guide doc for detailed explanation and examples. |
| timestamp_micros(microseconds) |
Creates timestamp from the number of microseconds since UTC epoch. |
| timestamp_millis(milliseconds) |
Creates timestamp from the number of milliseconds since UTC epoch. |
| timestamp_seconds(seconds) |
Creates timestamp from the number of seconds (can be fractional) since UTC epoch. |
| to_date(date_str[, fmt]) |
Parses the `date_str` expression with the `fmt` expression to
a date. Returns null with invalid input. By default, it follows casting rules to a date if
the `fmt` is omitted. |
| to_timestamp(timestamp_str[, fmt]) |
Parses the `timestamp_str` expression with the `fmt` expression
to a timestamp. Returns null with invalid input. By default, it follows casting rules to
a timestamp if the `fmt` is omitted. The result data type is consistent with the value of
configuration `spark.sql.timestampType`. |
| to_timestamp_ltz(timestamp_str[, fmt]) |
Parses the `timestamp_str` expression with the `fmt` expression
to a timestamp with local time zone. Returns null with invalid input. By default, it follows casting rules to
a timestamp if the `fmt` is omitted. |
| to_timestamp_ntz(timestamp_str[, fmt]) |
Parses the `timestamp_str` expression with the `fmt` expression
to a timestamp without time zone. Returns null with invalid input. By default, it follows casting rules to
a timestamp if the `fmt` is omitted. |
| to_unix_timestamp(timeExp[, fmt]) |
Returns the UNIX timestamp of the given time. |
| to_utc_timestamp(timestamp, timezone) |
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC. For example, 'GMT+1' would yield '2017-07-14 01:40:00.0'. |
| trunc(date, fmt) |
Returns `date` with the time portion of the day truncated to the unit specified by the format model `fmt`. |
| try_to_timestamp(timestamp_str[, fmt]) |
Parses the `timestamp_str` expression with the `fmt` expression
to a timestamp. The function always returns null on an invalid input with/without ANSI SQL
mode enabled. By default, it follows casting rules to a timestamp if the `fmt` is omitted.
The result data type is consistent with the value of configuration `spark.sql.timestampType`. |
| unix_date(date) |
Returns the number of days since 1970-01-01. |
| unix_micros(timestamp) |
Returns the number of microseconds since 1970-01-01 00:00:00 UTC. |
| unix_millis(timestamp) |
Returns the number of milliseconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision. |
| unix_seconds(timestamp) |
Returns the number of seconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision. |
| unix_timestamp([timeExp[, fmt]]) |
Returns the UNIX timestamp of current or specified time. |
| weekday(date) |
Returns the day of the week for date/timestamp (0 = Monday, 1 = Tuesday, ..., 6 = Sunday). |
| weekofyear(date) |
Returns the week of the year of the given date. A week is considered to start on a Monday and week 1 is the first week with >3 days. |
| window(time_column, window_duration[, slide_duration[, start_time]]) |
Bucketize rows into one or more time windows given a timestamp specifying column.
Window starts are inclusive but the window ends are exclusive, e.g. 12:05 will be in the window [12:05,12:10) but not in [12:00,12:05).
Windows can support microsecond precision. Windows in the order of months are not supported.
See 'Window Operations on Event Time' in Structured Streaming guide doc for detailed explanation and examples. |
| window_time(window_column) |
Extract the time value from time/session window column which can be used for event time value of window.
The extracted time is (window.end - 1) which reflects the fact that the aggregating
windows have exclusive upper bound - [start, end)
See 'Window Operations on Event Time' in Structured Streaming guide doc for detailed explanation and examples. |
| year(date) |
Returns the year component of the date/timestamp. |