| add_months(start_date, num_months) |
Returns the date that is `num_months` after `start_date`. |
| current_date() |
Returns the current date at the start of query evaluation. All calls of current_date within the same query return the same value. |
| current_date |
Returns the current date at the start of query evaluation. |
| current_timestamp() |
Returns the current timestamp at the start of query evaluation. All calls of current_timestamp within the same query return the same value. |
| current_timestamp |
Returns the current timestamp at the start of query evaluation. |
| current_timezone() |
Returns the current session local timezone. |
| date_add(start_date, num_days) |
Returns the date that is `num_days` after `start_date`. |
| date_format(timestamp, fmt) |
Converts `timestamp` to a value of string in the format specified by the date format `fmt`. |
| date_from_unix_date(days) |
Create date from the number of days since 1970-01-01. |
| date_part(field, source) |
Extracts a part of the date/timestamp or interval source. |
| date_sub(start_date, num_days) |
Returns the date that is `num_days` before `start_date`. |
| date_trunc(fmt, ts) |
Returns timestamp `ts` truncated to the unit specified by the format model `fmt`. |
| datediff(endDate, startDate) |
Returns the number of days from `startDate` to `endDate`. |
| day(date) |
Returns the day of month of the date/timestamp. |
| dayofmonth(date) |
Returns the day of month of the date/timestamp. |
| dayofweek(date) |
Returns the day of the week for date/timestamp (1 = Sunday, 2 = Monday, ..., 7 = Saturday). |
| dayofyear(date) |
Returns the day of year of the date/timestamp. |
| extract(field FROM source) |
Extracts a part of the date/timestamp or interval source. |
| from_unixtime(unix_time[, fmt]) |
Returns `unix_time` in the specified `fmt`. |
| from_utc_timestamp(timestamp, timezone) |
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in UTC, and renders that time as a timestamp in the given time zone. For example, 'GMT+1' would yield '2017-07-14 03:40:00.0'. |
| hour(timestamp) |
Returns the hour component of the string/timestamp. |
| last_day(date) |
Returns the last day of the month which the date belongs to. |
| make_date(year, month, day) |
Create date from year, month and day fields. If the configuration `spark.sql.ansi.enabled` is false, the function returns NULL on invalid inputs. Otherwise, it will throw an error instead. |
| make_dt_interval([days[, hours[, mins[, secs]]]]) |
Make DayTimeIntervalType duration from days, hours, mins and secs. |
| make_interval([years[, months[, weeks[, days[, hours[, mins[, secs]]]]]]]) |
Make interval from years, months, weeks, days, hours, mins and secs. |
| make_timestamp(year, month, day, hour, min, sec[, timezone]) |
Create timestamp from year, month, day, hour, min, sec and timezone fields. The result data type is consistent with the value of configuration `spark.sql.timestampType`. If the configuration `spark.sql.ansi.enabled` is false, the function returns NULL on invalid inputs. Otherwise, it will throw an error instead. |
| make_ym_interval([years[, months]]) |
Make year-month interval from years, months. |
| minute(timestamp) |
Returns the minute component of the string/timestamp. |
| month(date) |
Returns the month component of the date/timestamp. |
| months_between(timestamp1, timestamp2[, roundOff]) |
If `timestamp1` is later than `timestamp2`, then the result
is positive. If `timestamp1` and `timestamp2` are on the same day of month, or both
are the last day of month, time of day will be ignored. Otherwise, the difference is
calculated based on 31 days per month, and rounded to 8 digits unless roundOff=false. |
| next_day(start_date, day_of_week) |
Returns the first date which is later than `start_date` and named as indicated.
The function returns NULL if at least one of the input parameters is NULL.
When both of the input parameters are not NULL and day_of_week is an invalid input,
the function throws IllegalArgumentException if `spark.sql.ansi.enabled` is set to true, otherwise NULL. |
| now() |
Returns the current timestamp at the start of query evaluation. |
| quarter(date) |
Returns the quarter of the year for date, in the range 1 to 4. |
| second(timestamp) |
Returns the second component of the string/timestamp. |
| session_window(time_column, gap_duration) |
Generates session window given a timestamp specifying column and gap duration.
See 'Types of time windows' in Structured Streaming guide doc for detailed explanation and examples. |
| timestamp_micros(microseconds) |
Creates timestamp from the number of microseconds since UTC epoch. |
| timestamp_millis(milliseconds) |
Creates timestamp from the number of milliseconds since UTC epoch. |
| timestamp_seconds(seconds) |
Creates timestamp from the number of seconds (can be fractional) since UTC epoch. |
| to_date(date_str[, fmt]) |
Parses the `date_str` expression with the `fmt` expression to
a date. Returns null with invalid input. By default, it follows casting rules to a date if
the `fmt` is omitted. |
| to_timestamp(timestamp_str[, fmt]) |
Parses the `timestamp_str` expression with the `fmt` expression
to a timestamp. Returns null with invalid input. By default, it follows casting rules to
a timestamp if the `fmt` is omitted. The result data type is consistent with the value of
configuration `spark.sql.timestampType`. |
| to_unix_timestamp(timeExp[, fmt]) |
Returns the UNIX timestamp of the given time. |
| to_utc_timestamp(timestamp, timezone) |
Given a timestamp like '2017-07-14 02:40:00.0', interprets it as a time in the given time zone, and renders that time as a timestamp in UTC. For example, 'GMT+1' would yield '2017-07-14 01:40:00.0'. |
| trunc(date, fmt) |
Returns `date` with the time portion of the day truncated to the unit specified by the format model `fmt`. |
| unix_date(date) |
Returns the number of days since 1970-01-01. |
| unix_micros(timestamp) |
Returns the number of microseconds since 1970-01-01 00:00:00 UTC. |
| unix_millis(timestamp) |
Returns the number of milliseconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision. |
| unix_seconds(timestamp) |
Returns the number of seconds since 1970-01-01 00:00:00 UTC. Truncates higher levels of precision. |
| unix_timestamp([timeExp[, fmt]]) |
Returns the UNIX timestamp of current or specified time. |
| weekday(date) |
Returns the day of the week for date/timestamp (0 = Monday, 1 = Tuesday, ..., 6 = Sunday). |
| weekofyear(date) |
Returns the week of the year of the given date. A week is considered to start on a Monday and week 1 is the first week with >3 days. |
| window(time_column, window_duration[, slide_duration[, start_time]]) |
Bucketize rows into one or more time windows given a timestamp specifying column.
Window starts are inclusive but the window ends are exclusive, e.g. 12:05 will be in the window [12:05,12:10) but not in [12:00,12:05).
Windows can support microsecond precision. Windows in the order of months are not supported.
See 'Window Operations on Event Time' in Structured Streaming guide doc for detailed explanation and examples. |
| year(date) |
Returns the year component of the date/timestamp. |