pyspark.sql.functions.approx_count_distinct

pyspark.sql.functions.approx_count_distinct(col: ColumnOrName, rsd: Optional[float] = None) → pyspark.sql.column.Column

Aggregate function: returns a new Column for approximate distinct count of column col.

New in version 2.1.0.

Parameters

colColumn or str

rsdfloat, optional

maximum relative standard deviation allowed (default = 0.05). For rsd < 0.01, it is more efficient to use count_distinct()

Examples

>>> df.agg(approx_count_distinct(df.age).alias('distinct_ages')).collect()
[Row(distinct_ages=2)]

pyspark.sql.functions.approxCountDistinct pyspark.sql.functions.avg