distinctUntilChanged when applied to the valueChanges observable… …is probably never going to work! It only works as expected on an individual value (as you said). So you’ll get a new value even if nothing has changed. To track changes for the whole form you need to write a custom comparer, the simplest of which uses … Read more
Update 1 Better you use this against above. SELECT id, col2, col3, col4 FROM yourtable GROUP BY col2, col3; Demo The reason I am saying is because using CONCAT, I am not getting desired result in this case. First query is returning me 5 rows however CONCAT is returning me 4 rows which is INCORRECT. … Read more
EDIT: as noleto mentions in his answer below, there is now approx_count_distinct available since PySpark 2.1 that works over a window. Original answer – exact distinct count (not an approximation) We can use a combination of size and collect_set to mimic the functionality of countDistinct over a window: from pyspark.sql import functions as F, Window … Read more
This sounds like a non-repeating random number generator. There are several possible approaches to this. As described in this article, we can generate them by selecting a prime number p and satisfies p % 4 = 3 that is large enough (greater than the maximum value in the output range) and generate them this way: … Read more
Here is a possible comparer that compares an IEnumerable<T> by its elements. You still need to sort manually before adding. One could build the sorting into the comparer, but I don’t think that’s a wise choice. Adding a canonical form of the list seems wiser. This code will only work in .net 4 since it … Read more
Do you mean unique: R> x = c(1,1,2,3,4,4,4) R> x [1] 1 1 2 3 4 4 4 R> unique(x) [1] 1 2 3 4
In pySpark you could do something like this, using countDistinct(): from pyspark.sql.functions import col, countDistinct df.agg(*(countDistinct(col(c)).alias(c) for c in df.columns)) Similarly in Scala : import org.apache.spark.sql.functions.countDistinct import org.apache.spark.sql.functions.col df.select(df.columns.map(c => countDistinct(col(c)).alias(c)): _*) If you want to speed things up at the potential loss of accuracy, you could also use approxCountDistinct().
A data.table approach library(data.table) DT <- data.table(myvec) DT[, .(number_of_distinct_orders = length(unique(order_no))), by = name] data.table v >= 1.9.5 has a built in uniqueN function now DT[, .(number_of_distinct_orders = uniqueN(order_no)), by = name]
Java 8 Distinct by property