SELECT color, fruit, sum(rating) FROM medleys GROUP BY color, fruit Distinct is used to select distinct elements, nothing more, while you want to aggregate and for that you need GROUP BY and aggregation functions (SUM).
.distinct() is definitely doing a shuffle across partitions. To see more of what’s happening, run a .toDebugString on your RDD. val hashPart = new HashPartitioner(<number of partitions>) val myRDDPreStep = <load some RDD> val myRDD = myRDDPreStep.distinct.partitionBy(hashPart).setName(“myRDD”).persist(StorageLevel.MEMORY_AND_DISK_SER) myRDD.checkpoint println(myRDD.toDebugString) which for an RDD example I have (myRDDPreStep is already hash-partitioned by key, persisted by StorageLevel.MEMORY_AND_DISK_SER, … Read more
Update 1 Better you use this against above. SELECT id, col2, col3, col4 FROM yourtable GROUP BY col2, col3; Demo The reason I am saying is because using CONCAT, I am not getting desired result in this case. First query is returning me 5 rows however CONCAT is returning me 4 rows which is INCORRECT. … Read more
You are close. select DISTINCT(c.name) from Customer c
The following should work $ad->getcodes()->distinct()->count(‘pid’);
You can use the drop_duplicates method to get the unique rows in a DataFrame: In [29]: df = pd.DataFrame({‘a’:[1,2,1,2], ‘b’:[3,4,3,5]}) In [30]: df Out[30]: a b 0 1 3 1 2 4 2 1 3 3 2 5 In [32]: df.drop_duplicates() Out[32]: a b 0 1 3 1 2 4 3 2 5 You can … Read more
This should give you all distinct values from the table. I presume you’d want to add where clauses to select only for a particular question. However, this solution requires 5 subqueries and can be slow if your table is huge. SELECT DISTINCT(ans) FROM ( SELECT right AS ans FROM answers UNION SELECT wrong1 AS ans … Read more
You can just use underscore.js, which comes with Meteor – should be fine for the suggested use case. You might run into performance problems if you try this on a vast data set or run it repeatedly, but it should be adequate for common-or-garden usage: var distinctEntries = _.uniq(Collection.find({}, { sort: {myField: 1}, fields: {myField: … Read more
GROUP BY maps groups of rows to one row, per distinct value in specific columns, which don’t even necessarily have to be in the select-list. SELECT b, c, d FROM table1 GROUP BY a; This query is legal SQL (correction: only in MySQL; actually it’s not standard SQL and not supported by other brands). MySQL … Read more
A pure XPath 1.0 — one-liner: Use: count(/*/group/user[not(. = ../following-sibling::group/user)])