Another version of a hash with a key for each element in your array and value for the count of each element a = [ 1, 2, 3, 3, 4, 3] h = Hash.new(0) a.each { | v | h.store(v, h[v]+1) } # h = { 3=>3, 2=>1, 1=>1, 4=>1 }
If you’re just doing small batches that are relatively well-formed, then the compare.linkage() or compare.dedup() functions in the RecordLinkage package should be a great starting point. But if you have big batches, then you might have to do some more tinkering. I use the functions jarowinkler(), levenshteinSim(), and soundex() in RecordLinkage to write my own … Read more
Aggregate the column by COUNT, then use a HAVING clause to find values that appear greater than one time. SELECT column_name, COUNT(column_name) FROM table_name GROUP BY column_name HAVING COUNT(column_name) > 1;
Do dtEmp on your current working DataTable: DataTable distinctTable = dtEmp.DefaultView.ToTable( /*distinct*/ true); It’s nice.
Use array_unique(). Example: $array = array(1, 2, 2, 3); $array = array_unique($array); // Array is now (1, 2, 3)
Here is another way. No intermediate variables are saved. We used this to de-duplicate results from a variety of overlapping queries. $input = array_map(“unserialize”, array_unique(array_map(“serialize”, $input)));