The advantage would come when it looks something like this: trait Container[A <: Container[A]] extends Ordered[A] { def clone: A def pair: (A, A) = (clone, clone) } class MyContainer extends Container[MyContainer] { def clone = new MyContainer } Now you get pair for free, and you get the correct return type. Without something like … Read more
Akka Streams is using asynchronous message passing between Actors to implement stream processing stages. Passing data across an asynchronous boundary has an overhead that you are seeing here: your computation seems to take only about 160ns (derived from the single-threaded measurement) while the streaming solution takes roughly 1µs per element, which is dominated by the … Read more
Because the definition of “equals” for Arrays is that they refer to the same array. This is consistent with Java’s array equality, using Object.Equals, so it compares references. If you want to check pairwise elements, then use sameElements Array(‘a’,’b’).sameElements(Array(‘a’,’b’)) or deepEquals, which has been deprecated in 2.8, so instead use: Array(‘a’,’b’).deep.equals(Array(‘a’,’b’).deep) There’s a good Nabble … Read more
How about this: scala> def transpose[A](xs: List[List[A]]): List[List[A]] = xs.filter(_.nonEmpty) match { | case Nil => Nil | case ys: List[List[A]] => ys.map{ _.head }::transpose(ys.map{ _.tail }) | } warning: there were unchecked warnings; re-run with -unchecked for details transpose: [A](xs: List[List[A]])List[List[A]] scala> val ls = List(List(1, 2, 3), List(4, 5), List(6, 7, 8)) ls: … Read more
If your goal is to run external scala classes in runtime, I’d suggest using eval with scala.tools.reflect.ToolBox (it is included in REPL, but for normal usage you have to add scala-reflect.jar): import scala.reflect.runtime.universe import scala.tools.reflect.ToolBox val tb = universe.runtimeMirror(getClass.getClassLoader).mkToolBox() tb.eval(tb.parse(“””println(“hello!”)”””)) You also can compile files, using tb.compile. Modified with example: assume you have external file … Read more
Vals are initialized in the order they are declared (well, precisely, non-lazy vals are), so properties is getting initialized before loadedProps. Or in other words, loadedProps is still null when propertiesis getting initialized. The simplest solution here is to define loadedProps before properties: class Config { private val loadedProps = { val p = new … Read more
When doing pattern matching, the Scala compiler will interpret o1 c1 o2 the same as c1(o1, o2). That’s why :: works inside pattern matches too.
You could simply use regexp_replace df5.withColumn(“sentence_without_label”, regexp_replace($”sentence” , lit($”label”), lit(“” ))) or you can use simple udf function as below val df5 = spark.createDataFrame(Seq( (“Hi I heard about Spark”, “Spark”), (“I wish Java could use case classes”, “Java”), (“Logistic regression models are neat”, “models”) )).toDF(“sentence”, “label”) val replace = udf((data: String , rep : String)=>data.replaceAll(rep, … Read more
First for some shameless self-promotion: Eugene Burmako and I are giving a talk on type providers, a closely related topic, at Scalar 2014 tomorrow, and I encourage you to take a look at the example project we put together for the talk if you’re interested in this kind of thing. While type macros are no … Read more
It is an expected behavior. User defined functions have to be deterministic: The user-defined functions must be deterministic. Due to optimization, duplicate invocations may be eliminated or the function may even be invoked more times than it is present in the query. If you want to include non-deterministic function and preserve the output you should … Read more