It is not possible to create multiple top level columns from a single UDF call but you can create a new struct. It requires an UDF with specified returnType: from pyspark.sql.functions import udf from pyspark.sql.types import StructType, StructField, FloatType schema = StructType([ StructField(“foo”, FloatType(), False), StructField(“bar”, FloatType(), False) ]) def udf_test(n): return (n / 2, … Read more
You might be making this more complicated than you need to. As a programmer, I’m sure you appreciate that when learning a new system, a little time with the help, Google, “Dummies” books, etc. is time well spent. Excel emphasizes a very different programming style than, say, C#. So, here are some options, starting with … Read more
You must have any user-defined m-files on the MATLAB search path, just as if you were working normally inside MATLAB. I tested with the following example: C:\some\path\myfunc.m function myfunc() disp(‘hello from MYFUNC’) end HelloWorld.java import matlabcontrol.*; public class HelloWorld { public static void main(String[] args) throws MatlabConnectionException, MatlabInvocationException { // create proxy MatlabProxyFactoryOptions options = … Read more
You could simply use regexp_replace df5.withColumn(“sentence_without_label”, regexp_replace($”sentence” , lit($”label”), lit(“” ))) or you can use simple udf function as below val df5 = spark.createDataFrame(Seq( (“Hi I heard about Spark”, “Spark”), (“I wish Java could use case classes”, “Java”), (“Logistic regression models are neat”, “models”) )).toDF(“sentence”, “label”) val replace = udf((data: String , rep : String)=>data.replaceAll(rep, … Read more
Declare your arguments as Optional Variants, then you can test to see if they’re missing using IsMissing() or check their type using TypeName(), as shown in the following example: Public Function Foo(Optional v As Variant) As Variant If IsMissing(v) Then Foo = “Missing argument” ElseIf TypeName(v) = “String” Then Foo = v & ” plus … Read more
In this case, it’s an “inline table valued function” The optimiser simply expands (unnests) it if it’s useful (or view). If the function is treated as “black box” by the outer query, the quickest way is to compare IO shown in SSMS vs IO in profiler. Profler captures “black box” IO that SSMS does not. … Read more
No, in PHP that is not possible as of writing. Use array arguments: function doSomething($arguments = array()) { // set defaults $arguments = array_merge(array( “argument” => “default value”, ), $arguments); var_dump($arguments); } Example usage: doSomething(); // with all defaults, or: doSomething(array(“argument” => “other value”)); When changing an existing method: //function doSomething($bar, $baz) { function doSomething($bar, … Read more
I discovered the exact problem but I want to up-vote you all for trying to help me figure this out, and give GSerg the credit because, while I wasn’t completely out of luck, he was dead on with his suggestion that Excel does like to make certain properties of a range unavailable during certain stages … Read more
Everything that is passed to an UDF is interpreted as a column / column name. If you want to pass a literal you have two options: Pass argument using currying: def comparatorUDF(n): return udf(lambda c: c == n, BooleanType()) df.where(comparatorUDF(“Bonsanto”)(col(“name”))) This can be used with an argument of any type as long as it is … Read more
It’s a table-valued function, but you’re using it as a scalar function. Try: where Emp_Id IN (SELECT i.items FROM dbo.Splitfn(@Id,’,’) AS i) But… also consider changing your function into an inline TVF, as it’ll perform better.