Just use a little bit of currying:
def convertDateFunc(resolution: DateResolutionType) = udf((x:String) =>
SparkDateTimeConverter.convertDate(x, resolution))
and use it as follows:
case FieldDataType.Date => convertDateFunc(resolution(i))(allCols(i))
On a side note you should take a look at sql.functions.trunc
and sql.functions.date_format
. These should at least part of the job without using UDFs at all.
Note:
In Spark 2.2 or later you can use typedLit
function:
import org.apache.spark.sql.functions.typedLit
which support a wider range of literals like Seq
or Map
.