Tuning parameters for implicit pyspark.ml ALS matrix factorization model through pyspark.ml CrossValidator

by

Ignoring technical issues, strictly speaking neither method is correct given the input generated by ALS with implicit feedback.

  • you cannot use RegressionEvaluator because, as you already know, prediction can be interpreted as a confidence value and is represented as a floating point number in range [0, 1] and label column is just an unbound integer. These values are clearly not comparable.
  • you cannot use BinaryClassificationEvaluator because even if the prediction can be interpreted as probability label doesn’t represent binary decision. Moreover prediction column has invalid type and couldn’t be used directly with BinaryClassificationEvaluator

You can try to convert one of the columns so input fit the requirements but this is is not really a justified approach from a theoretical perspective and introduces additional parameters which are hard to tune.

  • map label column to [0, 1] range and use RMSE.

  • convert label column to binary indicator with fixed threshold and extend ALS / ALSModel to return expected column type. Assuming threshold value is 1 it could be something like this

    from pyspark.ml.recommendation import *
from pyspark.sql.functions import udf, col
from pyspark.mllib.linalg import DenseVector, VectorUDT

class BinaryALS(ALS):
    def fit(self, df):
        assert self.getImplicitPrefs()
        model = super(BinaryALS, self).fit(df)
        return ALSBinaryModel(model._java_obj)

class ALSBinaryModel(ALSModel):
    def transform(self, df):
        transformed = super(ALSBinaryModel, self).transform(df)
        as_vector = udf(lambda x: DenseVector([1 - x, x]), VectorUDT())
        return transformed.withColumn(
            "rawPrediction", as_vector(col("prediction")))

# Add binary label column
with_binary = dfCounts.withColumn(
    "label_binary", (col("rating") > 0).cast("double"))

als_binary_model = BinaryALS(implicitPrefs=True).fit(with_binary)

evaluatorB = BinaryClassificationEvaluator(
    metricName="areaUnderROC", labelCol="label_binary")

evaluatorB.evaluate(als_binary_model.transform(with_binary))
## 1.0

Generally speaking, material about evaluating recommender systems with implicit feedbacks is kind of missing in textbooks, I suggest you take a read on eliasah‘s answer about evaluating these kind of recommenders.

More Related Contents:

  1. How to split Vector into columns – using PySpark
  2. How to map features from the output of a VectorAssembler back to the column names in Spark ML?
  3. How to create a custom Estimator in PySpark
  4. How do I convert an array (i.e. list) column to Vector
  5. How to find median and quantiles using Spark
  6. Spark Dataframe distinguish columns with duplicated name
  7. creating spark data structure from multiline record
  8. How to change dataframe column names in pyspark?
  9. I can’t seem to get –py-files on Spark to work
  10. How can we JOIN two Spark SQL dataframes using a SQL-esque “LIKE” criterion?
  11. Spark RDD to DataFrame python
  12. Create a custom Transformer in PySpark ML
  13. Pyspark ‘NoneType’ object has no attribute ‘_jvm’ error
  14. Why is Apache-Spark – Python so slow locally as compared to pandas?
  15. How to explode multiple columns of a dataframe in pyspark
  16. Spark groupByKey alternative
  17. How to extract an element from a array in pyspark
  18. PySpark: multiple conditions in when clause
  19. PySpark create new column with mapping from a dict
  20. Create Spark DataFrame. Can not infer schema for type
  21. How to transform data with sliding window over time series data in Pyspark
  22. Create single row dataframe from list of list PySpark
  23. Build a hierarchy from a relational data-set using Pyspark
  24. When to cache a DataFrame?
  25. environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON
  26. PySpark Throwing error Method __getnewargs__([]) does not exist
  27. Pyspark 2.4.0, read avro from kafka with read stream – Python
  28. PySpark Evaluation
  29. Apache Spark — Assign the result of UDF to multiple dataframe columns
  30. What is the difference between spark-submit and pyspark?

Leave a Comment