Htykuut

I am trying to write generic add method for which can take any argument type and return result of that type

 def addExactUDF[T](x: T, y: T)(implicit num: Numeric[T]): T = {

    import num._

    x + y

  }



  def addExact(value1: Column, value2: Column, dataType: String): Column =

    dataType match {

      case "Int" => expr(s"addExactUDF(cast($value1 AS INT), cast($value2 AS INT))")

      case "Double" => expr(s"addExactUDF(cast($value1 AS DOUBLE), cast($value2 AS DOUBLE))")

    }

Now when I try to register the UDF

object FilterFunctionsUtil extends MathFunctionsNameSpace with StringFunctionsNameSpace {

  lazy val registerAsSparkUdf: UserDefinedFunction = {

    val sqlContext = SparkSessionFactory.getSparkSession(Map(), Nil).sqlContext



    sqlContext.udf.register("addExactUDF", addExactUDF _)

    sqlContext.udf.register("subtractExactUDF", subtractExactUDF _)

  }

}

It throw me error

   Error:(36, 44) could not find implicit value for parameter num: Numeric[Nothing]

        sqlContext.udf.register("addExactUDF", addExactUDF _)

Error:(36, 44) not enough arguments for method addExactUDF: (implicit num: Numeric[Nothing])Nothing.

Unspecified value parameter num.

    sqlContext.udf.register("addExactUDF", addExactUDF _)

how do I add the implicit to make this work

Specify the implicit type while registering the UDF. you may need to register separate UDF for each type when required.

  import sqlContext.implicits._

  val df = sc.parallelize(Seq((1,2),(3,4))).toDF("val1","val2")



  df.createOrReplaceTempView("tempTable")

  sqlContext.cacheTable("tempTable")



  sqlContext.udf.register("addExtractIntUDF",addExtractUDF[Int] _)



  sqlContext.sql("select addExtractIntUDF(val1,val2) from temptable").show(false)

Result

+---------------------------------------------+

|UDF:addExtractIntUDF(val1,val2) |

+---------------------------------------------+


|3                                                     |

|7                                                     |

+--------------------------------------------+