MythBusters Impala edition "Impala udf functions runs even after jar is removed" BUSTED
28 Oct 2014While I was working on a Java Impala UDF, a colleague pointed me to: Impala udf functions runs even after jar is removed
Thought that should give it a try - it might happen that all my work will be useless because of that. I prepared two jar files with the same class names (but different implementations!). UDF signatures were the same. The scenario was as follows:
- Create a function from first jar in schema A.
- Execute query with that function.
- Drop function.
- Try to execute query with function (failed - as expected).
- Replace the first UDF jar file in HDFS with second implementation.
- Create a function from the same path in HDFS in schema A (with the same UDF signature!).
- Execute query with that function - query worked and second version of code was invoked.
No signs of problems described in the original post. Version of Imapala used: 1.4.1. I’ve also tried a different scenario:
- two jars with the same name (but different paths in HDFS)
- same class names
- two different implementattions
- each implementation registered in different schema
Unfortunately - in this case it also worked fine ;)