presto_udf.md

Presto UDF

The core logic for the Presto functions is at FunctionRegistry.java which defineds window, aggregate and scalar types of functions.

We might want to add functions without changing Presto code. then we can implement & use experimental functions easily. But Presto doesn't have a room for UDF, UDAF and UDTF yet.

So here're an idea of how Presto could support UDF and UDAF in a othogonal way. We might not consider UDTF at this time to make thinghs simple but it would not be much different. Also supporting existing Hive UDF is out of scope of this topic and should be an issue of the HiveConnector.

• Presto already has well-defined plugin arhitecture. Sets of UDF and UDAF functions could be thought as a plugin having a FunctionFactory.class services
  • So UD(A)F functions can be easily a part of exsting connectors or they can build a seperate functions only plugin.
• FunctionFactory should be able to return List<FunctionInfo> by using FunctionRegistry.FunctionListBuilder. The constructor of FunctionRegistry has very good example of how to build the list.

+++ b/presto-spi/src/main/java/com/facebook/presto/spi/FunctionFactory.java
@@ -0,0 +1,9 @@
+package com.facebook.presto.spi;
+
+import java.util.List;
+import com.facebook.presto.metadata.FunctionInfo;
+
+public interface FunctionFactory
+{
+    List<FunctionInfo> listFunctions();
+}

diff --git a/presto-server/src/main/java/com/facebook/presto/server/PluginManager.java b/presto-serv
index f2bc881..cfefef8 100644
--- a/presto-server/src/main/java/com/facebook/presto/server/PluginManager.java
+++ b/presto-server/src/main/java/com/facebook/presto/server/PluginManager.java
@@ -153,6 +153,10 @@ public class PluginManager
             for (SystemTable systemTable : plugin.getServices(SystemTable.class)) {
                 systemTablesManager.addTable(systemTable);
             }
+
+            for (FunctionFactory functionFactory : plugin.getServices(FunctionFactory.class)) {
+                metadataManager.addCustomFunctions(functionFactory);
+            }
         }
     }

• MetadataManager pass the returned list of FunctionInfo to FunctionRegistry as it can merge/append the list

diff --git a/presto-main/src/main/java/com/facebook/presto/metadata/MetadataManager.java b/presto-ma
index 6287c99..ed91cf0 100644
--- a/presto-main/src/main/java/com/facebook/presto/metadata/MetadataManager.java
+++ b/presto-main/src/main/java/com/facebook/presto/metadata/MetadataManager.java
@@ -75,6 +75,10 @@ public class MetadataManager
         internalSchemas.add(new ConnectorMetadataEntry(connectorId, connectorMetadata));
     }

+    public void addCusttomFunctions(FunctionFactory functionFactory) {
+        functions.addCustomFunctions(functionFactory.listFunctions());
+    }
+
     @Override
     public FunctionInfo getFunction(QualifiedName name, List<Type> parameterTypes)
     {
     
diff --git a/presto-main/src/main/java/com/facebook/presto/metadata/FunctionRegistry.java b/presto-m
index 3ea97bd..07770a9 100644
--- a/presto-main/src/main/java/com/facebook/presto/metadata/FunctionRegistry.java
+++ b/presto-main/src/main/java/com/facebook/presto/metadata/FunctionRegistry.java
@@ -230,6 +230,10 @@ public class FunctionRegistry
         return functionsByHandle.get(handle);
     }

+    public void addCustomFunctions(List<FunctionInfo> functions) {
+        // TODO: Implement
+    }
+
     private static List<Type> types(MethodHandle handle)
     {
         ImmutableList.Builder<Type> types = ImmutableList.builder();

How to test the plugged functions

At first, we can create usual unit tests based on function tests at TestExpressionCompiler.java, even if it would not still be easy to create unit tests for window and aggregate functions.

In the future, we could invent a descriptive testing framework or development kit like the Hive PDT (Plugin Developer Kit): https://cwiki.apache.org/confluence/display/Hive/PluginDeveloperKit