Profile的选择:
1. hadoop-2.4和YARN勾选,复选框中显示黑色的对号
2. hbase-hadoop1,maven-3,scala-2.10默认选择,复选框中显示灰的对号
问题:
1. PermGen内存
[ERROR] PermGen space -> [Help 1]
java.lang.OutOfMemoryError: PermGen space
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at scala.tools.nsc.Global.formatExplain(Global.scala:1140)
at scala.tools.nsc.Global.supplementErrorMessage(Global.scala:1157)
at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1565)
at scala.tools.nsc.Global$Run.compileSources(Global.scala:1553)
at scala.tools.nsc.Global$Run.compile(Global.scala:1662)
Maven显示内存的使用情况Final Memory: 39M/827M,表示PermGen space为39M?
解决办法:https://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
在/etc/profile中添加如下Maven的内存设置(本机物理内存2G)
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m"
这种做法适用于使用mvn在shell终端进行构建,如果在Intellij Idea中构建报这个错,则需要打开Idea的Maven配置窗口,在
Maven
Runner的VM OPTS中输入如下参数:
-Xmx1024m -XX:MaxPermSize=512m
2. Maven编译两个小时多之后,报编译错误,CodeGenerator.scala报很多错误.不过系统仍然可以编译
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:387: error: value q is not a member of StringContext
q"""
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:403: error: value q is not a member of StringContext
q"""
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:411: error: value q is not a member of StringContext
q"""
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:418: error: value q is not a member of StringContext
q"""
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:440: error: value q is not a member of StringContext
q"""
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:456: error: value q is not a member of StringContext
q"""
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:468: error: value q is not a member of StringContext
q"""
^
/home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:486: error: value q is not a member of StringContext
q"""
^
3.EventBatch.scala找不到
编译两个多小时,这个文件依然不存在,
j
4。经过4个小时的构建,终于编完了。。。竟然编译成功了,但是EventBatch依然不存在(所以,FlumeBatchFetcher.scala报错EventBatch和SparkFlumeProtocol不存在)和CodeGenerator.scala中依然无数的第二步的错
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 18.895 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 55.038 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 41.395 s]
[INFO] Spark Project Core ................................. SUCCESS [11:58 min]
[INFO] Spark Project Bagel ................................ SUCCESS [02:01 min]
[INFO] Spark Project GraphX ............................... SUCCESS [09:00 min]
[INFO] Spark Project Streaming ............................ SUCCESS [18:49 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [24:13 min]
[INFO] Spark Project SQL .................................. SUCCESS [28:07 min]
[INFO] Spark Project ML Library ........................... SUCCESS [39:05 min]
[INFO] Spark Project Tools ................................ SUCCESS [02:20 min]
[INFO] Spark Project Hive ................................. SUCCESS [24:52 min]
[INFO] Spark Project REPL ................................. SUCCESS [08:47 min]
[INFO] Spark Project YARN ................................. SUCCESS [07:45 min]
[INFO] Spark Project Assembly ............................. SUCCESS [10:06 min]
[INFO] Spark Project External Twitter ..................... SUCCESS [01:38 min]
[INFO] Spark Project External Flume Sink .................. SUCCESS [01:48 min]
[INFO] Spark Project External Flume ....................... SUCCESS [03:02 min]
[INFO] Spark Project External MQTT ........................ SUCCESS [01:35 min]
[INFO] Spark Project External ZeroMQ ...................... SUCCESS [01:27 min]
[INFO] Spark Project External Kafka ....................... SUCCESS [02:34 min]
[INFO] Spark Project Examples ............................. SUCCESS [24:45 min]
[INFO] Spark Project YARN Shuffle Service ................. SUCCESS [01:22 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:47 h
[INFO] Finished at: 2015-01-10T03:29:36+08:00
[INFO] Final Memory: 102M/882M
[INFO] ------------------------------------------------------------------------
5.既然编译成功了,那就运行LocalPi.scala,触发Intellj Idea Make整个项目,又是一顿等待(这种情况产生的原因是什么?为什么不能只编译一个文件),make的结果是
Compilation completed with 44 errors and 37 warnings in 15 min 23 sec
查看出错信息,基本都是Spark SQL的catalyst相关的报错:
Error:(317, 109) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function1[_, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(317, 109) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function1[_, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(319, 112) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function2[_, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(319, 112) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function2[_, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(321, 115) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function3[_, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(321, 115) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function3[_, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(323, 118) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function4[_, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(323, 118) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function4[_, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(325, 121) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function5[_, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(325, 121) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function5[_, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(327, 124) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function6[_, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(327, 124) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function6[_, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(329, 127) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function7[_, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(329, 127) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function7[_, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(331, 130) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function8[_, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(331, 130) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function8[_, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(333, 133) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function9[_, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(333, 133) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function9[_, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(335, 137) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function10[_, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(335, 137) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function10[_, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(337, 140) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function11[_, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(337, 140) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function11[_, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(339, 143) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function12[_, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(339, 143) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function12[_, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(341, 146) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function13[_, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(341, 146) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function13[_, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(343, 149) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(343, 149) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(345, 152) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(345, 152) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(347, 155) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(347, 155) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(349, 158) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(349, 158) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(351, 161) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(351, 161) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(353, 164) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(353, 164) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(355, 167) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(355, 167) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(357, 170) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(357, 170) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(359, 173) polymorphic expression cannot be instantiated to expected type;
found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)]
implicit def functionToUdfBuilder[T: TypeTag](func: Function22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
Error:(359, 173) type mismatch;
found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T]
required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T]
implicit def functionToUdfBuilder[T: TypeTag](func: Function22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func)
^
EventBatch不能编译出来的解决
Log4j冲突问题的解决:
远程调试代码:
http://www.iteblog.com/archives/1192
参考:
http://www.cnblogs.com/luogankun/p/3905148.html