Profile的选择:
1. hadoop-2.4和YARN勾选,复选框中显示黑色的对号
2. hbase-hadoop1,maven-3,scala-2.10默认选择,复选框中显示灰的对号
问题:
1. PermGen内存
[ERROR] PermGen space -> [Help 1] java.lang.OutOfMemoryError: PermGen space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at scala.tools.nsc.Global.formatExplain(Global.scala:1140) at scala.tools.nsc.Global.supplementErrorMessage(Global.scala:1157) at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1565) at scala.tools.nsc.Global$Run.compileSources(Global.scala:1553) at scala.tools.nsc.Global$Run.compile(Global.scala:1662)
Maven显示内存的使用情况Final Memory: 39M/827M,表示PermGen space为39M?
解决办法:https://cwiki.apache.org/confluence/display/MAVEN/OutOfMemoryError
在/etc/profile中添加如下Maven的内存设置(本机物理内存2G)
export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m"
这种做法适用于使用mvn在shell终端进行构建,如果在Intellij Idea中构建报这个错,则需要打开Idea的Maven配置窗口,在
Maven
Runner的VM OPTS中输入如下参数:
-Xmx1024m -XX:MaxPermSize=512m
2. Maven编译两个小时多之后,报编译错误,CodeGenerator.scala报很多错误.不过系统仍然可以编译
^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:387: error: value q is not a member of StringContext q""" ^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:403: error: value q is not a member of StringContext q""" ^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:411: error: value q is not a member of StringContext q""" ^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:418: error: value q is not a member of StringContext q""" ^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:440: error: value q is not a member of StringContext q""" ^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:456: error: value q is not a member of StringContext q""" ^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:468: error: value q is not a member of StringContext q""" ^ /home/hadoop/sparksourcecode/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala:486: error: value q is not a member of StringContext q""" ^
3.EventBatch.scala找不到
编译两个多小时,这个文件依然不存在,
j
4。经过4个小时的构建,终于编完了。。。竟然编译成功了,但是EventBatch依然不存在(所以,FlumeBatchFetcher.scala报错EventBatch和SparkFlumeProtocol不存在)和CodeGenerator.scala中依然无数的第二步的错
[INFO] Reactor Summary: [INFO] [INFO] Spark Project Parent POM ........................... SUCCESS [ 18.895 s] [INFO] Spark Project Networking ........................... SUCCESS [ 55.038 s] [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 41.395 s] [INFO] Spark Project Core ................................. SUCCESS [11:58 min] [INFO] Spark Project Bagel ................................ SUCCESS [02:01 min] [INFO] Spark Project GraphX ............................... SUCCESS [09:00 min] [INFO] Spark Project Streaming ............................ SUCCESS [18:49 min] [INFO] Spark Project Catalyst ............................. SUCCESS [24:13 min] [INFO] Spark Project SQL .................................. SUCCESS [28:07 min] [INFO] Spark Project ML Library ........................... SUCCESS [39:05 min] [INFO] Spark Project Tools ................................ SUCCESS [02:20 min] [INFO] Spark Project Hive ................................. SUCCESS [24:52 min] [INFO] Spark Project REPL ................................. SUCCESS [08:47 min] [INFO] Spark Project YARN ................................. SUCCESS [07:45 min] [INFO] Spark Project Assembly ............................. SUCCESS [10:06 min] [INFO] Spark Project External Twitter ..................... SUCCESS [01:38 min] [INFO] Spark Project External Flume Sink .................. SUCCESS [01:48 min] [INFO] Spark Project External Flume ....................... SUCCESS [03:02 min] [INFO] Spark Project External MQTT ........................ SUCCESS [01:35 min] [INFO] Spark Project External ZeroMQ ...................... SUCCESS [01:27 min] [INFO] Spark Project External Kafka ....................... SUCCESS [02:34 min] [INFO] Spark Project Examples ............................. SUCCESS [24:45 min] [INFO] Spark Project YARN Shuffle Service ................. SUCCESS [01:22 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:47 h [INFO] Finished at: 2015-01-10T03:29:36+08:00 [INFO] Final Memory: 102M/882M [INFO] ------------------------------------------------------------------------
5.既然编译成功了,那就运行LocalPi.scala,触发Intellj Idea Make整个项目,又是一顿等待(这种情况产生的原因是什么?为什么不能只编译一个文件),make的结果是
Compilation completed with 44 errors and 37 warnings in 15 min 23 sec
查看出错信息,基本都是Spark SQL的catalyst相关的报错:
Error:(317, 109) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function1[_, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(317, 109) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function1[_, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(319, 112) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function2[_, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(319, 112) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function2[_, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(321, 115) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function3[_, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(321, 115) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function3[_, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(323, 118) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function4[_, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(323, 118) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function4[_, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(325, 121) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function5[_, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(325, 121) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function5[_, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(327, 124) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function6[_, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(327, 124) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function6[_, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(329, 127) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function7[_, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(329, 127) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function7[_, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(331, 130) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function8[_, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(331, 130) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function8[_, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(333, 133) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function9[_, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(333, 133) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function9[_, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(335, 137) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function10[_, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(335, 137) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function10[_, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(337, 140) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function11[_, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(337, 140) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function11[_, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(339, 143) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function12[_, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(339, 143) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function12[_, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(341, 146) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function13[_, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(341, 146) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function13[_, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(343, 149) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(343, 149) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function14[_, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(345, 152) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(345, 152) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function15[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(347, 155) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(347, 155) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function16[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(349, 158) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(349, 158) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function17[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(351, 161) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(351, 161) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function18[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(353, 164) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(353, 164) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function19[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(355, 167) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(355, 167) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function20[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(357, 170) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(357, 170) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function21[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(359, 173) polymorphic expression cannot be instantiated to expected type; found : [T(in method apply)]org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T(in method apply)] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T(in method functionToUdfBuilder)] implicit def functionToUdfBuilder[T: TypeTag](func: Function22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^ Error:(359, 173) type mismatch; found : org.apache.spark.sql.catalyst.dsl.ScalaUdfBuilder[T] required: org.apache.spark.sql.catalyst.dsl.package.ScalaUdfBuilder[T] implicit def functionToUdfBuilder[T: TypeTag](func: Function22[_, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, _, T]): ScalaUdfBuilder[T] = ScalaUdfBuilder(func) ^
EventBatch不能编译出来的解决
Log4j冲突问题的解决:
远程调试代码:
http://www.iteblog.com/archives/1192
参考:
http://www.cnblogs.com/luogankun/p/3905148.html