###1、需求
1、IntelliJ IDEA打开多个项目
2、每个同学开发一个项目，相互之前独立不影响
3、通过一个入口可以调用所有项目类、方法、属性，达到同时开发且检测代码
4、dependency只需要写一份，其余项目不用写，便可全部依赖
###2、注意事项（非常重要）
6个坑：
1、
项目中所有的groupId要一样
2、避免循环依赖，导致程序报错
3、
打包的服务器运行时候需要provided，本机调试的时候，需要注释
在一个maven项目中，如果存在编译需要而发布不需要的jar包，可以用scope标签，值设为provided
4、项目、module建好之后需要添加Scala的框架支持
5、在yhProject中，可以统一对所有的module进行清理、编译、打包
6、要运行依赖中的module，则必须要将module中的Jar包，打到maven中，需要使用install
下面，是我将所有module中的Jar打到Maven中的路径：
###3、建立Project和建立module
1、只需要建立一个项目，其他项目由module建立，所有module且放在项目中。
2、本文项目为yhproject，其余都为module，分别是：mainEntrance、yhutils、yhapp、yhweb、yhgame
项目建立步鄹：
Module建立步鄹：
项目、所有module、部分module代码展示：
###4、项目之前的依赖关系
###5、代码展示
mainEntrance
package com.yh.bi.dag
package com.yh.bi.dag
/**
* Created by yuhui on 2017/2/10.
*/
import java.time.{Duration, LocalDate}
import com.yh.bi._
import com.yh.bi.{UserAPP, UserGame, UserWEB}
import org.slf4j.LoggerFactory
import scala.collection.immutable.{ListMap, ListSet}
/**
* Created by yuhui on 2016/8/25.
* task --> Node --> DAG --> DAGExecutor
*/
case class Node[T](task: T, parent: T*) {
override def toString: String = {
s"$task(${parent.mkString(",")})"
}
}
case class DAG[T](nodes: Node[T]*)
case class DAGExecutor[T](dag: DAG[T]) {
private val LOG = LoggerFactory.getLogger(this.getClass)
private val _nodes: Map[T, Seq[T]] = dag.nodes.map(node => (node.task, node.parent.filter(_ != null))).toMap
private var _pending: Set[T] = ListSet()
private var _fails = ListMap[T, String]()
private var _success = Seq[T]()
//判断Node的task节点的父节点运行状态（flase ，true）
private def getPending: Option[T] = {
_pending.find { name =>
val parents = _nodes(name)
!parents.exists(name => !_success.contains(name))
}
}
private def fail(name: T, message: String): Unit = {
_pending -= name
_fails += name -> message
for (child <- _pending.filter(child => _nodes(child).contains(name))) {
fail(child, s"依赖的任务无法执行: $name")
}
}
private def success(name: T): Unit = {
_pending -= name
_success = _success :+ name
}
def execute(func: T => Unit): Unit = {
_pending = _nodes.keySet
_fails = ListMap()
_success = Seq()
var running = true
while (running) {
val taskOpt = getPending
if (taskOpt.nonEmpty) {
val task = taskOpt.get
val startMills = System.currentTimeMillis()
LOG.info("start task {}", task)
try {
println("=============")
func(task) //执行executor方法
println("+++++++++++++")
val time = Duration.ofMillis(System.currentTimeMillis() - startMills)
LOG.info(s"end task $task time=$time")
success(task)
} catch {
case e: Throwable => fail(task, e.getMessage)
LOG.error(e.getMessage, e)
LOG.info(s"fail task $task")
}
} else {
running = false
}
}
for (name <- _success) {
LOG.info(s"success task: $name")
}
for (name <- _fails) {
LOG.info(s"fail task: ${name._1} - ${name._2}")
}
}
}
object DAG {
val allSDKDAG = new DAG[Task](
Node(UserAPP),
Node(UserGame),
Node(UserWEB)
)
def main(args: Array[String]): Unit = {
DAGExecutor(allSDKDAG).execute { task =>task.executor("appkey": String, LocalDate.now(), LocalDate.now())}
}
}
yhutils
package com.yh.bi
/**
* Created by yuhui on 2017/2/10.
*/
import java.time.LocalDate
import org.apache.spark.sql.SQLContext
import org.slf4j.LoggerFactory
abstract class Executor extends Task with SQLContextAware {
override def run(appkey: String, startDay: LocalDate, endDay: LocalDate)={}
}
trait SQLContextAware {
implicit var ctx: SQLContext = _
}
abstract class Task {
protected val LOG = LoggerFactory.getLogger(this.getClass)
def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit
def run(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {
executor(appkey, startDay, endDay)
}
}
yhapp
package com.yh.bi
/**
* Created by yuhui on 2017/2/10.
*/
import java.time.LocalDate
object UserAPP extends Executor{
override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {
println("++++我的UserAPP的执行过程++++")
}
}
yhweb
package com.yh.bi
import java.time.LocalDate
object UserWEB extends Executor{
override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {
println("++++我的UserWEB的执行过程++++")
}
}
yhgame
package com.yh.bi
/**
* Created by yuhui on 2017/2/10.
*/
import java.time.LocalDate
object UserGame extends Executor{
override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {
println("++++我的UserGame的执行过程++++")
}
}
###6、项目中POM依赖展示
yhproject中POM文件展示：
4.0.0 com.yh.bi yhproject pom 1.0 mainEntrance yhapp yhweb yhgame yhutils
mainEntrance中POM文件展示：
4.0.0 mainEntrance com.yh.bi 1.0 yhutils com.yh.bi 1.0 yhapp com.yh.bi 1.0 yhgame com.yh.bi 1.0 yhweb com.yh.bi 1.0 org.apache.maven.plugins maven-compiler-plugin 1.8 1.8 org.apache.maven.plugins maven-shade-plugin 2.4 *:* **/log4j2.* META-INF/*.SF META-INF/*.DSA META-INF/*.RSA package shade ${project.build.directory}/${project.artifactId}.jar src/main/scala ${basedir}/src/main/resources **/*
yhutils中POM文件展示：
yhproject com.yh.bi 1.0 4.0.0 yhutils org.apache.spark spark-hive_2.11 1.6.1 org.apache.maven.plugins maven-compiler-plugin 1.8 1.8 org.apache.maven.plugins maven-shade-plugin 2.4 *:* **/log4j2.* META-INF/*.SF META-INF/*.DSA META-INF/*.RSA package shade ${project.build.directory}/${project.artifactId}.jar src/main/scala ${basedir}/src/main/resources **/*
yhapp中POM文件展示：
4.0.0 com.yh.bi yhapp 1.0 yhutils com.yh.bi 1.0 provided org.apache.maven.plugins maven-compiler-plugin 1.8 1.8 org.apache.maven.plugins maven-shade-plugin 2.4 *:* **/log4j2.* META-INF/*.SF META-INF/*.DSA META-INF/*.RSA package shade ${project.build.directory}/${project.artifactId}.jar src/main/scala ${basedir}/src/main/resources **/*
yhweb中POM文件展示：
4.0.0 com.yh.bi yhweb 1.0 yhutils com.yh.bi 1.0 provided org.apache.maven.plugins maven-compiler-plugin 1.8 1.8 org.apache.maven.plugins maven-shade-plugin 2.4 *:* **/log4j2.* META-INF/*.SF META-INF/*.DSA META-INF/*.RSA package shade ${project.build.directory}/${project.artifactId}.jar src/main/scala ${basedir}/src/main/resources **/*
yhgame中POM文件展示：
4.0.0 com.yh.bi yhgame 1.0 yhutils com.yh.bi 1.0 provided org.apache.maven.plugins maven-compiler-plugin 1.8 1.8 org.apache.maven.plugins maven-shade-plugin 2.4 *:* **/log4j2.* META-INF/*.SF META-INF/*.DSA META-INF/*.RSA package shade ${project.build.directory}/${project.artifactId}.jar src/main/scala ${basedir}/src/main/resources **/*
###7、运行结果展示
注意：我在mainEntrance执行DAG中的main文件，可以调用且执行了yhutils、yhapp、yhweb、yhgame中的代码
###8、如果建立Java 的module
以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持脚本之家。