[大数据]Scala 利用java 线程池实现FIFO调度器

Scala 利用java 线程池实现FIFO调度器

1 设计想法

1. 三个提交事件 继承任务事件 分别是 提交事件(TaskSubmited) 任务成功事件(TaskSucceeded) 任务失败事件(TaskFailed)
2. 事件管理抽象类 EventLoop 里面有阻塞队列 eventQueue 原子操作的停止标志 stop 事件处理线程eventThread 里面run方法调用onReceive 方法实现任务的完成 在onReceive方法中利用scala中的偏函数 PartialFunction 实现对不同任务的匹配工作
3. 具体定义的FiFO调度器 TaskProcessEventLoop 里面具体实现了事件管理抽象函数的抽象方法,
4. 启动对象 Bootstrap 启动10个事件 分别为不同的三种事件 然后观察提交后的调度情况
5. 实验结果发现 按照FiFO调度实现

2 代码

3.1 TaskEvent

trait  TaskEvent

case class TaskSubmitted(name: String) extends TaskEvent

case class TaskSucceeded(name: String) extends TaskEvent

case class TaskFailed(name: String) extends TaskEvent

3.2 EventLoop

import java.util.concurrent.atomic.AtomicBoolean
import java.util.concurrent.{BlockingQueue, LinkedBlockingDeque}

import scala.util.control.NonFatal

/**
 * Created by 57871 on 2018/10/31. */ abstract class EventLoop[E](name: String) {

  //java中的消息队列
  private val eventQueue: BlockingQueue[E] = new LinkedBlockingDeque[E]

  private val stopped = new AtomicBoolean(false)

  private val eventThread = new Thread(name){
    override def run(): Unit = {
      try {
        while (!stopped.get()){
          //从阻塞队列中取出第一个任务,如果没有就阻塞
  val event = eventQueue.take
          try{
            //利用偏函数,将某一类的任务交给一种类型去处理
  onReceive(event)
          }catch {
            case NonFatal(e) => {
              try {
                onError(e)
              }catch {
                case NonFatal(e) => println(s"Unexepeted error in $name",e)
              }
            }
          }
        }
      }catch {
        case ie : InterruptedException => // exit even if eventQueue is not empty
  case NonFatal(e) => println(s"Unexepeted error in $name",e)
      }
    }

  }
  def start(): Unit = {
    if(stopped.get()){
      throw new IllegalStateException(s"$name has already been stopped")
    }
    //// Call onStart before starting the event thread to make sure it happens
  onStart()
    eventThread.start()
  }

  /**
 * Put the event into the event queue. The event thread will process it later. */  def post(event: E): Unit = {
    eventQueue.put(event)
  }

  /**
 * Return if the event thread has already been started but not yet stopped. */  def isActive: Boolean = eventThread.isAlive

  /**
 * Invoked when `start()` is called but before the event thread starts.
 */  protected def onStart(): Unit = {}

  /**
 * Invoked when `stop()` is called and the event thread exits.
 */  protected def onStop(): Unit = {}

  /**
 * Invoked in the event thread when polling events from the event queue. * * Note: Should avoid calling blocking actions in `onReceive`, or the event thread will be blocked
 * and cannot process events in time. If you want to call some blocking actions, run them in * another thread. */  //protected def onReceive(event: E): Unit    protected def onReceive: PartialFunction[E, Unit]

  /**
 * Invoked if `onReceive` throws any non fatal error. Any non fatal error thrown from `onError`
  * will be ignored.
 */  protected def onError(e: Throwable): Unit   }

3.3 TaskProcessEventLoop

package jv.thread

/**
 * Created by 57871 on 2018/10/31. */ 
 class TaskProcessEventLoop (name: String) extends EventLoop[TaskEvent](name) {

  /**
 * Invoked in the event thread when polling events from the event queue. * * Note: Should avoid calling blocking actions in `onReceive`, or the event thread will be blocked
 * and cannot process events in time. If you want to call some blocking actions, run them in * another thread. */  
 override protected def onReceive: PartialFunction[TaskEvent, Unit] = {  //偏函数, 不同的任务作出不同的处理
  case TaskSubmitted(name) => {
      println(name + " submitted")
    }

    case TaskSucceeded(name) => {
      println(name + " succeeded")
    }

    case TaskFailed(name) => {
      println(name + " filled")
    }
  }
  override protected def onError(e: Throwable): Unit = {

  }
  /**
 * Invoked when `start()` is called but before the event thread starts.
 */  override protected def onStart(): Unit = {
    println("on start invoke")
  }
}

3.4 Bootstrap

/**
 * Created by 57871 on 2018/10/31. */ class TaskProcessEventLoop (name: String) extends EventLoop[TaskEvent](name) {

  /**
 * Invoked in the event thread when polling events from the event queue. * * Note: Should avoid calling blocking actions in `onReceive`, or the event thread will be blocked
 * and cannot process events in time. If you want to call some blocking actions, run them in * another thread. */  override protected def onReceive: PartialFunction[TaskEvent, Unit] = {  //偏函数, 不同的任务作出不同的处理
  case TaskSubmitted(name) => {
      println(name + " submitted")
    }

    case TaskSucceeded(name) => {
      println(name + " succeeded")
    }

    case TaskFailed(name) => {
      println(name + " filled")
    }
  }
  override protected def onError(e: Throwable): Unit = {

  }
  /**
 * Invoked when `start()` is called but before the event thread starts.
 */  override protected def onStart(): Unit = {
    println("on start invoke")
  }
}

3效果

实验按照FiFO调度规则完成调度

on start invoke
task-succedd-1 succeeded
task-2 submitted
task-fail-3 filled
task-4 submitted
task-succedd-5 succeeded
task-6 submitted
task-succedd-7 succeeded
task-8 submitted
task-fail-9 filled
task-10 submitted