公平调度器一次分配多个Task至TaskTracker算法

TaskTracker.heartbeat()机制请求处理任务,Fair Scheduler确认Job,然后给TaskTracker执行
此时可动态设置一个TaskTracker多个slot任务,具体条件算法如下(以Map为例)

1.mapsAssigned == mapCapacity
   用户设置的单次心跳最多执行task数量
2.runningMaps == runnableMaps
   正在执行map到达可用map数量,代码分析如下
     runnableMaps-
  -->PoolSchedulable.getDemand()    |PoolSchedulable
     -->for (JobSchedulable sched: jobScheds)  |JobSchedulable
          -->demand += JobSchedulable.getDemand();
              -->for (TaskInProgress tip: tips)  |JobSchedulable
                 -->if (tip.isRunning())
                    -->demand += tip.getActiveTasks().size() |TaskInProgress
                        -->activeTasks.put(taskid, taskTracker);

runningMaps
  -->PoolSchedulable.getRunningTasks()
     -->for (JobSchedulable sched: jobScheds)
         -->JobSchedulable.getRunningTasks()
             -->job.runningMaps()  |JobInProgress
                -->JobInProgress.getRunningTasks()

 
3.loadMgr .canAssignMap(tts, runnableMaps, totalMapSlots, mapsAssigned)
   LoadManager.canAssign 负载均衡管理器用于控制是否能分配,算法分析如下
     CapBasedLoadManager.canAssign()
       |-->int cap = getCap(totalRunnableMaps, tracker.getMaxMapSlots(),totalMapSlots);
               |-->double load = maxDiff + (( double )totalRunnableTasks) / totalSlots |平衡因子+集群负载因子
               |-->Math. ceil(localMaxTasks * Math. min(1.0, load)) |计算当前TT可负载状况
       |-->tracker.countMapTasks() + alreadyAssigned < cap
             Tasktracker当前使用map数量与已分配的数量小于,则alreadyAssigned 可继续增加

你可能感兴趣的:(公平调度器一次分配多个Task至TaskTracker算法)