深度强化学习系列之(11): “超参数”与“网络结构”自动化设置方法---DeepHyper
可扩展的异步神经网络和超参数搜索深度神经网络方法
前言：
在深度学习和机器学习算法学习和训练的过程中，有两个非常让人头疼的问题
- 超参数的设置
- 神经网络结构的设计
这两个问题一直困扰每一个与神经网络有关的学习者，为了解决这些问题，谷歌公司开源了AutoML(貌似收费)。此外还有Keras（后期详解）等，本篇文章介绍一个自动化学习包： DeepHyper
DeepHyper是一种用于深度神经网络的自动化机器学习（AutoML）软件包。 它包括两个组成部分:
(1）神经架构搜索是一种自动搜索高性能深度神经网络架构的方法。
(2）超参数搜索是一种自动搜索给定深度神经网络的高性能超参数的方法。
DeepHyper提供了一个基础架构，旨在针对HPC(Hyper Parameters Search)系统中的神经架构和超参数搜索方法，可扩展性和可移植性进行实验研究。 为可扩展的超参数和神经架构搜索方法的实现和研究提供了一个通用接口。 在这个包中，其为用户提供了不同的模块：
-
基准（benchmark）：超参数或神经架构搜索的一组问题，用户可以使用它来比较我们的不同搜索算法或作为构建自己问题的示例。
-
评估者（evaluator）：一组有助于在不同系统和不同情况下运行搜索的对象，例如快速和轻型实验或长时间和重度运行。
-
搜索（search）：一组用于超参数和神经架构搜索的算法。 您还将找到一种模块化方法来定义新的搜索算法和用于超参数或神经架构搜索的特定子模块。
其结构如下：
一、Hyperparameter Search (HPS)搜索
（1）定义超参数问题
首先导入deephyper包，并设置问题和纬度
from deephyper.benchmark import HpProblem
Problem = HpProblem()
Problem.add_dim('nunits', (10, 20), 10)
print(Problem)
Problem
{'nunits': (10, 20)}
Starting Point
{'nunits': 10}
通过运行模型的函数，结果为类似{‘nunits’：10}的字典，但每个键的值将根据搜索的选择而改变。 下面看看如何为mnist数据上的多层Perceptron模型训练定义一个简单的运行函数。
'''Trains a simple deep NN on the MNIST dataset.
Gets to 98.40% test accuracy after 20 epochs
(there is *a lot* of margin for parameter tuning).
2 seconds per epoch on a K520 GPU.
'''
from __future__ import print_function
import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.optimizers import RMSprop
def run(params):
nunits = params['nunits]
batch_size = 128
num_classes = 10
epochs = 20
# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)
model = Sequential()
model.add(Dense(nunits, activation='relu', input_shape=(784,)))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(num_classes, activation='softmax'))
model.summary()
model.compile(loss='categorical_crossentropy',
optimizer=RMSprop(),
metrics=['accuracy'])
history = model.fit(x_train, y_train,
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test, y_test))
score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
return -score[1]
现在，如果您想要搜索上一个问题和模型。 假设问题是在“package_name/problem.py” 中定义，模型在package_name/mnist_mlp.py 中定义。 如果使用命令行运行AMBS之类的搜索：
python ambs.py --problem package_name.problem.Problem --run package_name.mnist_mlp.run
（2）Asynchronous Model-Base Search (AMBS)
论文：点击阅读
class deephyper.search.hps.ambs.AMBS(problem, run, evaluator, **kwargs)
(3) Genetic Algorithm (GA)
接口类
class deephyper.search.hps.ga.GA(problem, run, evaluator, **kwargs)
完整代码
import signal
import random
from deephyper.search.hps.optimizer import GAOptimizer
from deephyper.search import Search
from deephyper.search import util
logger = util.conf_logger('deephyper.search.hps.ga')
SERVICE_PERIOD = 2 # Delay (seconds) between main loop iterations
CHECKPOINT_INTERVAL = 10 # How many jobs to complete between optimizer checkpoints
EXIT_FLAG = False
def on_exit(signum, stack):
global EXIT_FLAG
EXIT_FLAG = True
[docs]class GA(Search):
def __init__(self, problem, run, evaluator, **kwargs):
super().__init__(problem, run, evaluator, **kwargs)
logger.info("Initializing GA")
self.optimizer = GAOptimizer(self.problem, self.num_workers, self.args)
@staticmethod
def _extend_parser(parser):
parser.add_argument('--ga_num_gen',
default=100,
type=int,
help='number of generation for genetic algorithm'
)
parser.add_argument('--individuals_per_worker',
default=5,
type=int,
help='number of individuals per worker')
return parser
def run(self):
# opt = GAOptimizer(cfg)
# evaluator = evaluate.create_evaluator(cfg)
logger.info(f"Starting new run")
timer = util.DelayTimer(max_minutes=None, period=SERVICE_PERIOD)
timer = iter(timer)
elapsed_str = next(timer)
logger.info("Hyperopt GA driver starting")
logger.info(f"Elapsed time: {elapsed_str}")
if self.optimizer.pop is None:
logger.info("Generating initial population")
logger.info(f"{self.optimizer.INIT_POP_SIZE} individuals")
self.optimizer.pop = self.optimizer.toolbox.population(n=self.optimizer.INIT_POP_SIZE)
individuals = self.optimizer.pop
self.evaluate_fitnesses(individuals, self.optimizer, self.evaluator, self.args.eval_timeout_minutes)
self.optimizer.record_generation(num_evals=len(self.optimizer.pop))
with open('ga_logbook.log', 'w') as fp:
fp.write(str(self.optimizer.logbook))
print("best:", self.optimizer.halloffame[0])
while self.optimizer.current_gen < self.optimizer.NGEN:
self.optimizer.current_gen += 1
logger.info(f"Generation {self.optimizer.current_gen} out of {self.optimizer.NGEN}")
logger.info(f"Elapsed time: {elapsed_str}")
# Select the next generation individuals
offspring = self.optimizer.toolbox.select(self.optimizer.pop, len(self.optimizer.pop))
# Clone the selected individuals
offspring = list(map(self.optimizer.toolbox.clone, offspring))
# Apply crossover and mutation on the offspring
for child1, child2 in zip(offspring[::2], offspring[1::2]):
if random.random() < self.optimizer.CXPB:
self.optimizer.toolbox.mate(child1, child2)
del child1.fitness.values
del child2.fitness.values
for mutant in offspring:
if random.random() < self.optimizer.MUTPB:
self.optimizer.toolbox.mutate(mutant)
del mutant.fitness.values
# Evaluate the individuals with an invalid fitness
invalid_ind = [ind for ind in offspring if not ind.fitness.valid]
logger.info(f"Evaluating {len(invalid_ind)} invalid individuals")
self.evaluate_fitnesses(invalid_ind, self.optimizer, self.evaluator,
self.args.eval_timeout_minutes)
# The population is entirely replaced by the offspring
self.optimizer.pop[:] = offspring
self.optimizer.record_generation(num_evals=len(invalid_ind))
with open('ga_logbook.log', 'w') as fp:
fp.write(str(self.optimizer.logbook))
print("best:", self.optimizer.halloffame[0])
def evaluate_fitnesses(self, individuals, opt, evaluator, timeout_minutes):
points = map(opt.space_encoder.decode_point, individuals)
points = [{key:x for key,x in zip(self.problem.space.keys(), point)}
for point in points]
evaluator.add_eval_batch(points)
logger.info(f"Waiting on {len(points)} individual fitness evaluations")
results = evaluator.await_evals(points, timeout=timeout_minutes*60)
for ind, (x,fit) in zip(individuals, results):
ind.fitness.values = (fit,)
if __name__ == "__main__":
args = GA.parse_args()
search = GA(**vars(args))
#signal.signal(signal.SIGINT, on_exit)
#signal.signal(signal.SIGTERM, on_exit)
search.run()
二、神经网络搜索
Neural Architecture Search (NAS)
###（1）异步搜索 NAS A3C (PPO) Asynchronous
搜索接口类
class deephyper.search.nas.ppo_a3c_async.NasPPOAsyncA3C(problem, run, evaluator, **kwargs)
具体搜索运行代码：
mpirun -np 2 python ppo_a3c_async.py --problem deephyper.benchmark.nas.mnist1D.problem.Problem --run deephyper.search.nas.model.run.alpha.run --evaluator subprocess
完整代码：
import os
import json
from math import ceil, log
from pprint import pprint, pformat
from mpi4py import MPI
import math
from deephyper.evaluator import Evaluator
from deephyper.search import util, Search
from deephyper.search.nas.agent import nas_ppo_async_a3c
logger = util.conf_logger('deephyper.search.nas.ppo_a3c_async')
def print_logs(runner):
logger.debug('num_episodes = {}'.format(runner.global_episode))
logger.debug(' workers = {}'.format(runner.workers))
def key(d):
return json.dumps(dict(arch_seq=d['arch_seq']))
LAUNCHER_NODES = int(os.environ.get('BALSAM_LAUNCHER_NODES', 1))
WORKERS_PER_NODE = int(os.environ.get('DEEPHYPER_WORKERS_PER_NODE', 1))
[docs]class NasPPOAsyncA3C(Search):
"""Neural Architecture search using proximal policy gradient with asynchronous optimization.
"""
def __init__(self, problem, run, evaluator, **kwargs):
super().__init__(problem, run, evaluator, **kwargs)
# set in super : self.problem
# set in super : self.run_func
# set in super : self.evaluator
self.evaluator = Evaluator.create(self.run_func,
cache_key=key,
method=evaluator)
self.num_episodes = kwargs.get('num_episodes')
if self.num_episodes is None:
self.num_episodes = math.inf
self.reward_rule = util.load_attr_from('deephyper.search.nas.agent.utils.'+kwargs['reward_rule'])
self.space = self.problem.space
logger.debug(f'evaluator: {type(self.evaluator)}')
self.num_agents = MPI.COMM_WORLD.Get_size() - 1 # one is the parameter server
self.rank = MPI.COMM_WORLD.Get_rank()
logger.debug(f'num_agents: {self.num_agents}')
logger.debug(f'rank: {self.rank}')
@staticmethod
def _extend_parser(parser):
parser.add_argument('--num-episodes', type=int, default=None,
help='maximum number of episodes')
parser.add_argument('--reward-rule', type=str,
default='final_reward_for_all_timesteps',
choices=[
'final_reward_for_all_timesteps',
'episode_reward_for_final_timestep'
],
help='A function which describe how to spread the episodic reward on all timesteps of the corresponding episode.')
return parser
def main(self):
# Settings
#num_parallel = self.evaluator.num_workers - 4 #balsam launcher & controller of search for cooley
# num_nodes = self.evaluator.num_workers - 1 #balsam launcher & controller of search for theta
num_nodes = LAUNCHER_NODES * WORKERS_PER_NODE - 1 # balsam launcher
num_nodes -= 1 # parameter server is neither an agent nor a worker
if num_nodes > self.num_agents:
num_episodes_per_batch = (num_nodes-self.num_agents)//self.num_agents
else:
num_episodes_per_batch = 1
if self.rank == 0:
logger.debug(f' num_nodes: {num_nodes}' )
logger.debug(f' num_episodes_per_batch: {num_episodes_per_batch}' )
logger.debug(f' starting training...' )
nas_ppo_async_a3c.train(
num_episodes=self.num_episodes,
seed=2018,
space=self.problem.space,
evaluator=self.evaluator,
num_episodes_per_batch=num_episodes_per_batch,
reward_rule=self.reward_rule
)
if __name__ == "__main__":
args = NasPPOAsyncA3C.parse_args()
search = NasPPOAsyncA3C(**vars(args))
search.main()
（2）同步更新 NAS A3C (PPO) Synchronous
class deephyper.search.nas.ppo_a3c_sync.NasPPOSyncA3C(problem, run, evaluator, **kwargs)
使用近端策略梯度进行神经结构搜索和同步优化
python -m deephyper.search.nas.ppo_a3c_sync --evaluator subprocess --problem 'deephyper.benchmark.nas.linearReg.problem.Problem' --run 'deephyper.search.nas.model.run.alpha.run'
或者使用MPI来启动n个代理，其中n = np，因为所有代理都是将与第一个代理同步：
mpirun -np 2 python ppo_a3c_async.py --problem deephyper.benchmark.nas.mnist1D.problem.Problem --run deephyper.search.nas.model.run.alpha.run --evaluator subprocess
完整代码实现
import os
import json
from pprint import pprint, pformat
from mpi4py import MPI
import math
from deephyper.evaluator import Evaluator
from deephyper.search import util, Search
from deephyper.search.nas.agent import nas_ppo_sync_a3c
logger = util.conf_logger('deephyper.search.nas.ppo_a3c_sync')
def print_logs(runner):
logger.debug('num_episodes = {}'.format(runner.global_episode))
logger.debug(' workers = {}'.format(runner.workers))
def key(d):
return json.dumps(dict(arch_seq=d['arch_seq']))
LAUNCHER_NODES = int(os.environ.get('BALSAM_LAUNCHER_NODES', 1))
WORKERS_PER_NODE = int(os.environ.get('DEEPHYPER_WORKERS_PER_NODE', 1))
[docs]class NasPPOSyncA3C(Search):
"""Neural Architecture search using proximal policy gradient with synchronous optimization.
"""
def __init__(self, problem, run, evaluator, **kwargs):
super().__init__(problem, run, evaluator, **kwargs)
# set in super : self.problem
# set in super : self.run_func
# set in super : self.evaluator
self.evaluator = Evaluator.create(self.run_func,
cache_key=key,
method=evaluator)
self.num_episodes = kwargs.get('num_episodes')
if self.num_episodes is None:
self.num_episodes = math.inf
self.reward_rule = util.load_attr_from('deephyper.search.nas.agent.utils.'+kwargs['reward_rule'])
self.space = self.problem.space
logger.debug(f'evaluator: {type(self.evaluator)}')
self.num_agents = MPI.COMM_WORLD.Get_size() - 1 # one is the parameter server
self.rank = MPI.COMM_WORLD.Get_rank()
logger.debug(f'num_agents: {self.num_agents}')
logger.debug(f'rank: {self.rank}')
@staticmethod
def _extend_parser(parser):
parser.add_argument('--num-episodes', type=int, default=None,
help='maximum number of episodes')
parser.add_argument('--reward-rule', type=str,
default='reward_for_final_timestep',
choices=[
'reward_for_all_timesteps',
'reward_for_final_timestep'
],
help='A function which describe how to spread the episodic reward on all timesteps of the corresponding episode.')
return parser
def main(self):
# Settings
#num_parallel = self.evaluator.num_workers - 4 #balsam launcher & controller of search for cooley
# num_nodes = self.evaluator.num_workers - 1 #balsam launcher & controller of search for theta
num_nodes = LAUNCHER_NODES * WORKERS_PER_NODE - 1 # balsam launcher
if num_nodes > self.num_agents:
num_episodes_per_batch = (num_nodes-self.num_agents)//self.num_agents
else:
num_episodes_per_batch = 1
if self.rank == 0:
logger.debug(f' num_nodes: {num_nodes}' )
logger.debug(f' num_episodes_per_batch: {num_episodes_per_batch}' )
logger.debug(f' starting training...' )
nas_ppo_sync_a3c.train(
num_episodes=self.num_episodes,
seed=2018,
space=self.problem.space,
evaluator=self.evaluator,
num_episodes_per_batch=num_episodes_per_batch,
reward_rule=self.reward_rule
)
if __name__ == "__main__":
args = NasPPOSyncA3C.parse_args()
search = NasPPOSyncA3C(**vars(args))
search.main()
（3）随机搜索 NAS Random
class deephyper.search.nas.random.NasRandom(problem, run, evaluator, **kwargs)
好了，下面先安装
From pip:
pip install deephyper
From github:
git clone https://github.com/deephyper/deephyper.git
cd deephyper/
pip install -e .
#if you want to install deephyper with test and documentation packages:
# From Pypi
pip install 'deephyper[tests,docs]'
# From github
git clone https://github.com/deephyper/deephyper.git
cd deephyper/
pip install -e '.[tests,docs]'
以上为本文内容，详细内容请学习文档内容
deephyper DocumentationRelease alpha v0.0.5 ：https://github.com/NeuronDance/DeepRL/blob/master/DRL书籍/电子书原版/deephyper.pdf
Github仓库：https://deephyper.readthedocs.io/en/latest/?badge=latest
参考文献：https://deephyper.readthedocs.io/en/latest/?badge=latest
相关内容已同步至知乎专栏：点击查看知乎专栏