weixin_47278555

paddlepaddle百度飞桨七日打卡营——强化学习parl

paddlepaddle百度飞桨七日打卡营——强化学习parl##

百度七日打卡营来到了强化学习领域，主要学习parl等工具，非常给力。话不多说，奥利给！

作业任务实现Sarsa算法玩迷宫游戏
预期效果：最终输出的 test reward 为1.0

加粗样式Step1 安装依赖

!pip install gym

Step2 导入依赖

import gym
import numpy as np
import time

Step3 Agent

Agent是和环境environment交互的主体。
predict()方法：输入观察值observation（或者说状态state），输出动作值
sample()方法：在predict()方法基础上使用ε-greedy增加探索
learn()方法：输入训练数据，完成一轮Q表格的更新

class SarsaAgent(object):
    def __init__(self, obs_n, act_n, learning_rate=0.01, gamma=0.9, e_greed=0.1):
        self.act_n = act_n      # 动作维度，有几个动作可选
        self.lr = learning_rate # 学习率
        self.gamma = gamma      # reward的衰减率
        self.epsilon = e_greed  # 按一定概率随机选动作
        self.Q = np.zeros((obs_n, act_n))

    # 根据输入观察值，采样输出的动作值，带探索
    def sample(self, obs):
        if np.random.uniform(0,1) < (1.0 - self.epsilon):
            action = self.predict(obs)
        else:
            action = np.random.choice(self.act_n)
        return action

    # 根据输入观察值，预测输出的动作值
    def predict(self, obs):
        Q_list = self.Q[obs, :]
        maxQ = np.max(Q_list)
        action_list = np.where(Q_list == maxQ)[0]
        action = np.random.choice(action_list)
        return action

    # 学习方法，也就是更新Q-table的方法
    def learn(self, obs, action, reward, next_obs, next_action, done):
        """ on-policy
            obs: 交互前的obs, s_t
            action: 本次交互选择的action, a_t
            reward: 本次动作获得的奖励r
            next_obs: 本次交互后的obs, s_t+1
            next_action: 根据当前Q表格, 针对next_obs会选择的动作, a_t+1
            done: episode是否结束
        """
        predict_Q = self.Q[obs,action]
        if done:
            target_Q = reward
        else:
            target_Q = reward + self.gamma * self.Q[next_obs,next_action]
        self.Q[obs,action] += self.lr * (target_Q - predict_Q)

    # 保存Q表格数据到文件
    def save(self):
        npy_file = './q_table.npy'
        np.save(npy_file, self.Q)
        print(npy_file + ' saved.')
    
    # 从文件中读取数据到Q表格中
    def restore(self, npy_file='./q_table.npy'):
        self.Q = np.load(npy_file)
        print(npy_file + ' loaded.')

Step4 Training && Test（训练&&测试）

def run_episode(env, agent, render=False):
    total_steps = 0 # 记录每个episode走了多少step
    total_reward = 0

    obs = env.reset() # 重置环境, 重新开一局（即开始新的一个episode）
    action = agent.sample(obs) # 根据算法选择一个动作

    while True:
        next_obs, reward, done, _ = env.step(action) # 与环境进行一个交互
        next_action = agent.sample(next_obs) # 根据算法选择一个动作
        # 训练 Sarsa 算法
        agent.learn(obs, action, reward, next_obs, next_action, done)

        action = next_action
        obs = next_obs  # 存储上一个观察值
        total_reward += reward
        total_steps += 1 # 计算step数
        if render:
            env.render() #渲染新的一帧图形
        if done:
            break
    return total_reward, total_steps

def test_episode(env, agent):
    total_reward = 0
    obs = env.reset()
    while True:
        action = agent.predict(obs) # greedy
        next_obs, reward, done, _ = env.step(action)
        total_reward += reward
        obs = next_obs
        # time.sleep(0.5)
        # env.render()
        if done:
            break
    return total_reward

Step 5 创建环境和Agent，启动训练

# 使用gym创建迷宫环境，设置is_slippery为False降低环境难度
env = gym.make("FrozenLake-v0", is_slippery=False)  # 0 left, 1 down, 2 right, 3 up

# 创建一个agent实例，输入超参数
agent = SarsaAgent(
        obs_n=env.observation_space.n,
        act_n=env.action_space.n,
        learning_rate=0.1,
        gamma=0.9,
        e_greed=0.1)


# 训练500个episode，打印每个episode的分数
for episode in range(500):
    ep_reward, ep_steps = run_episode(env, agent, False)
    print('Episode %s: steps = %s , reward = %.1f' % (episode, ep_steps, ep_reward))

# 全部训练结束，查看算法效果
test_reward = test_episode(env, agent)
print('test reward = %.1f' % (test_reward))

运行结果：
Episode 0: steps = 11 , reward = 0.0
Episode 1: steps = 8 , reward = 0.0
Episode 2: steps = 7 , reward = 0.0
Episode 3: steps = 5 , reward = 0.0
Episode 4: steps = 8 , reward = 0.0
Episode 5: steps = 2 , reward = 0.0
Episode 6: steps = 9 , reward = 0.0
Episode 7: steps = 2 , reward = 0.0
Episode 8: steps = 12 , reward = 0.0
Episode 9: steps = 2 , reward = 0.0
Episode 10: steps = 18 , reward = 0.0
Episode 11: steps = 2 , reward = 0.0
Episode 12: steps = 3 , reward = 0.0
Episode 13: steps = 5 , reward = 0.0
Episode 14: steps = 5 , reward = 0.0
Episode 15: steps = 5 , reward = 0.0
Episode 16: steps = 2 , reward = 0.0
Episode 17: steps = 11 , reward = 0.0
Episode 18: steps = 12 , reward = 0.0
Episode 19: steps = 7 , reward = 0.0
Episode 20: steps = 11 , reward = 0.0
Episode 21: steps = 19 , reward = 0.0
Episode 22: steps = 3 , reward = 0.0
Episode 23: steps = 4 , reward = 0.0
Episode 24: steps = 28 , reward = 1.0
Episode 25: steps = 6 , reward = 0.0
Episode 26: steps = 5 , reward = 0.0
Episode 27: steps = 2 , reward = 0.0
Episode 28: steps = 16 , reward = 0.0
Episode 29: steps = 11 , reward = 0.0
Episode 30: steps = 12 , reward = 0.0
Episode 31: steps = 2 , reward = 0.0
Episode 32: steps = 5 , reward = 0.0
Episode 33: steps = 9 , reward = 0.0
Episode 34: steps = 5 , reward = 0.0
Episode 35: steps = 9 , reward = 0.0
Episode 36: steps = 4 , reward = 0.0
Episode 37: steps = 6 , reward = 0.0
Episode 38: steps = 6 , reward = 0.0
Episode 39: steps = 3 , reward = 0.0
Episode 40: steps = 5 , reward = 0.0
Episode 41: steps = 4 , reward = 0.0
Episode 42: steps = 13 , reward = 0.0
Episode 43: steps = 8 , reward = 0.0
Episode 44: steps = 16 , reward = 0.0
Episode 45: steps = 20 , reward = 0.0
Episode 46: steps = 6 , reward = 0.0
Episode 47: steps = 9 , reward = 0.0
Episode 48: steps = 10 , reward = 0.0
Episode 49: steps = 26 , reward = 0.0
Episode 50: steps = 11 , reward = 0.0
Episode 51: steps = 28 , reward = 0.0
Episode 52: steps = 3 , reward = 0.0
Episode 53: steps = 9 , reward = 1.0
Episode 54: steps = 5 , reward = 0.0
Episode 55: steps = 17 , reward = 0.0
Episode 56: steps = 5 , reward = 0.0
Episode 57: steps = 9 , reward = 0.0
Episode 58: steps = 6 , reward = 1.0
Episode 59: steps = 12 , reward = 0.0
Episode 60: steps = 3 , reward = 0.0
Episode 61: steps = 3 , reward = 0.0
Episode 62: steps = 2 , reward = 0.0
Episode 63: steps = 15 , reward = 0.0
Episode 64: steps = 5 , reward = 0.0
Episode 65: steps = 10 , reward = 0.0
Episode 66: steps = 6 , reward = 0.0
Episode 67: steps = 4 , reward = 0.0
Episode 68: steps = 10 , reward = 1.0
Episode 69: steps = 12 , reward = 1.0
Episode 70: steps = 9 , reward = 1.0
Episode 71: steps = 6 , reward = 1.0
Episode 72: steps = 6 , reward = 1.0
Episode 73: steps = 3 , reward = 0.0
Episode 74: steps = 8 , reward = 1.0
Episode 75: steps = 6 , reward = 1.0
Episode 76: steps = 8 , reward = 0.0
Episode 77: steps = 9 , reward = 1.0
Episode 78: steps = 10 , reward = 1.0
Episode 79: steps = 6 , reward = 1.0
Episode 80: steps = 6 , reward = 1.0
Episode 81: steps = 6 , reward = 1.0
Episode 82: steps = 6 , reward = 1.0
Episode 83: steps = 6 , reward = 0.0
Episode 84: steps = 6 , reward = 1.0
Episode 85: steps = 6 , reward = 1.0
Episode 86: steps = 6 , reward = 1.0
Episode 87: steps = 6 , reward = 1.0
Episode 88: steps = 6 , reward = 1.0
Episode 89: steps = 6 , reward = 0.0
Episode 90: steps = 6 , reward = 1.0
Episode 91: steps = 5 , reward = 0.0
Episode 92: steps = 6 , reward = 1.0
Episode 93: steps = 6 , reward = 1.0
Episode 94: steps = 6 , reward = 1.0
Episode 95: steps = 6 , reward = 1.0
Episode 96: steps = 6 , reward = 1.0
Episode 97: steps = 6 , reward = 1.0
Episode 98: steps = 6 , reward = 1.0
Episode 99: steps = 6 , reward = 1.0
Episode 100: steps = 6 , reward = 1.0
Episode 101: steps = 6 , reward = 1.0
Episode 102: steps = 7 , reward = 1.0
Episode 103: steps = 6 , reward = 1.0
Episode 104: steps = 8 , reward = 1.0
Episode 105: steps = 7 , reward = 1.0
Episode 106: steps = 2 , reward = 0.0
Episode 107: steps = 2 , reward = 0.0
Episode 108: steps = 3 , reward = 0.0
Episode 109: steps = 6 , reward = 1.0
Episode 110: steps = 6 , reward = 1.0
Episode 111: steps = 6 , reward = 1.0
Episode 112: steps = 2 , reward = 0.0
Episode 113: steps = 6 , reward = 1.0
Episode 114: steps = 6 , reward = 1.0
Episode 115: steps = 6 , reward = 1.0
Episode 116: steps = 2 , reward = 0.0
Episode 117: steps = 6 , reward = 1.0
Episode 118: steps = 6 , reward = 1.0
Episode 119: steps = 6 , reward = 1.0
Episode 120: steps = 6 , reward = 1.0
Episode 121: steps = 6 , reward = 1.0
Episode 122: steps = 6 , reward = 1.0
Episode 123: steps = 6 , reward = 1.0
Episode 124: steps = 6 , reward = 1.0
Episode 125: steps = 2 , reward = 0.0
Episode 126: steps = 8 , reward = 1.0
Episode 127: steps = 6 , reward = 1.0
Episode 128: steps = 7 , reward = 1.0
Episode 129: steps = 2 , reward = 0.0
Episode 130: steps = 6 , reward = 1.0
Episode 131: steps = 6 , reward = 0.0
Episode 132: steps = 6 , reward = 1.0
Episode 133: steps = 6 , reward = 1.0
Episode 134: steps = 6 , reward = 0.0
Episode 135: steps = 6 , reward = 1.0
Episode 136: steps = 7 , reward = 1.0
Episode 137: steps = 6 , reward = 1.0
Episode 138: steps = 6 , reward = 1.0
Episode 139: steps = 4 , reward = 0.0
Episode 140: steps = 6 , reward = 1.0
Episode 141: steps = 6 , reward = 1.0
Episode 142: steps = 6 , reward = 1.0
Episode 143: steps = 6 , reward = 1.0
Episode 144: steps = 6 , reward = 1.0
Episode 145: steps = 6 , reward = 1.0
Episode 146: steps = 6 , reward = 1.0
Episode 147: steps = 8 , reward = 1.0
Episode 148: steps = 6 , reward = 1.0
Episode 149: steps = 6 , reward = 1.0
Episode 150: steps = 10 , reward = 1.0
Episode 151: steps = 6 , reward = 1.0
Episode 152: steps = 6 , reward = 1.0
Episode 153: steps = 6 , reward = 1.0
Episode 154: steps = 6 , reward = 1.0
Episode 155: steps = 7 , reward = 1.0
Episode 156: steps = 8 , reward = 1.0
Episode 157: steps = 6 , reward = 1.0
Episode 158: steps = 6 , reward = 1.0
Episode 159: steps = 4 , reward = 0.0
Episode 160: steps = 6 , reward = 1.0
Episode 161: steps = 6 , reward = 1.0
Episode 162: steps = 7 , reward = 1.0
Episode 163: steps = 6 , reward = 1.0
Episode 164: steps = 6 , reward = 1.0
Episode 165: steps = 6 , reward = 1.0
Episode 166: steps = 6 , reward = 1.0
Episode 167: steps = 6 , reward = 1.0
Episode 168: steps = 6 , reward = 1.0
Episode 169: steps = 4 , reward = 0.0
Episode 170: steps = 6 , reward = 1.0
Episode 171: steps = 6 , reward = 1.0
Episode 172: steps = 6 , reward = 1.0
Episode 173: steps = 8 , reward = 1.0
Episode 174: steps = 6 , reward = 1.0
Episode 175: steps = 6 , reward = 1.0
Episode 176: steps = 6 , reward = 1.0
Episode 177: steps = 6 , reward = 1.0
Episode 178: steps = 6 , reward = 1.0
Episode 179: steps = 10 , reward = 1.0
Episode 180: steps = 6 , reward = 1.0
Episode 181: steps = 6 , reward = 1.0
Episode 182: steps = 9 , reward = 1.0
Episode 183: steps = 6 , reward = 1.0
Episode 184: steps = 6 , reward = 1.0
Episode 185: steps = 6 , reward = 1.0
Episode 186: steps = 6 , reward = 1.0
Episode 187: steps = 9 , reward = 0.0
Episode 188: steps = 6 , reward = 1.0
Episode 189: steps = 4 , reward = 0.0
Episode 190: steps = 6 , reward = 1.0
Episode 191: steps = 6 , reward = 1.0
Episode 192: steps = 6 , reward = 1.0
Episode 193: steps = 6 , reward = 1.0
Episode 194: steps = 10 , reward = 1.0
Episode 195: steps = 4 , reward = 0.0
Episode 196: steps = 6 , reward = 1.0
Episode 197: steps = 6 , reward = 0.0
Episode 198: steps = 6 , reward = 1.0
Episode 199: steps = 6 , reward = 1.0
Episode 200: steps = 5 , reward = 0.0
Episode 201: steps = 6 , reward = 1.0
Episode 202: steps = 6 , reward = 1.0
Episode 203: steps = 8 , reward = 1.0
Episode 204: steps = 6 , reward = 1.0
Episode 205: steps = 6 , reward = 1.0
Episode 206: steps = 6 , reward = 1.0
Episode 207: steps = 6 , reward = 1.0
Episode 208: steps = 6 , reward = 1.0
Episode 209: steps = 6 , reward = 1.0
Episode 210: steps = 6 , reward = 1.0
Episode 211: steps = 6 , reward = 1.0
Episode 212: steps = 6 , reward = 1.0
Episode 213: steps = 6 , reward = 1.0
Episode 214: steps = 2 , reward = 0.0
Episode 215: steps = 6 , reward = 1.0
Episode 216: steps = 7 , reward = 1.0
Episode 217: steps = 6 , reward = 1.0
Episode 218: steps = 11 , reward = 0.0
Episode 219: steps = 9 , reward = 1.0
Episode 220: steps = 6 , reward = 1.0
Episode 221: steps = 6 , reward = 1.0
Episode 222: steps = 6 , reward = 1.0
Episode 223: steps = 7 , reward = 1.0
Episode 224: steps = 6 , reward = 1.0
Episode 225: steps = 6 , reward = 1.0
Episode 226: steps = 6 , reward = 1.0
Episode 227: steps = 6 , reward = 1.0
Episode 228: steps = 6 , reward = 1.0
Episode 229: steps = 8 , reward = 1.0
Episode 230: steps = 6 , reward = 1.0
Episode 231: steps = 8 , reward = 1.0
Episode 232: steps = 6 , reward = 1.0
Episode 233: steps = 6 , reward = 1.0
Episode 234: steps = 6 , reward = 1.0
Episode 235: steps = 6 , reward = 1.0
Episode 236: steps = 6 , reward = 1.0
Episode 237: steps = 6 , reward = 1.0
Episode 238: steps = 6 , reward = 1.0
Episode 239: steps = 6 , reward = 1.0
Episode 240: steps = 6 , reward = 1.0
Episode 241: steps = 10 , reward = 1.0
Episode 242: steps = 6 , reward = 1.0
Episode 243: steps = 6 , reward = 1.0
Episode 244: steps = 6 , reward = 1.0
Episode 245: steps = 6 , reward = 1.0
Episode 246: steps = 6 , reward = 1.0
Episode 247: steps = 6 , reward = 1.0
Episode 248: steps = 6 , reward = 0.0
Episode 249: steps = 6 , reward = 1.0
Episode 250: steps = 6 , reward = 1.0
Episode 251: steps = 6 , reward = 1.0
Episode 252: steps = 6 , reward = 1.0
Episode 253: steps = 6 , reward = 1.0
Episode 254: steps = 6 , reward = 1.0
Episode 255: steps = 7 , reward = 1.0
Episode 256: steps = 7 , reward = 1.0
Episode 257: steps = 6 , reward = 1.0
Episode 258: steps = 6 , reward = 1.0
Episode 259: steps = 6 , reward = 1.0
Episode 260: steps = 11 , reward = 1.0
Episode 261: steps = 6 , reward = 1.0
Episode 262: steps = 10 , reward = 1.0
Episode 263: steps = 6 , reward = 1.0
Episode 264: steps = 6 , reward = 1.0
Episode 265: steps = 7 , reward = 1.0
Episode 266: steps = 2 , reward = 0.0
Episode 267: steps = 6 , reward = 1.0
Episode 268: steps = 6 , reward = 1.0
Episode 269: steps = 10 , reward = 1.0
Episode 270: steps = 6 , reward = 1.0
Episode 271: steps = 6 , reward = 1.0
Episode 272: steps = 6 , reward = 1.0
Episode 273: steps = 2 , reward = 0.0
Episode 274: steps = 8 , reward = 1.0
Episode 275: steps = 7 , reward = 1.0
Episode 276: steps = 6 , reward = 1.0
Episode 277: steps = 6 , reward = 1.0
Episode 278: steps = 6 , reward = 1.0
Episode 279: steps = 6 , reward = 0.0
Episode 280: steps = 6 , reward = 1.0
Episode 281: steps = 10 , reward = 1.0
Episode 282: steps = 7 , reward = 1.0
Episode 283: steps = 7 , reward = 1.0
Episode 284: steps = 6 , reward = 1.0
Episode 285: steps = 4 , reward = 0.0
Episode 286: steps = 6 , reward = 1.0
Episode 287: steps = 5 , reward = 0.0
Episode 288: steps = 6 , reward = 1.0
Episode 289: steps = 7 , reward = 1.0
Episode 290: steps = 6 , reward = 0.0
Episode 291: steps = 8 , reward = 1.0
Episode 292: steps = 6 , reward = 1.0
Episode 293: steps = 6 , reward = 1.0
Episode 294: steps = 6 , reward = 1.0
Episode 295: steps = 8 , reward = 1.0
Episode 296: steps = 6 , reward = 1.0
Episode 297: steps = 5 , reward = 0.0
Episode 298: steps = 7 , reward = 1.0
Episode 299: steps = 6 , reward = 1.0
Episode 300: steps = 4 , reward = 0.0
Episode 301: steps = 6 , reward = 1.0
Episode 302: steps = 8 , reward = 1.0
Episode 303: steps = 6 , reward = 1.0
Episode 304: steps = 6 , reward = 1.0
Episode 305: steps = 6 , reward = 1.0
Episode 306: steps = 6 , reward = 1.0
Episode 307: steps = 7 , reward = 1.0
Episode 308: steps = 2 , reward = 0.0
Episode 309: steps = 12 , reward = 1.0
Episode 310: steps = 8 , reward = 1.0
Episode 311: steps = 6 , reward = 1.0
Episode 312: steps = 6 , reward = 1.0
Episode 313: steps = 8 , reward = 1.0
Episode 314: steps = 9 , reward = 1.0
Episode 315: steps = 5 , reward = 0.0
Episode 316: steps = 2 , reward = 0.0
Episode 317: steps = 8 , reward = 1.0
Episode 318: steps = 10 , reward = 1.0
Episode 319: steps = 7 , reward = 1.0
Episode 320: steps = 6 , reward = 1.0
Episode 321: steps = 6 , reward = 1.0
Episode 322: steps = 6 , reward = 1.0
Episode 323: steps = 6 , reward = 1.0
Episode 324: steps = 6 , reward = 1.0
Episode 325: steps = 6 , reward = 1.0
Episode 326: steps = 6 , reward = 1.0
Episode 327: steps = 5 , reward = 0.0
Episode 328: steps = 2 , reward = 0.0
Episode 329: steps = 6 , reward = 1.0
Episode 330: steps = 2 , reward = 0.0
Episode 331: steps = 6 , reward = 1.0
Episode 332: steps = 6 , reward = 1.0
Episode 333: steps = 6 , reward = 1.0
Episode 334: steps = 6 , reward = 1.0
Episode 335: steps = 6 , reward = 1.0
Episode 336: steps = 10 , reward = 1.0
Episode 337: steps = 7 , reward = 1.0
Episode 338: steps = 7 , reward = 1.0
Episode 339: steps = 6 , reward = 1.0
Episode 340: steps = 6 , reward = 1.0
Episode 341: steps = 6 , reward = 1.0
Episode 342: steps = 9 , reward = 1.0
Episode 343: steps = 6 , reward = 1.0
Episode 344: steps = 10 , reward = 1.0
Episode 345: steps = 6 , reward = 1.0
Episode 346: steps = 9 , reward = 1.0
Episode 347: steps = 6 , reward = 1.0
Episode 348: steps = 9 , reward = 1.0
Episode 349: steps = 6 , reward = 1.0
Episode 350: steps = 8 , reward = 1.0
Episode 351: steps = 6 , reward = 1.0
Episode 352: steps = 8 , reward = 1.0
Episode 353: steps = 6 , reward = 1.0
Episode 354: steps = 6 , reward = 1.0
Episode 355: steps = 6 , reward = 1.0
Episode 356: steps = 8 , reward = 1.0
Episode 357: steps = 5 , reward = 0.0
Episode 358: steps = 4 , reward = 0.0
Episode 359: steps = 8 , reward = 1.0
Episode 360: steps = 8 , reward = 1.0
Episode 361: steps = 8 , reward = 1.0
Episode 362: steps = 4 , reward = 0.0
Episode 363: steps = 6 , reward = 1.0
Episode 364: steps = 6 , reward = 1.0
Episode 365: steps = 6 , reward = 1.0
Episode 366: steps = 8 , reward = 1.0
Episode 367: steps = 7 , reward = 1.0
Episode 368: steps = 6 , reward = 1.0
Episode 369: steps = 8 , reward = 1.0
Episode 370: steps = 6 , reward = 0.0
Episode 371: steps = 6 , reward = 1.0
Episode 372: steps = 6 , reward = 1.0
Episode 373: steps = 6 , reward = 1.0
Episode 374: steps = 6 , reward = 1.0
Episode 375: steps = 6 , reward = 1.0
Episode 376: steps = 8 , reward = 1.0
Episode 377: steps = 5 , reward = 0.0
Episode 378: steps = 7 , reward = 0.0
Episode 379: steps = 6 , reward = 1.0
Episode 380: steps = 6 , reward = 1.0
Episode 381: steps = 6 , reward = 1.0
Episode 382: steps = 2 , reward = 0.0
Episode 383: steps = 6 , reward = 1.0
Episode 384: steps = 7 , reward = 1.0
Episode 385: steps = 8 , reward = 1.0
Episode 386: steps = 12 , reward = 1.0
Episode 387: steps = 6 , reward = 1.0
Episode 388: steps = 8 , reward = 1.0
Episode 389: steps = 11 , reward = 1.0
Episode 390: steps = 6 , reward = 1.0
Episode 391: steps = 6 , reward = 1.0
Episode 392: steps = 7 , reward = 1.0
Episode 393: steps = 6 , reward = 1.0
Episode 394: steps = 4 , reward = 0.0
Episode 395: steps = 6 , reward = 1.0
Episode 396: steps = 6 , reward = 1.0
Episode 397: steps = 6 , reward = 1.0
Episode 398: steps = 5 , reward = 0.0
Episode 399: steps = 6 , reward = 1.0
Episode 400: steps = 6 , reward = 1.0
Episode 401: steps = 6 , reward = 1.0
Episode 402: steps = 6 , reward = 1.0
Episode 403: steps = 8 , reward = 1.0
Episode 404: steps = 6 , reward = 1.0
Episode 405: steps = 6 , reward = 1.0
Episode 406: steps = 6 , reward = 1.0
Episode 407: steps = 6 , reward = 1.0
Episode 408: steps = 7 , reward = 0.0
Episode 409: steps = 6 , reward = 1.0
Episode 410: steps = 8 , reward = 1.0
Episode 411: steps = 6 , reward = 1.0
Episode 412: steps = 8 , reward = 1.0
Episode 413: steps = 6 , reward = 1.0
Episode 414: steps = 6 , reward = 1.0
Episode 415: steps = 6 , reward = 1.0
Episode 416: steps = 6 , reward = 1.0
Episode 417: steps = 6 , reward = 1.0
Episode 418: steps = 7 , reward = 1.0
Episode 419: steps = 6 , reward = 1.0
Episode 420: steps = 7 , reward = 1.0
Episode 421: steps = 8 , reward = 1.0
Episode 422: steps = 6 , reward = 1.0
Episode 423: steps = 6 , reward = 1.0
Episode 424: steps = 9 , reward = 1.0
Episode 425: steps = 6 , reward = 1.0
Episode 426: steps = 7 , reward = 1.0
Episode 427: steps = 5 , reward = 0.0
Episode 428: steps = 6 , reward = 1.0
Episode 429: steps = 6 , reward = 1.0
Episode 430: steps = 6 , reward = 1.0
Episode 431: steps = 4 , reward = 0.0
Episode 432: steps = 6 , reward = 1.0
Episode 433: steps = 6 , reward = 1.0
Episode 434: steps = 6 , reward = 1.0
Episode 435: steps = 6 , reward = 1.0
Episode 436: steps = 6 , reward = 1.0
Episode 437: steps = 6 , reward = 1.0
Episode 438: steps = 7 , reward = 1.0
Episode 439: steps = 6 , reward = 1.0
Episode 440: steps = 6 , reward = 1.0
Episode 441: steps = 6 , reward = 1.0
Episode 442: steps = 6 , reward = 1.0
Episode 443: steps = 8 , reward = 1.0
Episode 444: steps = 6 , reward = 1.0
Episode 445: steps = 6 , reward = 1.0
Episode 446: steps = 6 , reward = 1.0
Episode 447: steps = 6 , reward = 1.0
Episode 448: steps = 6 , reward = 1.0
Episode 449: steps = 8 , reward = 1.0
Episode 450: steps = 6 , reward = 1.0
Episode 451: steps = 6 , reward = 1.0
Episode 452: steps = 9 , reward = 1.0
Episode 453: steps = 6 , reward = 1.0
Episode 454: steps = 4 , reward = 0.0
Episode 455: steps = 8 , reward = 1.0
Episode 456: steps = 10 , reward = 1.0
Episode 457: steps = 6 , reward = 1.0
Episode 458: steps = 2 , reward = 0.0
Episode 459: steps = 6 , reward = 1.0
Episode 460: steps = 9 , reward = 1.0
Episode 461: steps = 6 , reward = 1.0
Episode 462: steps = 6 , reward = 1.0
Episode 463: steps = 9 , reward = 1.0
Episode 464: steps = 6 , reward = 1.0
Episode 465: steps = 6 , reward = 1.0
Episode 466: steps = 6 , reward = 1.0
Episode 467: steps = 9 , reward = 1.0
Episode 468: steps = 6 , reward = 1.0
Episode 469: steps = 6 , reward = 1.0
Episode 470: steps = 6 , reward = 1.0
Episode 471: steps = 6 , reward = 1.0
Episode 472: steps = 2 , reward = 0.0
Episode 473: steps = 6 , reward = 1.0
Episode 474: steps = 8 , reward = 1.0
Episode 475: steps = 6 , reward = 0.0
Episode 476: steps = 6 , reward = 1.0
Episode 477: steps = 8 , reward = 1.0
Episode 478: steps = 7 , reward = 1.0
Episode 479: steps = 6 , reward = 1.0
Episode 480: steps = 6 , reward = 1.0
Episode 481: steps = 8 , reward = 1.0
Episode 482: steps = 7 , reward = 1.0
Episode 483: steps = 8 , reward = 1.0
Episode 484: steps = 6 , reward = 1.0
Episode 485: steps = 2 , reward = 0.0
Episode 486: steps = 6 , reward = 1.0
Episode 487: steps = 6 , reward = 1.0
Episode 488: steps = 7 , reward = 1.0
Episode 489: steps = 6 , reward = 1.0
Episode 490: steps = 6 , reward = 1.0
Episode 491: steps = 2 , reward = 0.0
Episode 492: steps = 6 , reward = 1.0
Episode 493: steps = 6 , reward = 1.0
Episode 494: steps = 6 , reward = 1.0
Episode 495: steps = 6 , reward = 1.0
Episode 496: steps = 6 , reward = 1.0
Episode 497: steps = 8 , reward = 1.0
Episode 498: steps = 4 , reward = 0.0
Episode 499: steps = 6 , reward = 1.0
test reward = 1.0
完全收敛到1.0，OK

android系统selinux中添加新属性property 辉色投像
1.定位/android/system/sepolicy/private/property_contexts声明属性开头：persist.charge声明属性类型：u:object_r:system_prop:s0图12.定位到android/system/sepolicy/public/domain.te删除neverallow{domain-init}default_prop:property
C语言宏函数南林yan C语言 c语言
一、什么是宏函数？通过宏定义的函数是宏函数。如下，编译器在预处理阶段会将Add(x,y)替换为((x)*(y))#defineAdd(x,y)((x)*(y))#defineAdd(x,y)((x)*(y))intmain(){inta=10;intb=20;intd=10;intc=Add(a+d,b)*2;cout<
c++ 的iostream 和 c++的stdio的区别和联系黄卷青灯77 c++算法开发语言 iostream stdio
在C++中，iostream和C语言的stdio.h都是用于处理输入输出的库，但它们在设计、用法和功能上有许多不同。以下是两者的区别和联系：区别1.编程风格iostream（C++风格）：C++标准库中的输入输出流类库，支持面向对象的输入输出操作。典型用法是cin（输入）和cout（输出），使用>操作符来处理数据。更加类型安全，支持用户自定义类型的输入输出。#includeintmain(){in
LocalDateTime 转 String igotyback java 开发语言
importjava.time.LocalDateTime;importjava.time.format.DateTimeFormatter;publicclassMain{publicstaticvoidmain(String[]args){//获取当前时间LocalDateTimenow=LocalDateTime.now();//定义日期格式化器DateTimeFormatterformat
绘本讲师训练营【24期】8/21阅读原创《独生小孩》 1784e22615e0
24016-孟娟《独生小孩》图片发自App今天我想分享一个蛮特别的绘本，讲的是一个特殊的群体，我也是属于这个群体，80后的独生小孩。这是一本中国绘本，作者郭婧，也是一个80厚。全书一百多页，均为铅笔绘制，虽然为黑白色调，但并不显得沉闷。全书没有文字，犹如“默片”，但并不影响读者对该作品的理解，反而显得神秘，梦幻，給读者留下想象的空间。作者在前蝴蝶页这样写到：“我更希望父母和孩子一起分享这本书，使他
店群合一模式下的社区团购新发展——结合链动 2+1 模式、AI 智能名片与 S2B2C 商城小程序源码说私域人工智能小程序
摘要：本文探讨了店群合一的社区团购平台在当今商业环境中的重要性和优势。通过分析店群合一模式如何将互联网社群与线下终端紧密结合，阐述了链动2+1模式、AI智能名片和S2B2C商城小程序源码在这一模式中的应用价值。这些创新元素的结合为社区团购带来了新的机遇，提升了用户信任感、拓展了营销渠道，并实现了线上线下的完美融合。一、引言随着互联网技术的不断发展，社区团购作为一种新兴的商业模式，在满足消费者日常需
每日一题——第八十九题互联网打工人no1 C语言程序设计每日一练 c语言
题目：在字符串中找到提取数字，并统计一共找到多少整数，a123xxyu23&8889，那么找到的整数为123，23，8889//思想：#include#include#includeintmain(){charstr[]="a123xxyu23&8889";intcount=0;intnum=0;//用于临时存放当前正在构建的整数。boolinNum=false;//用于标记当前是否正在读取一个整
每日一题——第八十一题互联网打工人no1 C语言程序设计每日一练 c语言
打印如下图案:#includeintmain(){inti,j;charch='A';for(i=1;i<5;i++,ch++){for(j=0;j<5-i;j++){printf("");//控制空格输出}for(j=1;j<2*i;j++)//条件j<2*i{printf("%c",ch);//控制字符输出}printf("\n");}return0;}
2022-04-18 Apbenz
语重心长的和我说，不要老是说不行，人至而立之年危机四伏，内在的，外在的，感觉就是心力憔悴，让人无所适从。面对职场的无情，突然好羡慕干体力劳动的外卖小哥。难道命运是想让我去送外卖了吗？干体力活才能让我活下去？fastadmin打卡成功,淘宝金币任务完成。ㅏㅓㅗㅜㅡㅣㅐㅔㅑㅕㅛㅠㅢㅒㅖY行。야자여자요리우유의사얘기예
每日一题——第八十二题互联网打工人no1 C语言程序设计每日一练 c语言
题目：将一个控制台输入的字符串中的所有元音字母复制到另一字符串中#include#include#include#include#defineMAX_INPUT1024boolisVowel(charp);intmain(){charinput[MAX_INPUT];charoutput[MAX_INPUT];printf("请输入一串字符串：\n");fgets(input,sizeof(inp
每日一题——第八十三题互联网打工人no1 C语言程序设计每日一练 c语言
题目：将输入的整形数字输出,输出1990，输出"1990"#include#defineMAX_INPUT1024intmain(){intarrr_num[MAX_INPUT];intnum,i=0;printf("请输入一个数字：");scanf_s("%d",&num);while(num!=0){arrr_num[i++]=num%10;num/=10;}printf("\"");for(
网易严选官方旗舰店，优质商品，卓越服务高省_飞智666600
网易严选官方旗舰店是网易旗下的一家电商平台，以提供优质商品和卓越服务而闻名。作为一名SEO优化师，我将为您详细介绍网易严选官方旗舰店，并重点强调其特点和优势。大家好！我是高省APP最大团队&联合创始人飞智导师。相较于其他返利app，高省APP的佣金更高，模式更好，最重要的是，终端用户不会流失！高省APP佣金更高，模式更好，终端用户不流失。【高省】是一个自用省钱佣金高，分享推广赚钱多的平台，百度有几
git常用命令笔记咩酱-小羊 git 笔记
###用习惯了idea总是不记得git的一些常见命令，需要用到的时候总是担心旁边站了人~~~记个笔记@_@，告诉自己看笔记不丢人初始化初始化一个新的Git仓库gitinit配置配置用户信息gitconfig--globaluser.name"YourName"gitconfig--globaluser.email"[email protected]"基本操作克隆远程仓库gitclone查看
将cmd中命令输出保存为txt文本文件落难Coder Windows cmd window
最近深度学习本地的训练中我们常常要在命令行中运行自己的代码，无可厚非，我们有必要保存我们的炼丹结果，但是复制命令行输出到txt是非常麻烦的，其实Windows下的命令行为我们提供了相应的操作。其基本的调用格式就是：运行指令>输出到的文件名称或者具体保存路径测试下，我打开cmd并且ping一下百度：pingwww.baidu.com>./data.txt看下相同目录下data.txt的输出：如果你再
直返最高等级与直返APP：无需邀请码的返利新体验古楼
随着互联网的普及和电商的兴起，直返模式逐渐成为一种流行的商业模式。在这种模式下，消费者通过购买产品或服务，获得一定的返利，并可以分享给更多的人。其中，直返最高等级和直返APP是直返模式中的重要概念和工具。本文将详细介绍直返最高等级的概念、直返APP的使用以及与邀请码的关系。【高省】APP（高佣金领导者）是一个自用省钱佣金高，分享推广赚钱多的平台，百度有几百万篇报道，运行三年，稳定可靠。高省APP，
探索OpenAI和LangChain的适配器集成：轻松切换模型提供商 nseejrukjhad langchain easyui 前端 python
#探索OpenAI和LangChain的适配器集成：轻松切换模型提供商##引言在人工智能和自然语言处理的世界中，OpenAI的模型提供了强大的能力。然而，随着技术的发展，许多人开始探索其他模型以满足特定需求。LangChain作为一个强大的工具，集成了多种模型提供商，通过提供适配器，简化了不同模型之间的转换。本篇文章将介绍如何使用LangChain的适配器与OpenAI集成，以便轻松切换模型提供商
使用Faiss进行高效相似度搜索 llzwxh888 faiss python
在现代AI应用中，快速和高效的相似度搜索是至关重要的。Faiss（FacebookAISimilaritySearch）是一个专门用于快速相似度搜索和聚类的库，特别适用于高维向量。本文将介绍如何使用Faiss来进行相似度搜索，并结合Python代码演示其基本用法。什么是Faiss？Faiss是一个由FacebookAIResearch团队开发的开源库，主要用于高维向量的相似性搜索和聚类。Faiss
利用LangChain的StackExchange组件实现智能问答系统 nseejrukjhad langchain microsoft 数据库 python
利用LangChain的StackExchange组件实现智能问答系统引言在当今的软件开发世界中，StackOverflow已经成为程序员解决问题的首选平台之一。而LangChain作为一个强大的AI应用开发框架，提供了StackExchange组件，使我们能够轻松地将StackOverflow的海量知识库集成到我们的应用中。本文将详细介绍如何使用LangChain的StackExchange组件
如何部分格式化提示模板:LangChain中的高级技巧 nseejrukjhad langchain java 服务器 python
标题:如何部分格式化提示模板:LangChain中的高级技巧内容:如何部分格式化提示模板:LangChain中的高级技巧引言在使用大型语言模型(LLM)时,提示工程是一个关键环节。LangChain提供了强大的提示模板功能,让我们能更灵活地构建和管理提示。本文将介绍LangChain中一个高级特性-部分格式化提示模板,这个技巧可以让你的提示管理更加高效和灵活。什么是部分格式化提示模板?部分格式化提
人工智能时代，程序员如何保持核心竞争力？ jmoych 人工智能
随着AIGC（如chatgpt、midjourney、claude等）大语言模型接二连三的涌现，AI辅助编程工具日益普及，程序员的工作方式正在发生深刻变革。有人担心AI可能取代部分编程工作，也有人认为AI是提高效率的得力助手。面对这一趋势,程序员应该如何应对?是专注于某个领域深耕细作，还是广泛学习以适应快速变化的技术环境?又或者，我们是否应该将重点转向AI无法轻易替代的软技能？让我们一起探讨程序员
2022现在哪个打车软件比较好用又便宜实惠的打车软件合集高省APP珊珊
这是一个信息高速传播的社会。信息可以通过手机，微信，自媒体，抖音等方式进行传播。但同时这也是一个交通四通发达的社会。高省APP，是2022年推出的平台，0投资，0风险、高省APP佣金更高，模式更好，终端用户不流失。【高省】是一个自用省钱佣金高，分享推广赚钱多的平台，百度有几百万篇报道，也期待你的加入。珊珊导师，高省邀请码777777，注册送2皇冠会员，送万元推广大礼包，教你如何1年做到百万团队。高
Faiss Tips：高效向量搜索与聚类的利器焦习娜Samantha
FaissTips：高效向量搜索与聚类的利器faiss_tipsSomeusefultipsforfaiss项目地址:https://gitcode.com/gh_mirrors/fa/faiss_tips项目介绍Faiss是由FacebookAIResearch开发的一个用于高效相似性搜索和密集向量聚类的库。它支持多种硬件平台，包括CPU和GPU，能够在海量数据集上实现快速的近似最近邻搜索（AN
pyecharts——绘制柱形图折线图 2224070247 信息可视化 python java 数据可视化
一、pyecharts概述自2013年6月百度EFE(ExcellentFrontEnd）数据可视化团队研发的ECharts1.0发布到GitHub网站以来，ECharts一直备受业界权威的关注并获得广泛好评，成为目前成熟且流行的数据可视化图表工具，被应用到诸多数据可视化的开发领域。Python作为数据分析领域最受欢迎的语言，也加入ECharts的使用行列，并研发出方便Python开发者使用的数据
如何成为段子手欣雅阅读
我是一个尬聊大师，与朋友聊天经常把话题聊死，留我一个人在群里，望着自己打下的最后一句话无语凝噎。看到风趣幽默的朋友与人聊天，很是艳羡，觉得自己何时才能成为这样的段子手呢？一、段子是什么？“段子”一词在百度百科上的解释：本是相声中的一个艺术术语，指的是相声作品中一节或一段艺术内容。我的理解：段子就是一些搞笑的故事或者笑话。二、为什么要会说段子？不知道大家有没有这样的朋友，本来很无趣的聚会，只要有他参
Python爬虫解析工具之xpath使用详解 eqa11 python 爬虫开发语言
文章目录Python爬虫解析工具之xpath使用详解一、引言二、环境准备1、插件安装2、依赖库安装三、xpath语法详解1、路径表达式2、通配符3、谓语4、常用函数四、xpath在Python代码中的使用1、文档树的创建2、使用xpath表达式3、获取元素内容和属性五、总结Python爬虫解析工具之xpath使用详解一、引言在Python爬虫开发中，数据提取是一个至关重要的环节。xpath作为一门
展现思维导图魅力，不断挖掘人生宝藏思维导图讲师Mandy
第13期最强思维导图训练营已经结束一周了，但是我依旧是感觉所有学员还在努力的学习，这些学员中有教师、学生、白领、公务员、宝妈等等，只要你努力，只要你想改变自己，任何行业，任何岗位都可以参与进来，28天足以让你见成效，在这28天中，我们的学员不仅仅是收获了一枚毕业证，最重要的是让自己的思维方式得到升级，今天的你为自己投资，明天的你就会感谢你今天的付出，我们来听一听来自13期最强思维导图训练营优秀学员
【PG】常见数据库、表属性设置江无羡数据库
PG的常见属性配置方法数据库复制、备份相关表的复制标识单表操作批量表操作链接数据库复制、备份相关表的复制标识单表操作通过ALTER语句单独更改一张表的复制标识。ALTERTABLE[tablename]REPLICAIDENTITYFULL;批量表操作通过代码块的方式，对某个schema中的所有表一起更新其复制标识。SELECTtablename,CASErelreplidentWHEN'd'TH
2019-3-23晨间日记红红火火小耳朵
今天是什么日子起床：7点40就寝：23点半天气：有太阳，不过一会儿出来一会儿进去特别清爽的凉意，还蛮舒服的心情：小激动要给女朋友过生日啦纪念日：田田女士过生日任务清单昨日完成的任务，最重要的三件事：1.英语一对一2.运动计划3.认真护肤习惯养成：调整状态周目标·完成进度英语七天打卡（5/7）轻课阅读（87/180）音标课（25/30）读书（福尔摩斯一章）学习·信息·阅读#英语课#Cookingte
Faiss：高效相似性搜索与聚类的利器网络·魚大数据 faiss
Faiss是一个针对大规模向量集合的相似性搜索库，由FacebookAIResearch开发。它提供了一系列高效的算法和数据结构，用于加速向量之间的相似性搜索，特别是在大规模数据集上。本文将介绍Faiss的原理、核心功能以及如何在实际项目中使用它。Faiss原理：近似最近邻搜索：Faiss的核心功能之一是近似最近邻搜索，它能够高效地在大规模数据集中找到与给定查询向量最相似的向量。这种搜索是近似的，
每日一题——第八十八题互联网打工人no1 C语言程序设计每日一练 c语言
题目：输入一个9位的无符号整数，判断其是否有重复数字#include#include#includeintmain(){charnum_str[10];printf("请输入一个9位数的无符号数：");scanf_s("%9d",&num_str);if(strlen(num_str)!=9){printf("输入的不是一个9位无符号整数，请重新输入");}else{if(hasDuplicate
Spring4.1新特性——综述 jinnianshilongnian spring 4.1
目录 Spring4.1新特性——综述 Spring4.1新特性——Spring核心部分及其他 Spring4.1新特性——Spring缓存框架增强 Spring4.1新特性——异步调用和事件机制的异常处理 Spring4.1新特性——数据库集成测试脚本初始化 Spring4.1新特性——Spring MVC增强 Spring4.1新特性——页面自动化测试框架Spring MVC T
Schema与数据类型优化 annan211 数据结构 mysql
目前商城的数据库设计真是一塌糊涂，表堆叠让人不忍直视，无脑的架构师，说了也不听。在数据库设计之初，就应该仔细揣摩可能会有哪些查询，有没有更复杂的查询，而不是仅仅突出很表面的业务需求，这样做会让你的数据库性能成倍提高，当然，丑陋的架构师是不会这样去考虑问题的。选择优化的数据类型 1 更小的通常更好更小的数据类型通常更快，因为他们占用更少的磁盘、内存和cpu缓存，
第一节 HTML概要学习 chenke html Web css
第一节 HTML概要学习 1. 什么是HTML HTML是英文Hyper Text Mark-up Language(超文本标记语言)的缩写，它规定了自己的语法规则，用来表示比“文本”更丰富的意义，比如图片，表格，链接等。浏览器（IE,FireFox等）软件知道HTML语言的语法，可以用来查看HTML文档。目前互联网上的绝大部分网页都是使用HTML编写的。打开记事本输入一下内
MyEclipse里部分习惯的更改 Array_06 eclipse
继续补充中---------------------- 1.更改自己合适快捷键windows-->prefences-->java-->editor-->Content Assist--> Activation triggers for java的右侧“.”就可以改变常用的快捷键选中 Text
近一个月的面试总结 cugfy 面试
本文是在学习中的总结，欢迎转载但请注明出处：http://blog.csdn.net/pistolove/article/details/46753275 前言打算换个工作，近一个月面试了不少的公司，下面将一些面试经验和思考分享给大家。另外校招也快要开始了，为在校的学生提供一些经验供参考，希望都能找到满意的工作。
HTML5一个小迷宫游戏 357029540 html5
通过《HTML5游戏开发》摘抄了一个小迷宫游戏，感觉还不错，可以画画，写字，把摘抄的代码放上来分享下，喜欢的同学可以拿来玩玩！ <html> <head> <title>创建运行迷宫</title> <script type="text/javascript"
10步教你上传githib数据张亚雄 git
官方的教学还有其他博客里教的都是给懂的人说得，对已我们这样对我大菜鸟只能这么来锻炼，下面先不玩什么深奥的，先暂时用着10步干净利索。等玩顺溜了再用其他的方法。操作过程（查看本目录下有哪些文件NO.1）ls （跳转到子目录NO.2）cd+空格+目录（继续NO.3）ls （匹配到子目录NO.4）cd+ 目录首写字母+tab键+（首写字母“直到你所用文件根就不再按TAB键了”）（查看文件
MongoDB常用操作命令大全 adminjun mongodb 操作命令
成功启动MongoDB后，再打开一个命令行窗口输入mongo，就可以进行数据库的一些操作。输入help可以看到基本操作命令，只是MongoDB没有创建数据库的命令，但有类似的命令如：如果你想创建一个“myTest”的数据库，先运行use myTest命令，之后就做一些操作（如：db.createCollection('user')）,这样就可以创建一个名叫“myTest”的数据库。一
bat调用jar包并传入多个参数 aijuans
下面的主程序是通过eclipse写的： 1.在Main函数接收bat文件传递的参数（String[] args）如： String ip =args[0]; String user=args[1]; &nbs
Java中对类的主动引用和被动引用 ayaoxinchao java 主动引用对类的引用被动引用类初始化
在Java代码中，有些类看上去初始化了，但其实没有。例如定义一定长度某一类型的数组，看上去数组中所有的元素已经被初始化，实际上一个都没有。对于类的初始化，虚拟机规范严格规定了只有对该类进行主动引用时，才会触发。而除此之外的所有引用方式称之为对类的被动引用，不会触发类的初始化。虚拟机规范严格地规定了有且仅有四种情况是对类的主动引用，即必须立即对类进行初始化。四种情况如下：1.遇到ne
导出数据库提示 outfile disabled BigBird2012 mysql
在windows控制台下，登陆mysql，备份数据库： mysql>mysqldump -u root -p test test > D:\test.sql 使用命令 mysqldump 格式如下： mysqldump -u root -p *** DBNAME > E:\\test.sql。注意：执行该命令的时候不要进入mysql的控制台再使用，这样会报
Javascript 中的 && 和 || bijian1013 JavaScript &&||
准备两个对象用于下面的讨论 var alice = { name: "alice", toString: function () { return this.name; } } var smith = { name: "smith",
[Zookeeper学习笔记之四]Zookeeper Client Library会话重建 bit1129 zookeeper
为了说明问题，先来看个简单的示例代码： package com.tom.zookeeper.book; import com.tom.Host; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.ZooKeeper; import org.apache.zookeeper.Wat
【Scala十一】Scala核心五：case模式匹配 bit1129 scala
package spark.examples.scala.grammars.caseclasses object CaseClass_Test00 { def simpleMatch(arg: Any) = arg match { case v: Int => "This is an Int" case v: (Int, String)
运维的一些面试题 yuxianhua linux
1、Linux挂载Winodws共享文件夹 mount -t cifs //1.1.1.254/ok /var/tmp/share/ -o username=administrator,password=yourpass 或 mount -t cifs -o username=xxx,password=xxxx //1.1.1.1/a /win
Java lang包-Boolean BrokenDreams boolean
Boolean类是Java中基本类型boolean的包装类。这个类比较简单，直接看源代码吧。 public final class Boolean implements java.io.Serializable,
读《研磨设计模式》-代码笔记-命令模式-Command bylijinnan java 设计模式
声明：本文只为方便我个人查阅和理解，详细的分析以及源代码请移步原作者的博客http://chjavach.iteye.com/ import java.util.ArrayList; import java.util.Collection; import java.util.List; /** * GOF 在《设计模式》一书中阐述命令模式的意图：“将一个请求封装
matlab下GPU编程笔记 cherishLC matlab
不多说，直接上代码 gpuDevice % 查看系统中的gpu,,其中的DeviceSupported会给出matlab支持的GPU个数。 g=gpuDevice(1); %会清空 GPU 1中的所有数据,,将GPU1 设为当前GPU reset(g) %也可以清空GPU中数据。 a=1; a=gpuArray(a); %将a从CPU移到GPU中 onGP
SVN安装过程 crabdave SVN
SVN安装过程 subversion-1.6.12 ./configure --prefix=/usr/local/subversion --with-apxs=/usr/local/apache2/bin/apxs --with-apr=/usr/local/apr --with-apr-util=/usr/local/apr --with-openssl=/
sql　行列转换 daizj sql 行列转换行转列列转行
行转列的思想是通过case when 来实现列转行的思想是通过union all 来实现下面具体例子：假设有张学生成绩表(tb)如下: Name Subject Result 张三语文　　74 张三数学　　83 张三物理　　93 李四语文　　74 李四数学　　84 李四物理　　94 */ /* 想变成姓名 &
MySQL--主从配置 dcj3sjt126com mysql
linux下的mysql主从配置：说明：由于MySQL不同版本之间的(二进制日志)binlog格式可能会不一样，因此最好的搭配组合是Master的MySQL版本和Slave的版本相同或者更低， Master的版本肯定不能高于Slave版本。（版本向下兼容） mysql1 : 192.168.100.1 //master mysq
关于yii 数据库添加新字段之后model类的修改 dcj3sjt126com Model
rules: array('新字段','safe','on'=>'search') 1、array('新字段', 'safe')//这个如果是要用户输入的话，要加一下， 2、array('新字段', 'numerical'),//如果是数字的话 3、array('新字段', 'length', 'max'=>100),//如果是文本 1、2、3适当的最少要加一条，新字段才会被
sublime text3 中文乱码解决 dyy_gusi Sublime Text
sublime text3中文乱码解决原因：缺少转换为UTF-8的插件目的：安装ConvertToUTF8插件包第一步：安装能自动安装插件的插件，百度“Codecs33”，然后按照步骤可以得到以下一段代码： import urllib.request,os,hashlib; h = 'eb2297e1a458f27d836c04bb0cbaf282' + 'd0e7a30980927
概念了解：CGI，FastCGI，PHP-CGI与PHP-FPM geeksun PHP
CGI CGI全称是“公共网关接口”(Common Gateway Interface)，HTTP服务器与你的或其它机器上的程序进行“交谈”的一种工具，其程序须运行在网络服务器上。 CGI可以用任何一种语言编写，只要这种语言具有标准输入、输出和环境变量。如php,perl,tcl等。 FastCGI FastCGI像是一个常驻(long-live)型的CGI，它可以一直执行着，只要激活后，不
Git push 报错 "error: failed to push some refs to " 解决 hongtoushizi git
Git push 报错 "error: failed to push some refs to " . 此问题出现的原因是：由于远程仓库中代码版本与本地不一致冲突导致的。由于我在第一次git pull --rebase 代码后，准备push的时候，有别人往线上又提交了代码。所以出现此问题。解决方案： 1： git pull 2：
第四章 Lua模块开发 jinnianshilongnian nginx lua
在实际开发中，不可能把所有代码写到一个大而全的lua文件中，需要进行分模块开发；而且模块化是高性能Lua应用的关键。使用require第一次导入模块后，所有Nginx 进程全局共享模块的数据和代码，每个Worker进程需要时会得到此模块的一个副本（Copy-On-Write），即模块可以认为是每Worker进程共享而不是每Nginx Server共享；另外注意之前我们使用init_by_lua中初
java.lang.reflect.Proxy liyonghui160com
1.简介 Proxy 提供用于创建动态代理类和实例的静态方法（1）动态代理类的属性代理类是公共的、最终的，而不是抽象的未指定代理类的非限定名称。但是，以字符串 "$Proxy" 开头的类名空间应该为代理类保留代理类扩展 java.lang.reflect.Proxy 代理类会按同一顺序准确地实现其创建时指定的接口
Java中getResourceAsStream的用法 pda158 java
1.Java中的getResourceAsStream有以下几种： 1. Class.getResourceAsStream(String path) ： path 不以’/'开头时默认是从此类所在的包下取资源，以’/'开头则是从ClassPath根下获取。其只是通过path构造一个绝对路径，最终还是由ClassLoader获取资源。　　2. Class.getClassLoader.get
spring 包官方下载地址（非maven） sinnk spring
SPRING官方网站改版后，建议都是通过 Maven和Gradle下载，对不使用Maven和Gradle开发项目的，下载就非常麻烦，下给出Spring Framework jar官方直接下载路径: http://repo.springsource.org/libs-release-local/org/springframework/spring/ s
Oracle学习笔记(7) 开发PLSQL子程序和包 vipbooks oracle sql 编程
哈哈，清明节放假回去了一下，真是太好了，回家的感觉真好啊！现在又开始出差之旅了，又好久没有来了，今天继续Oracle的学习！这是第七章的学习笔记，学习完第六章的动态SQL之后，开始要学习子程序和包的使用了……，希望大家能多给俺一些支持啊！编程时使用的工具是PLSQL

paddlepaddle百度飞桨七日打卡营——强化学习parl

你可能感兴趣的:(Python入门AI爬虫,百度七日打卡营)