gym_unity学习笔记

最近学了一段时间gym_unity，把一些资料留在这里

实例

1. 实例gym_unity训练RollerBall：https://blog.csdn.net/alibutter/article/details/120908687
2. 实例gyn_unity训练3DBall：https://zhuanlan.zhihu.com/p/554927641?utm_id=0
   源码：https://github.com/Delta-King/UnityPPO
   实现时下载release-19稳定版本：https://github.com/Unity-Technologies/ml-agents/releases/tag/release_19

官方资料

1. ml-agents官网：https://github.com/Unity-Technologies/ml-agents/tree/develop
2. ml-agents中文文档：https://github.com/Hustacds/ml-agents/tree/master/docs/localized/zh-CN
3. mlagents_env对外gym接口文档：https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Python-Gym-API.md
4. mlagents_env多智能体接口文档：https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Python-PettingZoo-API.md
5. mlagents_env python接口文档：https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Python-LLAPI.md
6. gym_unity的官方解释：https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Python-Gym-API-Documentation.md
7. 多智能体接口：https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Python-PettingZoo-API.md
8. ML-agents训练时的configutation文件说明(yaml文件)：https://github.com/Unity-Technologies/ml-agents/blob/develop/docs/Training-Configuration-File.md

过程报错：

报错：\ml-agents-develop\com.unity.ml-agents\Runtime\Sensors\RayPerceptionSensor.cs(459,39): error CS0103: The name ‘QueryParameters’ does not exist in the current context
原因：ML-agents的版本过高，对Unity的版本有要求。换release-19即可
参考：https://github.com/Unity-Technologies/ml-agents/issues/5958

报错：TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

1. Downgrade the protobuf package to 3.20.x or lower.
2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure

Python parsing and will be much slower).
解决办法(把protobuf的版本从4.多降下来)：pip install protobuf==3.20.*

报错：TypeError: can’t convert cuda:0 device type tensor to numpy. Use Tensor.cpu() to copy the tensor to
解决方案：参考：https://blog.csdn.net/weixin_45887062/article/details/126417024

报错：ImportError: cannot import name ‘cygrpc’ from ‘grpc._cython’ (D:\A\envs\pythonProject\lib\site-packa
解决：pip install grpcio==1.27.2
参考：https://github.com/GoogleCloudPlatform/python-docs-samples/issues/1540

ｇｙｍ＿ｕｎｉｔｙ模板：

from mlagents_envs.environment import UnityEnvironment
from gym_unity.envs import UnityToGymWrapper

    env_directory = 'D:\\Desktop\\3DBall\\UnityEnvironment.exe'
    unity_env = UnityEnvironment(env_directory, base_port=5005, no_graphics=False)
env = UnityToGymWrapper(unity_env, uint8_visual=True)
执行env.step(action)和env.reset()就可以

UnityGymWrapper：

UnityGymWrapper是Unity和gym的封装接口，封装Unity环境给当作gym使用。UnityGymWrapper只适用于单智能体，UnityGymWrapper5可以用于多智能体
Multi-agent使用UnityGymWrapper5例子：
源码：https://github.com/leehe228/LogisticsEnv
论文：Multi agent reinforcement learning based UAV control for Urban Aerial Mobility logistics
UnityGymWrapper使用模板：

参考：https://lab.uwa4d.com/lab/624a2749a8103dabd0e58f10

疑问：

自定义的unity环境env的step和reset()在哪写？？？
gym_unity训练完的模型怎么保存？？？？
.pt 文件通常是指 PyTorch 的模型文件，它是 PyTorch 框架中用于保存和加载模型权重和结构的一种格式。
参考：https://blog.csdn.net/weixin_44943389/article/details/131940271在这里插入图片描述