為了嘗試一下 cs294 課程的作業(yè)，配環(huán)境配了一天多。
Deep RL Assignment 1: Imitation Learning Fall 2017 作業(yè)要求上一句輕描淡寫(xiě)：
- MuJoCo: We will use MuJoCo for physics simulation in this assignment. Download version 1.31 from mujoco.org and obtain a license from the course instructors. Note that we use version 1.31 and not the latest version 1.50. The python interface to mujoco bundled with OpenAI gym is also compatible with this version of mujoco.
前面一直出 bug 的是 gym：
Traceback (most recent call last):
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 143, in spec
return self.env_specs[id]
KeyError: 'Humanoid-v1'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run_expert.py", line 71, in <module>
main()
File "run_expert.py", line 37, in main
env = gym.make(args.envname)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 167, in make
return registry.make(id)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 118, in make
spec = self.spec(id)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 151, in spec
raise error.DeprecatedEnv('Env {} not found (valid versions include {})'.format(id, matching_envs))
gym.error.DeprecatedEnv: Env Humanoid-v1 not found (valid versions include ['Humanoid-v2'])
感謝最后一句的提醒，估計(jì)是什么更新了吧，v1沒(méi)有了，我換成了 v2，這一句終于能過(guò)掉了。由于還沒(méi)有裝 mujoco，錯(cuò)誤變成了：
(cs294hw1) C:\ProjectsData\machineLearning\ACMCourse2\cs294\homework\hw1>python run_expert.py experts\Humanoid-
v1.pkl Humanoid-v2 --render --num_rollouts 20
loading and building expert policy
obs (1, 376) (1, 376)
loaded and built
……（此處略去幾十行）
Traceback (most recent call last):
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\mujoco\mujoco_env.py", line 11, in <module>
import mujoco_py
ModuleNotFoundError: No module named 'mujoco_py'
安裝 mujoco_py 和 mujoco
我開(kāi)始安裝 mujoco_py。據(jù) 博客 提示，1.3.1 mjpro對(duì)應(yīng)的0.5.7py，我就去 pip3.5 install mujoco_py==0.5.7 ，一切順利。
然后 import mujoco_py時(shí)，發(fā)生錯(cuò)誤：
>>> import mujoco_py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\__init__.py", line 2, in <module>
init_config()
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\config.py", line 33, in init_config
raise error.MujocoDependencyError('To use MuJoCo, you need to either populate ~/.mujoco/mjkey.txt and ~/.mujoco/mjpro131, or set the MUJOCO_PY_MJKEY_PATH and MUJOCO_PY_MJPRO_PATH environment variables appropriately. Follow the instructions on https://github.com/openai/mujoco-py for where to obtain these.')
mujoco_py.error.MujocoDependencyError: To use MuJoCo, you need to either populate ~/.mujoco/mjkey.txt and ~/.mujoco/mjpro131, or set the MUJOCO_PY_MJKEY_PATH and MUJOCO_PY_MJPRO_PATH environment variables appropriately. Follow the instructions on https://github.com/openai/mujoco-py for where to obtain these.
好吧，我確實(shí)還沒(méi)有找 key。我到官網(wǎng)注冊(cè)了MuJoCo Pro Trial License: 30 days，從郵箱下載了 LICENSE.txt 和 mjkey.txt。同時(shí)在官網(wǎng) 下載了mjpro131 win64，解壓到了C:\Program Files\mujoco文件夾。我不僅把兩個(gè)文件瘋狂地復(fù)制到了C:\Program Files\mujoco、C:\Program Files\mujoco\mjpro131 以及 C:\Program Files\mujoco\mjpro131\bin，系統(tǒng)環(huán)境變量也加進(jìn)去了，如下圖，（這里其實(shí)還寫(xiě)錯(cuò)了，KEY的應(yīng)該具體到文件）結(jié)果還是不行。
這都不行，issue里也搜索不到有幫助的信息，那，只能，去改掉報(bào)錯(cuò)的文件了（微笑）。我去看了文件C:\anaconda3\envs\cs294hw1\Lib\site-packages\mujoco_py\config.py，發(fā)現(xiàn)寫(xiě)道 _key_path = os.environ.get('MUJOCO_PY_MJKEY_PATH')
自己嘗試這句話，果然是拿不到的。
（后來(lái)發(fā)現(xiàn)還是我蠢，我沒(méi)有重啟一下命令行，好像重開(kāi)一個(gè)才會(huì)拿到新的環(huán)境變量= =這樣的話應(yīng)該是只要添加了環(huán)境變量就OK了）
C:\Users\shens
λ python
Python 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> _key_path = os.environ.get('MUJOCO_PY_MJKEY_PATH')
>>> print(_key_path)
None
然后下面給了 default 值竟然是按照 Linux 的寫(xiě)法寫(xiě)的，怪不得沒(méi)有這個(gè)路徑……為了杜絕后患，我把這一大段 if 不 if 的都改掉了，直接暴力賦值
_key_path = os.path.expanduser('C:\\Program Files\\mujoco\\mjpro131\\bin\\mjkey.txt')
mjpro_path = os.path.expanduser('C:\\Program Files\\mujoco\\mjpro131')
這下終于過(guò)了，不過(guò)報(bào)錯(cuò)變成了
(cs294hw1) C:\ProjectsData\machineLearning\ACMCourse2\cs294\homework\hw1>python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 12:30:02) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mujoco_py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\__init__.py", line 4, in <module>
from .mjviewer import MjViewer
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\mjviewer.py", line 7, in <module>
from . import mjcore, mjconstants, glfw
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\mjcore.py", line 6, in <module>
from .mjlib import mjlib
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\mujoco_py\mjlib.py", line 21, in <module>
mjlib = cdll.LoadLibrary(os.path.abspath(libfile))
File "C:\anaconda3\envs\cs294hw1\lib\ctypes\__init__.py", line 426, in LoadLibrary
return self._dlltype(name)
File "C:\anaconda3\envs\cs294hw1\lib\ctypes\__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 不是有效的 Win32 應(yīng)用程序。
這次 stackoverflow拯救了我。按照高贊回答說(shuō)的，修改掉一個(gè)文件名，順便直接把系統(tǒng)標(biāo)成 win 后，這位大爺終于不報(bào)錯(cuò)了（您這個(gè)庫(kù)完全是按照 Linux 的文件系統(tǒng)寫(xiě)的，根本沒(méi)有照顧過(guò) win 的情緒好不好）。去跑了 mujoco 的示例模型，能跑出來(lái)了。
收拾 gym
這時(shí)候我去運(yùn)行題目， gym 又出問(wèn)題了，
>>> import gym
>>> env = gym.make('Humanoid-v2')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 167, in make
return registry.make(id)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 119, in make
env = spec.make()
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\registration.py", line 86, in make
env = cls(**self._kwargs)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\mujoco\humanoid.py", line 12, in __init__
mujoco_env.MujocoEnv.__init__(self, 'humanoid.xml', 5)
File "C:\anaconda3\envs\cs294hw1\lib\site-packages\gym\envs\mujoco\mujoco_env.py", line 27, in __init__
self.model = mujoco_py.load_model_from_path(fullpath)
AttributeError: module 'mujoco_py' has no attribute 'load_model_from_path'
還好 github 上有兩個(gè) issue 244, 192，說(shuō)使用python 3.6, mujoco 0.5.7 with the mjpro131 package installed and gym 0.9.1這一整套配置就可以了。我驚喜地發(fā)現(xiàn)我默認(rèn)安裝了gym 0.10.5，我改成pip install gym==0.9.1，終于 work 了……
現(xiàn)在我已經(jīng)沒(méi)有耐心寫(xiě)作業(yè)了，手動(dòng)再見(jiàn)……
下午更新
我想用 jupyter 寫(xiě)作業(yè)，發(fā)現(xiàn)那邊虛擬環(huán)境還要配置。參考了簡(jiǎn)書(shū)文章的后半部分，
# 安裝ipykernel 庫(kù)
conda install -n cs294hw1 ipykernel
# （在虛擬環(huán)境中）注冊(cè)環(huán)境
python -m ipykernel install --user
應(yīng)該是我遺漏了參數(shù)，這個(gè)環(huán)境被命名為 python3 了，我去C:\Users\shens\AppData\Roaming\jupyter\kernels\python3中找 json 文件改了一下。好啦開(kāi)始寫(xiě)作業(yè)。
