author : Samuel Berrien
The final goal of this project is to implement any kind of optimisation algorithm in order to teach creature to walk, run etc.
This project requires multiple libraries.
- physic engine stuff : Bullet
- graphic stuff : OpenGL, GLM, GLEW, GLFW
- reinforcement learning : LibTorch CXX 11 ABI (not pre-cxx 11 ABI) according to your CUDA version. Put
libtorch
folder in/opt/
, (see cmake files) - utils stuff : p-ranav argparse, p-ranav indicators, nlohmann json
Install dependencies with pacman
:
$ # as root
$ pacman -Syu cmake bullet glm glew glfw
Download libtorch
from torch official website and then copy the extracted folder in /opt/
(here version 2.3.0 with
CUDA 12.1) :
$ # as root
$ unzip /path/to/libtorch-cxx11-abi-shared-with-deps-2.3.0+cu121.zip -d /opt/
Any attempt to build this project on other OS will be appreciated to complete this section.
To build this project you need a decent C++ compiler with cmake and make. An internet connexion is also required in order to download dependencies inside cmake file.
-
Clone the project
$ git clone https://github.com/Ipsedo/EvoMotion.git
-
Then build the project:
$ cd /path/to/EvoMotion $ mkdir build && cd build && cmake .. && make
-
Ready for training ! You can choose the environment on which train the agent you want.
Run training on 3D cartpole
$ cd /path/to/EvoMotion/build $ evo_motion cartpole3d actor_critic --seed 1234 --cuda --hidden_size 32 train ./out/cartpole3d_a2c --episodes 1024 --nb_saves 1024 --learning_rate 1e-3
Run training on creature muscles
$ cd /path/to/EvoMotion/build $ evo_motion muscles actor_critic_liquid --seed 1234 --cuda --hidden_size 32 train ./out/muscles_a2c_liquid --episodes 1024 --nb_saves 1024 --learning_rate 1e-3
-
After the first save (here after 1024 episodes), you can now evaluate your trained agent.
Evaluate agent on 3D cartpole (here the first model save) with GLFW window of 1920 * 1024 pixels
$ evo_motion cartpole3d actor_critic --seed 1234 --hidden_size 32 --cuda run ./out/cartpole3d_a2c/save_0 -w 1920 -h 1024
Evaluate agent on creature muscles (here the first model save) with GLFW window of 1920 * 1024 pixels
$ evo_motion muscles actor_critic_liquid --seed 1234 --hidden_size 32 --cuda run ./out/muscles_a2c_liquid/save_0 -w 1920 -h 1024
[1] Reinforcement Learning: An Introduction - Richard S. Sutton and Andrew G. Barto - Second Edition 2018
[2] Liquid Time-constant Networks - Ramin Hasani, Mathias Lechner, Alexander Amini, Daniela Rus, Radu Grosu - 8 Jun 2020