Newer
Older
[See illustration video here.](https://youtu.be/RAdbk1a0JFY)
This package contains:
- Tools for running artificial skin simulation on `iCub` humanoid robot in `Gazebo 11`
- `OpenAI Gym` environment for communication with `iCub` `Gazebo` simulation and `YARP` ports
- Implementation of several learning algorithms using active learning and intrinsic motivation
This package was developed and tested in `Ubuntu 20.04.2 LTS`
## YARP
https://www.yarp.it/git-master/install_yarp_linux.html
```
sudo sh -c 'echo "deb http://www.icub.org/ubuntu focal contrib/science" > /etc/apt/sources.list.d/icub.list'
sudo sh -c 'echo "deb http://www.icub.org/debian buster contrib/science" > /etc/apt/sources.list.d/icub.list'
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 57A5ACB6110576A6
sudo apt-get update
sudo apt-get install yarp
http://wiki.ros.org/noetic/Installation/Ubuntu
```
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
curl -sSL 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xC1CF6E31E6BADE8868B172B4F42ED6FBAB17C654' | sudo apt-key add -
sudo apt update
sudo apt install ros-noetic-desktop-full
echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
sudo apt install python3-rosdep python3-rosinstall python3-rosinstall-generator python3-wstool build-essential
sudo rosdep init
rosdep update
https://icub-tech-iit.github.io/documentation/sw_installation/linux_binaries/
sudo sh -c 'echo "deb http://www.icub.org/ubuntu `lsb_release -cs` contrib/science" > /etc/apt/sources.list.d/icub.list'
sudo apt update
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 57A5ACB6110576A6
sudo apt install icub
```
You can check the installation by running command `iCub_SIM`
## gazebo-yarp-plugins
http://robotology.github.io/gazebo-yarp-plugins/master/install.html
```
git clone https://github.com/robotology/gazebo-yarp-plugins.git
cd gazebo-yarp-plugins
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE="Release ..
make
```
## icub-models
https://github.com/robotology/icub-models
```
git clone https://github.com/robotology/icub-models.git
cd icub-models
mkdir build
cd build
cmake ..
cmake --build . --target install
```
## YARP Python Bindings
https://www.yarp.it/git-master/yarp_swig.html
1. Create build directory: `mkdir ~/dip/yarp-python`
2. Run `sudo cmake-gui`
3. For the source code directory, select `/usr/share/yarp/bindings`
4. For the build directory, select `/home/maxymczech/dip/yarp-python`
5. Run `Configure`, check box `CREATE_PYTHON`, run `Configure` again and then `Generate`
6. Go to build directory and run `sudo make`
7. Add directory with generated python library to environment variable `PYTHONPATH`
8. Change owner to the build directory to your user: `sudo chown -R maxymczech:maxymczech ~/dip/yarp-python`
9. Test by running `python` interpreter and then:
```
import yarp
```
## Environment Variables
You need to set some environment variables for the system to work properly. Append the following to your `.bashrc` file (modify the paths according to your local settings):
```
source /opt/ros/noetic/setup.bash
export GAZEBO_PLUGIN_PATH=${GAZEBO_PLUGIN_PATH}:/home/maxymczech/dip/gazebo-yarp-plugins/build/lib:/home/maxymczech/dip/code-icub-gazebo-skin/gazebo_contactsensor_plugin/build
export YARP_DATA_DIRS=${YARP_DATA_DIRS}:/home/maxymczech/dip/icub-models/build/share/iCub
export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:/home/maxymczech/dip/icub-models/build/share
export AMENT_PREFIX_PATH=${AMENT_PREFIX_PATH}:/home/maxymczech/dip/icub-models/build/share
export GAZEBO_MODEL_PATH=${GAZEBO_MODEL_PATH}:/home/maxymczech/dip/icub-models/build/share/iCub/robots:/home/maxymczech/dip/icub-models/build/share
export YARP_DATA_DIRS=${YARP_DATA_DIRS}:/usr/share/yarp
export PYTHONPATH=$PYTHONPATH:/home/maxymczech/dip/yarp-python/lib/python3
## OpenAI Gym & `gym-icub-skin` Environment
https://gym.openai.com/docs/#installation
```
pip3 install gym
cd ~/dip/code-icub-gazebo-skin/gym-icub-skin
pip3 install -e .
```
## Explauto
For information about the Explauto library, please refer to the repository for my [Bachelor's thesis](https://gitlab.fel.cvut.cz/body-schema/code-nao-simulation/-/tree/master/gazebo9).
## Skin generation
This package already contains `iCub` model in `URDF` format with artificial skin added on top of model `iCubGazeboV2_5_visuomanip`. If you need to modify it, you can use the scripts in [skin-generation](./skin-generation/) folder:
```
cd skin-generation
node make-skin.js > output_model.urdf
```
The script reads configuration parameters from [config.json](./skin-generation/config.json).
## Launch and Manual Control
You can launch the simulation using `yarpmanager`:
```
cd ~/dip/code-icub-gazebo-skin/yarp_scripts
yarpserver && yarpmanager
```
If for some reason you do not want to use `yarpmanager`, you can perform the next
steps manually. To launch the simulation, you will need several terminal windows:
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
1. Start `YARP` server:
```
yarpserver
```
2. Either launch `Gazebo` and add the model `iCubGazeboV2_5_skin` manually:
```
gazebo
```
Or use `Gazebo` world file `icub_skin.world`:
```
gazebo icub_skin.world
```
3. Launch `iCubSkinGui` for corresponding body part(s):
```
iCubSkinGui --from left_arm.ini --useCalibration
iCubSkinGui --from left_forearm.ini --useCalibration
iCubSkinGui --from left_hand.ini --useCalibration
iCubSkinGui --from right_arm.ini --useCalibration
iCubSkinGui --from right_forearm.ini --useCalibration
iCubSkinGui --from right_hand.ini --useCalibration
iCubSkinGui --from torso.ini --useCalibration
```
4. Connect `icubSim` output ports to corresponding `skinGui` input ports:
```
yarp connect /icubSim/skin/left_arm_comp /skinGui/left_arm:i
yarp connect /icubSim/skin/left_forearm_comp /skinGui/left_forearm:i
yarp connect /icubSim/skin/left_hand_comp /skinGui/left_hand:i
yarp connect /icubSim/skin/right_arm_comp /skinGui/right_arm:i
yarp connect /icubSim/skin/right_forearm_comp /skinGui/right_forearm:i
yarp connect /icubSim/skin/right_hand_comp /skinGui/right_hand:i
yarp connect /icubSim/skin/torso_comp /skinGui/torso:i
```
5. To control the robot manually, you can use `yarpmotorgui`:
```
yarpmotorgui --robot icubSim --parts "(head torso left_arm right_arm left_leg right_leg)"
An example home pose with left hand index finger touching torso skin can be loaded from file [home_pose_index.posleft_arm](misc/home_pose_index.posleft_arm) or [home_pose_thumb.posleft_arm](misc/home_pose_thumb.posleft_arm).
6. To display skin taxel activations in terminal, run:
```
yarp read /reader /icubSim/skin/left_arm_comp
yarp read /reader /icubSim/skin/left_forearm_comp
yarp read /reader /icubSim/skin/left_hand_comp
yarp read /reader /icubSim/skin/right_arm_comp
yarp read /reader /icubSim/skin/right_forearm_comp
yarp read /reader /icubSim/skin/right_hand_comp
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
# Miscellaneous notes
## Skin tester script
Python script `misc/skin-tester.py` can be used to visually highlight taxels
on the artificial skin patches and create a mapping from taxel index to skin
coordinates. The script has three parameters:
- `output_port_name`: name of `YARP` output port, e.g. `"/icubSim/skin/right_forearm_comp"`
- `input_port_name`: name of `YARP` input port, e.g. `"/skinGui/right_forearm:i"`
- `n_taxels`: number of taxels on the skin patch being tested
This is how the script should be used:
1. Start `yarpserver`
2. Start skin GUI for corresponding skin part:
`iCubSkinGui --from right_forearm.ini --useCalibration`
3. Run the script: `python skin-tester.py`
The script will connect input and output YARP ports that were set as script
parameters and then will go over each taxel index in range `(0, n_taxels - 1)`,
highlighting it for around `0.3` seconds.
## Explanation of Gazebo contact sensor plugin
In our previous work with `Nao` humanoid robot, we have used Gazebo
[sensor plugin](http://gazebosim.org/tutorials?tut=contact_sensor&cat=sensors)
to register contacts on each body part and send contact data to corresponding
`ROS` topic. This approach caused the simulation to slow down significantly
when touch events were fired.
In this work, I have changed this approach slightly: instead of several
contact sensor plugins, I am using a single __model plugin__ that subscribes
directly to Gazebo topic `~/physics/contacts`, filters relevant touches and
forwards touch information to `YARP` ports. The plugin code is in
`gazebo_contactsensor_plugin` folder, in `ContactPlugin.cc` and `ContactPlugin.hh`
files:
- Touches are registered on 7 body parts: `left_arm`, `left_forearm`, `left_hand`,
`right_arm`, `right_forearm`, `right_hand`, `torso`
- In `Load` method, the plugin opens output `YARP` ports, resets taxel values,
subscribes to Gazebo `~/physics/contacts` topic and forces contact visualization
in Gazebo
- In `OnContact` method, the plugin computes touch forces and locations
- In `OnUpdate` method, the plugin sends recorded touch data to `YARP` ports
and resets taxels to their default values
## More information and pilot experiments
Shcherban, M. (2021), 'Efficient Exploration of Body Surface with Tactile Sensors on Humanoid Robots', Master's thesis, Faculty of Electrical Engineering, Czech Technical University in Prague. [[link to thesis page]](https://dspace.cvut.cz/handle/10467/97041) [[pdf]](https://dspace.cvut.cz/bitstream/handle/10467/82501/F3-BP-2019-Shcherban-Maksym-BAK-shchema1-final.pdf)