Skip to content
Snippets Groups Projects
README.md 9.48 KiB
Newer Older
[See illustration video here.](https://youtu.be/RAdbk1a0JFY)


Maksym Shcherban's avatar
Maksym Shcherban committed
This package contains:

- Tools for running artificial skin simulation on `iCub` humanoid robot in `Gazebo 11`
- `OpenAI Gym` environment for communication with `iCub` `Gazebo` simulation and `YARP` ports
- Implementation of several learning algorithms using active learning and intrinsic motivation

Maksym Shcherban's avatar
Maksym Shcherban committed
# Installation
Matěj Hoffmann's avatar
Matěj Hoffmann committed

Maksym Shcherban's avatar
Maksym Shcherban committed
This package was developed and tested in `Ubuntu 20.04.2 LTS`

## YARP

https://www.yarp.it/git-master/install_yarp_linux.html

```
sudo sh -c 'echo "deb http://www.icub.org/ubuntu focal contrib/science" > /etc/apt/sources.list.d/icub.list'
sudo sh -c 'echo "deb http://www.icub.org/debian buster contrib/science" > /etc/apt/sources.list.d/icub.list'
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 57A5ACB6110576A6
sudo apt-get update
sudo apt-get install yarp
Maksym Shcherban's avatar
Maksym Shcherban committed
## ROS noetic
Maksym Shcherban's avatar
Maksym Shcherban committed
http://wiki.ros.org/noetic/Installation/Ubuntu

```
sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt-key adv --keyserver 'hkp://keyserver.ubuntu.com:80' --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654
curl -sSL 'http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xC1CF6E31E6BADE8868B172B4F42ED6FBAB17C654' | sudo apt-key add -
sudo apt update
sudo apt install ros-noetic-desktop-full
echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
sudo apt install python3-rosdep python3-rosinstall python3-rosinstall-generator python3-wstool build-essential
sudo rosdep init
rosdep update
Maksym Shcherban's avatar
Maksym Shcherban committed
## iCub Software
Maksym Shcherban's avatar
Maksym Shcherban committed
https://icub-tech-iit.github.io/documentation/sw_installation/linux_binaries/
Maksym Shcherban's avatar
Maksym Shcherban committed
sudo sh -c 'echo "deb http://www.icub.org/ubuntu `lsb_release -cs` contrib/science" > /etc/apt/sources.list.d/icub.list'
sudo apt update
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 57A5ACB6110576A6
sudo apt install icub
```

You can check the installation by running command `iCub_SIM`

## gazebo-yarp-plugins
Maksym Shcherban's avatar
Maksym Shcherban committed
http://robotology.github.io/gazebo-yarp-plugins/master/install.html

```
git clone https://github.com/robotology/gazebo-yarp-plugins.git
cd gazebo-yarp-plugins
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE="Release ..
make
```

## icub-models

https://github.com/robotology/icub-models

```
git clone https://github.com/robotology/icub-models.git
cd icub-models
mkdir build
cd build
cmake ..
cmake --build . --target install
```

## YARP Python Bindings

https://www.yarp.it/git-master/yarp_swig.html

1. Create build directory: `mkdir ~/dip/yarp-python`

2. Run `sudo cmake-gui`

3. For the source code directory, select `/usr/share/yarp/bindings`

4. For the build directory, select `/home/maxymczech/dip/yarp-python`

5. Run `Configure`, check box `CREATE_PYTHON`, run `Configure` again and then `Generate`

6. Go to build directory and run `sudo make`

7. Add directory with generated python library to environment variable `PYTHONPATH`

8. Change owner to the build directory to your user: `sudo chown -R maxymczech:maxymczech ~/dip/yarp-python`

9. Test by running `python` interpreter and then:
```
import yarp
```

Maksym Shcherban's avatar
Maksym Shcherban committed
## Environment Variables

You need to set some environment variables for the system to work properly. Append the following to your `.bashrc` file (modify the paths according to your local settings):

```
source /opt/ros/noetic/setup.bash
export GAZEBO_PLUGIN_PATH=${GAZEBO_PLUGIN_PATH}:/home/maxymczech/dip/gazebo-yarp-plugins/build/lib:/home/maxymczech/dip/code-icub-gazebo-skin/gazebo_contactsensor_plugin/build
export YARP_DATA_DIRS=${YARP_DATA_DIRS}:/home/maxymczech/dip/icub-models/build/share/iCub
export ROS_PACKAGE_PATH=${ROS_PACKAGE_PATH}:/home/maxymczech/dip/icub-models/build/share
export AMENT_PREFIX_PATH=${AMENT_PREFIX_PATH}:/home/maxymczech/dip/icub-models/build/share
export GAZEBO_MODEL_PATH=${GAZEBO_MODEL_PATH}:/home/maxymczech/dip/icub-models/build/share/iCub/robots:/home/maxymczech/dip/icub-models/build/share
export YARP_DATA_DIRS=${YARP_DATA_DIRS}:/usr/share/yarp
export PYTHONPATH=$PYTHONPATH:/home/maxymczech/dip/yarp-python/lib/python3
## OpenAI Gym & `gym-icub-skin` Environment

https://gym.openai.com/docs/#installation

```
pip3 install gym
cd ~/dip/code-icub-gazebo-skin/gym-icub-skin
pip3 install -e .
```

## Explauto

Maksym Shcherban's avatar
Maksym Shcherban committed
For information about the Explauto library, please refer to the repository for my [Bachelor's thesis](https://gitlab.fel.cvut.cz/body-schema/code-nao-simulation/-/tree/master/gazebo9).
Maksym Shcherban's avatar
Maksym Shcherban committed
# Usage

Maksym Shcherban's avatar
Maksym Shcherban committed
## Skin generation

This package already contains `iCub` model in `URDF` format with artificial skin added on top of model `iCubGazeboV2_5_visuomanip`. If you need to modify it, you can use the scripts in [skin-generation](./skin-generation/) folder:

```
cd skin-generation
node make-skin.js > output_model.urdf
```

The script reads configuration parameters from [config.json](./skin-generation/config.json).

## Launch and Manual Control

You can launch the simulation using `yarpmanager`:

```
cd ~/dip/code-icub-gazebo-skin/yarp_scripts
yarpserver && yarpmanager
```

If for some reason you do not want to use `yarpmanager`, you can perform the next
steps manually. To launch the simulation, you will need several terminal windows:
Maksym Shcherban's avatar
Maksym Shcherban committed

1. Start `YARP` server:
```
yarpserver
```

2. Either launch `Gazebo` and add the model `iCubGazeboV2_5_skin` manually:
```
gazebo
```
Or use `Gazebo` world file `icub_skin.world`:
```
gazebo icub_skin.world
```

3. Launch `iCubSkinGui` for corresponding body part(s):
```
iCubSkinGui --from left_arm.ini --useCalibration
iCubSkinGui --from left_forearm.ini --useCalibration
iCubSkinGui --from left_hand.ini --useCalibration
iCubSkinGui --from right_arm.ini --useCalibration
iCubSkinGui --from right_forearm.ini --useCalibration
iCubSkinGui --from right_hand.ini --useCalibration
iCubSkinGui --from torso.ini --useCalibration
```

4. Connect `icubSim` output ports to corresponding `skinGui` input ports:
```
yarp connect /icubSim/skin/left_arm_comp /skinGui/left_arm:i
yarp connect /icubSim/skin/left_forearm_comp /skinGui/left_forearm:i
yarp connect /icubSim/skin/left_hand_comp /skinGui/left_hand:i
yarp connect /icubSim/skin/right_arm_comp /skinGui/right_arm:i
yarp connect /icubSim/skin/right_forearm_comp /skinGui/right_forearm:i
yarp connect /icubSim/skin/right_hand_comp /skinGui/right_hand:i
yarp connect /icubSim/skin/torso_comp /skinGui/torso:i
```

5. To control the robot manually, you can use `yarpmotorgui`:
```
yarpmotorgui --robot icubSim --parts "(head torso left_arm right_arm left_leg right_leg)"
Maksym Shcherban's avatar
Maksym Shcherban committed
```
An example home pose with left hand index finger touching torso skin can be loaded from file [home_pose_index.posleft_arm](misc/home_pose_index.posleft_arm) or [home_pose_thumb.posleft_arm](misc/home_pose_thumb.posleft_arm).
Maksym Shcherban's avatar
Maksym Shcherban committed

6. To display skin taxel activations in  terminal, run:
```
yarp read /reader /icubSim/skin/left_arm_comp
yarp read /reader /icubSim/skin/left_forearm_comp
yarp read /reader /icubSim/skin/left_hand_comp
yarp read /reader /icubSim/skin/right_arm_comp
yarp read /reader /icubSim/skin/right_forearm_comp
yarp read /reader /icubSim/skin/right_hand_comp
Maksym Shcherban's avatar
Maksym Shcherban committed
yarp read /reader /icubSim/skin/torso_comp
```

# Miscellaneous notes

## Skin tester script

Python script `misc/skin-tester.py` can be used to visually highlight taxels
on the artificial skin patches and create a mapping from taxel index to skin
coordinates. The script has three parameters:

- `output_port_name`: name of `YARP` output port, e.g. `"/icubSim/skin/right_forearm_comp"`
- `input_port_name`: name of `YARP` input port, e.g. `"/skinGui/right_forearm:i"`
- `n_taxels`: number of taxels on the skin patch being tested

This is how the script should be used:

1. Start `yarpserver`
2. Start skin GUI for corresponding skin part:
`iCubSkinGui --from right_forearm.ini --useCalibration`
3. Run the script: `python skin-tester.py`

The script will connect input and output YARP ports that were set as script
parameters and then will go over each taxel index in range `(0, n_taxels - 1)`,
highlighting it for around `0.3` seconds.

## Explanation of Gazebo contact sensor plugin

Maksym Shcherban's avatar
Maksym Shcherban committed
In our previous work with `Nao` humanoid robot, we have used Gazebo
[sensor plugin](http://gazebosim.org/tutorials?tut=contact_sensor&cat=sensors)
to register contacts on each body part and send contact data to corresponding
`ROS` topic. This approach caused the simulation to slow down significantly
when touch events were fired.

In this work, I have changed this approach slightly: instead of several
contact sensor plugins, I am using a single __model plugin__ that subscribes
directly to Gazebo topic `~/physics/contacts`, filters relevant touches and
forwards touch information to `YARP` ports. The plugin code is in
`gazebo_contactsensor_plugin` folder, in `ContactPlugin.cc` and `ContactPlugin.hh`
files:

- Touches are registered on 7 body parts: `left_arm`, `left_forearm`, `left_hand`,
`right_arm`, `right_forearm`, `right_hand`, `torso`
- In `Load` method, the plugin opens output `YARP` ports, resets taxel values,
subscribes to Gazebo `~/physics/contacts` topic and forces contact visualization
in Gazebo
- In `OnContact` method, the plugin computes touch forces and locations
- In `OnUpdate` method, the plugin sends recorded touch data to `YARP` ports
and resets taxels to their default values
Matěj Hoffmann's avatar
Matěj Hoffmann committed

## More information and pilot experiments
Shcherban, M. (2021), 'Efficient Exploration of Body Surface with Tactile Sensors on Humanoid Robots', Master's thesis, Faculty of Electrical Engineering, Czech Technical University in Prague. [[link to thesis page]](https://dspace.cvut.cz/handle/10467/97041)  [[pdf]](https://dspace.cvut.cz/bitstream/handle/10467/82501/F3-BP-2019-Shcherban-Maksym-BAK-shchema1-final.pdf)