Leap Motion hand tracking controller

V.1.1

Difficulty: medium

Time: ~30 min

Note

This tutorial is working from:

The version v3.0.0 of the ned_ros_stack

The version v3.0.0 of Niryo Studio

The version v3.1.0 of PyNiryo

You can find the project in our GitHub.

Introduction

This documentation contains all the information on how to set up and use the Leap Motion and control Ned/Ned2 with it. A Leap Motion is an optical hand tracking module that captures the movements of your hands with unparalleled accuracy. With this application, you will be able to control Ned/Ned2 with your hand with a Leap Motion hand tracker.

Requirements

To run this application you will need a Leap Motion controller from Ultraleap and a Ned/Ned2 (real or simulated).

You can buy and find all the information about the Leap Motion following this link: Leap Motion.

How to set up the Leap Motion

Note

For this application you don’t have to set up the Leap Motion because it’s already done in the package. You can still have a look at it if you want to know how to use the Leap Motion.

Before using the Leap Motion with Ned/Ned2, you will need to set up it.

First you will need to download the Leap Motion SDK on the repository you want. You can download it here.

Then you will have to install the package you just downloaded. To do that you will need to open a terminal and go on the repository where you downloaded the previous SDK.

cd <your leap motion repository>
sudo dpkg -i Leap-2.3.1+31549-x64.deb

Run the Leap Motion

Now you can plug the Leap Motion to your computer and run it:

sudo leapd

If your Leap Motion is working and properly connected you should have this:

If you want to, you can launch the Leap Motion Visualizer on a terminal to have a first look on how it is working:

cd <your leap motion repository>
Visualizer

You should see a 3D simulated environment and if you put your hands up to the Leap Motion you can see them.

Now that your Leap Motion is well working you can use it with Ned/Ned2.

How to launch the application

Note

Before running the application you will need to launch the Leap Motion (see below how to do it).

Link the Leap library to your code

Before doing that, you need to link the Leap library to your code. To do so, open the robot_V3.py script. You will need to change the path of the Leap library on the second line:

import os, sys, inspect, thread, time
sys.path.insert(0, "YOUR_PATH/LeapDeveloperKit_2.3.1+31549_linux/LeapSDK/lib")
import Leap

Run the robot

To launch this application you will need Ned/Ned2 (real or simulated).

Note

We advise you to first run this application in simulation to get used to controlling the robot with the Leap Motion.

According to the choice you make you will need to change the IP address of the robot on the code robot_V3.py.

robot_ip = "127.0.0.1" #"192.168.1.15" => ip of the robot

Danger

Before running the application, make sure the environment around the real robot is clear.

To launch the application you need to run the code/robot_V3.py script:

cd <your leap motion repository/code>
python leap_motion_ned.py

After launching the application, you should see this menu:

If you open your hand, the gripper should open and if you close your hand it should close. If you move your hand, the robot will do the same movement.

Note

We advise you to move your hand slowly not to have unreachable or sudden movement.

Tips

To have a fluid movement, we advise you to use an Ethernet connexion with Ned/Ned2.