HomeLeapmotion with arduinoControl Robotic arm gripper with Leap Motion and arduino using processing for robotic hand

Control Robotic arm gripper with Leap Motion and arduino using processing for robotic hand

Control Robotic Hand Gripper With Leap Motion controller using Arduino and Processing 

How  would you feel watching hand gripper moves with your natural hand gripper gestures? Leap Motion technology made this dream true for you. Controlling robotic hand with natural hand without touch any sensors and actuators and without any delay of time in the field. Its a natural-user interface and having plug and play facilities. Working with accuracy, delay calibration and multi tracking system increasing  popularity of Leap Motion. Platform of non-contact hands tracking systems under real conditions develops real tasks for controlling Robotic hand gripper. We are controlling Robotic hand gripper with the help of arduino and processing software.

Here we learn

1) What is Leap Motion controller gestures detector sensor.
2) Introduction of softwares Arduino + Processing.
3) How to control Leap Motion sensor with Arduino.
4) How to control Robotic hand gripper.
5) How to Code for Leap Motion.

1)  What is Leap Motion Controller gestures detector sensor

3D hand detector sensor change the vision of world. A non-contact sensor tracks Five fingers, Hands, Arm.
This Motion detector sensor detects the position of Hands, Wrist, Palm and also X, Y, Z orientations of hands. Its highly effective accuracy provides many software’s to interface with it. Leap Motion controller consist of two multi-tracking cameras that takes 200 frames per second. Its three Infra Rays leds gives a wide range of 1*1 feet tracking area. The minimum distance for tracking hands is 1 inch. This motion detector sensor developed to overcome limitations of non-contact sensors. X, Y, Z coordinates system are established in it to make stability of gestures reorganization.

 The main purpose of Leap Motion is to give way of controlling real world hardware with 3D technology controls. Until yet people play games with gestures in computers. Now time to change and come up in real world hardware. Motion sensor turns the intentions of projects like if you remember that when the touch screen mobiles are introduced no one is interested in phone with buttons and that established a great change in life style. Think about computers. We used keyboard and mouse with it. If we can able to work without keyboards and mouse that’s another impressive part. This 3D technology gives us way to control computers with gestures. Detecting a human hand is very difficult because orientations, fingers, position of hands. Knowing this device its easier more stable more efficient usable without delays have high accuracy.  

2)  Introduction of software’s Arduino + Processing

Arduino is open source software and hardware which is extremely used in projects. It has simplest programming and programming software called Arduino IDE. Hardware side of arduino is consist of many types of Arduino boards like UNO, NANO, MEGA etc. It has many pins for controlling purpose depending on board type you are using. Its has analog pins, digital pins for inputs and outputs. The popularity of Arduino is because its friendly programming behavior. It has too simplest programming that’s why it is called a sketch. Easy to interface with all type of sensors. It made a great contribution in students projects and incredible boost to knowledge. 

Easy to use with Leap Motion technology even don’t have to add any additional library in it for motion detection integration. further we will discuss the details of coding with motion detector sensor. For more detail about Arduino software and hardware boards Read : getting-started-with-arduino-microcontroller  

Processing is a open source Java based programming language and a flexible software IDE integrated development environment made for controlling electronics, visual arts, image processing, gaming and technology. Like Arduino software the main idea of processing is to give a platform for non programmers to start computer programming without any advance knowledge of programming. It is simplified computer user interface graphics software. Processing software programming is also called sketch as its software gives variety of libraries and classes. You can also make your own classes and it has very basic programming so that students of any age easily manipulate it for graphics, image-processing, Leap Motion controller etc. It encourage and support engineers,programmers to design creative ideas. for more detail Read: Processing software for leap motion
   

3)  How to control Leap Motion Controller with Arduino

We cannot directly controlled Leap Motion with Arduino we must need another software to receive data from motion detector sensor. Arduino converted gestures recognizing data into physical electronics controlling purpose. We used Processing software to recognize and tracks the gestures of hands, fingers, arm, wrist positions in coordinates system and send data to Arduino. Arduino used the data of gestures from processing and process it to move motors. It have builtin libraries to communicate with processing so we do not install any extra library in it. On the other hand processing software needs to add extra library to communicate with gestures recognition technology.

Adding library in Processing softtware.

  • Go to Sketck.
  • Click Import library
  • Add liberary

Search screen open then type ” Leap Motion for Processing ” and add library. This library is completely fully developed for taking inputs from motion detector device. If you check its examples this library have three examples for conform working of  motion detecting device that detects and shows your both hands, fingers, id, hand strength, palm position, hand name either its right hand or left hand and also shows time duration. I am giving you an example just copy and paste it on processing and enjoy the working of gestures recognizing technology. Remember first you have to install library and plugin Leap Motion controller device then its example working.

import de.voidplus.leapmotion.*;

// ======================================================
// Table of Contents:
// ├─ 1. Swipe Gesture
// ├─ 2. Circle Gesture
// ├─ 3. Screen Tap Gesture
// └─ 4. Key Tap Gesture
// ======================================================

LeapMotion leap;

void setup(){
  size(800, 500);
  background(255);
  // …

  leap = new LeapMotion(this).allowGestures();  // All gestures
  // leap = new LeapMotion(this).allowGestures(“circle, swipe, screen_tap, key_tap”);
  // leap = new LeapMotion(this).allowGestures(“swipe”);  // Leap detects only swipe gestures
}

void draw(){
  background(255);
  // …
}

// ======================================================
// 1. Swipe Gesture

void leapOnSwipeGesture(SwipeGesture g, int state){
  int     id               = g.getId();
  Finger  finger           = g.getFinger();
  PVector position         = g.getPosition();
  PVector positionStart    = g.getStartPosition();
  PVector direction        = g.getDirection();
  float   speed            = g.getSpeed();
  long    duration         = g.getDuration();
  float   durationSeconds  = g.getDurationInSeconds();

  switch(state){
    case 1: // Start
      break;
    case 2: // Update
      break;
    case 3: // Stop
      println(“SwipeGesture: ” + id);
      break;
  }
}

// ======================================================
// 2. Circle Gesture

void leapOnCircleGesture(CircleGesture g, int state){
  int     id               = g.getId();
  Finger  finger           = g.getFinger();
  PVector positionCenter   = g.getCenter();
  float   radius           = g.getRadius();
  float   progress         = g.getProgress();
  long    duration         = g.getDuration();
  float   durationSeconds  = g.getDurationInSeconds();
  int     direction        = g.getDirection();

  switch(state){
    case 1: // Start
      break;
    case 2: // Update
      break;
    case 3: // Stop
      println(“CircleGesture: ” + id);
      break;
  }

  switch(direction){
    case 0: // Anticlockwise/Left gesture
      break;
    case 1: // Clockwise/Right gesture
      break;
  }
}

// ======================================================
// 3. Screen Tap Gesture

void leapOnScreenTapGesture(ScreenTapGesture g){
  int     id               = g.getId();
  Finger  finger           = g.getFinger();
  PVector position         = g.getPosition();
  PVector direction        = g.getDirection();
  long    duration         = g.getDuration();
  float   durationSeconds  = g.getDurationInSeconds();

  println(“ScreenTapGesture: ” + id);
}

// ======================================================
// 4. Key Tap Gesture

void leapOnKeyTapGesture(KeyTapGesture g){
  int     id               = g.getId();
  Finger  finger           = g.getFinger();
  PVector position         = g.getPosition();
  PVector direction        = g.getDirection();
  long    duration         = g.getDuration();
  float   durationSeconds  = g.getDurationInSeconds();

  println(“KeyTapGesture: ” + id);
}
    

4)  How to control Robotic hand gripper

For controlling Robotic hand gripper with leap motion the motor will be used is servo motor. Servo motor is an rotatory actuator that controls angular movement. Servo motor works between 0 to 180 degree. For gripper we are using 0 to 180 degree. When the gripper is open it means servo is rotates to 180 degree and when gripper is closed it means servo is at 0 degree or vice versa.

Here we discuss the servo controlling and connections with Arduino. Servo motors have three wires Red, Black, White. The Red wire attached with arduino 5v  while the Black one is ground and the White is on pin number which is used to control servo angle in programming. We are using pin number 3 in arduino microcontroller. 

5)  How to Code for Leap Motion Controller

Wait is over and we come the interesting part of programming. Here we develop the concepts of programming of motion detector device. In processing software side first we are calling the command of leap motion controller library which is discussed above in detail. Remember we are using serial communications between arduino and processing for controlling hand gripper with 3D gestures recognizing technology.

                                                    import de.voidplus.leapmotion.*;

                                                    import processing.serial.*;

 
 First command tells the Leap motion availability and library. The second one is to serial communication. Now we mention the fingers to control grab strength. By grab strength we give the commands to open and close the paw. In example above you will see the grab strength is between 0 to 1 in float values. Fingers are identified by these commands.

                                                   PVector Thumb=null,Index=null,Middle=null;

This command describe the thumb, index, and middle finger. PVector describes the position vectors. For serial monitor with arduino microcontroller must tell the port number in which your arduino microcontroller is attached with your pc, laptop etc also mention the baud rate speed for transfer data from processing. Here we use 

                                                    port=new Serial(this,”com0″,9600);
You can change baud rate if needed. Change the “com0” with com you are connected your microcontroller. Now time to place positions for fingures. By taking these positions it begins to differentiate the grab strength.
                                                   
                                                                      Thumb=hand.getThumb().getPosition();                    
Getting index and middle finger position is same like getting thumb. As mention above strength of hand is from 0 to 1 these values used in map command to integrate all values between 0 to 1 into servo motor angles 0 to 180 degree. This command saved in some variable then send to arduino software then the actual working of arduino software begins. 
                                                                         map(strength,0,0.8,0,150);                                                      
                                                                          port.write;
Arduino side we define the servo motor pin number in which servo is attached. Read data via serially from processing and write them to hardware. If serial available read the values coming from leap motion controller via processing for hand gripper and write it on servo motor. Now servo motor moves with closing and opening the hands. It detects the values between 0 to 1 in float type and the map command make it divide in small steps for servo 180 degree angles and write it to servo.  

                                                                           myservo1.attach(3);         
                                                                      while(Serial.available())

                                                                           Serial.read();  


                                                                                                   myservo1.write;

For code in align form see: servo-motor-using-leapmotion-arduino  

  


Hope you find it knowledgeable and learn a lot with this post. Keep in touch with us. If you have any query we are always be here for your help. Stay connected with us. There will be much more tutorials and lessons regarding Leap motion technology.

                                                         So click Follow button!  Thanks.


 

previous article
next article
No comments

leave a comment