Youtube Video Link: https://www.youtube.com/watch?v=m3oWHSV_KkA
About:
This project develops a robotic arm that mirrors human finger gestures in real time using computer vision. A camera and image processing with OpenCV and MediaPipe detect specific finger positions, like a raised thumb, which are then translated into commands for a microcontroller controlling the arm’s motors. The robotic arm mimics the user’s gestures, demonstrating a simple approach to gesture-based control with potential applications in assistive robotics and human-machine interaction.
🔑 Key Features:
1. Gesture Control: A person uses hand gestures to control a robotic hand. Likely uses sensors (e.g., flex or IMU sensors) attached to a glove or arm to detect hand movement.
2. Signal Processing and Control Flow: Input Processing Module: Captures and interprets gesture signals. Controller (e.g., Arduino, Raspberry Pi): Converts processed signals into control commands. Motor Driver: Acts as an interface between the controller and motors, enabling motor actuation. Motor: Drives the fingers/joints of the robotic hand. Power Supply: Powers the entire system, including motors and electronics.
3. Robotic Hand: 3D-printed or mechanical hand that mimics human hand movements. Likely driven by servo motors or actuators.
4. Real-Time Mimicking: The robotic hand mimics the gestures of the human hand in real time, suggesting minimal latency and effective control logic.
5. Human-Machine Interaction (HMI): Demonstrates intuitive interaction between a human operator and a robotic device.
🧰Hardware Components:
1. Flex Sensors / IMU Sensors (e.g., MPU6050) Used to detect finger bending or hand orientation for gesture recognition.
2. Microcontroller (e.g., Arduino Uno / Nano) Reads sensor data and processes gestures. Sends control signals to the robotic hand.
3. Motor Driver Module (e.g., L293D / L298N) Drives the motors based on control signals from the microcontroller.
4. Servo Motors (typically SG90 or MG996R) Control the movement of each finger of the robotic hand.
5. Robotic Hand (3D-Printed or Mechanical) Designed to replicate human hand movements. Actuated by servo motors or similar actuators.
6. Power Supply (e.g., 5V–12V DC Adapter or Battery Pack) Powers the microcontroller and servo motors.
7. Glove or Wristband Holds the sensors in place on the user’s hand.
8. Connecting Wires / Jumper Cables For interfacing all the electronic components.
9. Breadboard or PCB For circuit assembly and prototyping.
10. Optional: Bluetooth / RF Module (e.g., HC-05 or NRF24L01) For wireless transmission of gesture data (if not wired).
💻Software Components:
1. Arduino IDE Used to write, compile, and upload code to the microcontroller (e.g., Arduino Uno/Nano). Supports libraries for sensors (like MPU6050) and motor control.
2. Sensor Libraries (Arduino): MPU6050.h / Wire.h – For reading data from the IMU sensor. Servo.h – For controlling servo motors. FlexSensor.h (if flex sensors are used) – For analog input processing.
3. Serial Monitor / Plotter (in Arduino IDE) Used for debugging sensor readings and verifying gesture data in real-time.
4. Embedded C/C++ Programming Language used to program the microcontroller for sensor input processing and motor control.
5. Optional: Python (for PC-side visualization or testing) If the system includes computer interfacing (e.g., via serial or Bluetooth), Python can be used with libraries like pySerial for visualization or gesture logging.
6. CAD Software (e.g., SolidWorks, Fusion 360, or Tinkercad) For designing the 3D-printed robotic hand components.
7. Slicing Software (e.g., Cura, PrusaSlicer) Converts 3D hand models into G-code for 3D printing.
8. Firmware (Pre-loaded on Microcontroller) Core firmware that handles input-output timing, motor control logic, and system safety checks.
Circuit Diagram:
Arduino Code:
#include <Servo.h>
// Define servo objects
Servo thumbServo;
Servo indexServo;
Servo middleServo;
Servo ringServo;
Servo pinkyServo;
// Flex sensor input pins
const int thumbPin = A0;
const int indexPin = A1;
const int middlePin = A2;
const int ringPin = A3;
const int pinkyPin = A4;
// Servo output pins
const int thumbServoPin = 3;
const int indexServoPin = 5;
const int middleServoPin = 6;
const int ringServoPin = 9;
const int pinkyServoPin = 10;
void setup() {
Serial.begin(9600);
// Attach servo motors
thumbServo.attach(thumbServoPin);
indexServo.attach(indexServoPin);
middleServo.attach(middleServoPin);
ringServo.attach(ringServoPin);
pinkyServo.attach(pinkyServoPin);
}
void loop() {
// Read flex sensor values
int thumbVal = analogRead(thumbPin);
int indexVal = analogRead(indexPin);
int middleVal = analogRead(middlePin);
int ringVal = analogRead(ringPin);
int pinkyVal = analogRead(pinkyPin);
// Map flex sensor values (adjust 200–700 based on your sensor’s range)
int thumbAngle = map(thumbVal, 200, 700, 0, 180);
int indexAngle = map(indexVal, 200, 700, 0, 180);
int middleAngle = map(middleVal, 200, 700, 0, 180);
int ringAngle = map(ringVal, 200, 700, 0, 180);
int pinkyAngle = map(pinkyVal, 200, 700, 0, 180);
// Write angles to servos
thumbServo.write(thumbAngle);
indexServo.write(indexAngle);
middleServo.write(middleAngle);
ringServo.write(ringAngle);
pinkyServo.write(pinkyAngle);
// Print to Serial Monitor (for debugging)
Serial.print(“Thumb: “); Serial.print(thumbAngle);
Serial.print(” | Index: “); Serial.print(indexAngle);
Serial.print(” | Middle: “); Serial.print(middleAngle);
Serial.print(” | Ring: “); Serial.print(ringAngle);
Serial.print(” | Pinky: “); Serial.println(pinkyAngle);
delay(100); // Short delay for stability
}
For More Details & Project Support: