Robotics is a constantly evolving field nowadays. A robot is a mechanical device that can perform physical tasks while being controlled and supervised by humans. Several robots have been created to perform risky tasks impossible for humans to act directly. The robot arm is one of the most popular types. In this project, it is proposed to control a robot arm using finger and hand movements. However, the traditional way of gesture control employs sensor-based gloves to track motion, which is time and energy-intensive due to its weight. To overcome this problem, using vision for motion tracking can be the most suited method at hand currently. The ML-based computer vision method provides real-time landmarks on different hand points. With these landmarks, we can calculate various mathematical parameters that can be used to control the robot arm. Movements that are landmarked are processed and formulated using mathematical parameters. The final output of mathematical parameters is fed to the microcontroller. Signals from the microcontroller will be the desired input signal to the motors. The rotation of the robot arm is controlled using the movement of the human arm. This project’s end application is like human operation in dull, dirty, and dangerous environments. The project’s primary goal is to protect those who come into direct contact with these environments. Bomb defusing, painting, welding, and other applications are a few examples of where this project is helpful.