Mastodawn

Mikha Ky.Dec 7, 2025

After a few fruitless attempts it becomes apparent that the problem is probably
in the lack of degrees of freedom of the robot movements in the 3d space. The #handeye calibration solves equations in 6 dimensions: 3 translational + 3 rotational. Our robot, however, cannot jump up and down. So there's no variability in the Z axis translation. Also the robot cannot tilt. It simply rolls on a flat floor and can only rotate left and right. So there's no rotations of the robot, and Therefore of the camera, around X and Y axes. So, out of 6 possible degrees of freedom in the 3d space, the 3 translational dimensions and 3 rotations we can only utilize 3 of them:
- translation in X directions,
- translation in Y directions, and
- rotation around Z axis.

This really sounds that we are very short of measurements variability. We gotta get more creative on how we can enrich our measurements, or maybe introduce the constraints into the algorithm somehow. Think. Think...

#PhotonVision #openCV #computerVision #Limelight #WPILib

Show thread

Mikha Ky.Nov 8, 2025

OK, we tried. The kids on our robotics team tried to implement this Hand-Eye Calibration on the 2025 season robot to calibrate the robot-to-camera transformation based on the Swerve drive odometry and the April tag localization. It wouldn't "work at the first try." We checked and rechecked that the measurements are in correct order with good timing. But the calibration results are all over the place. The rotation is somewhat close to what's expected but the translation reports like a couple of meters outside the robot perimeter. And it feels like the estimate is not very stable, it vary a lot from one series of measurements to another. So I'm thinking, maybe the accuracy of the measurements affects the result more than we expected. Maybe the calibration algorithm requires more precision than we can get from the robot sensors. We're going to be trying more but...

If anybody has an experience with Hand-Eye Calibration in the real world applications, or knows the theory behind this algorithm and can suggest what to focus on to get a usable result please do.

#HandEyeCalibration #OpenCV #ComputerVision #AprilTags #PhotonVision

Mikha Ky.Nov 1, 2025

Context: #FRCRobot, #AprilTags, Camera mounted on the #robot

Problem: Can we find the translation and the rotation of the camera relative to the robot chassis programmatically fully or somewhat automated?

Idea: #OpenCV library has a powerful calibration subroutine called "Hand Eye Calibration" which can calculate the relationship between the "hand" and the "eye" in a setup that the hand holds the eye in a fixed manner and moves relative the "base". The camera (the eye) can see a target that is also fixed relative to the base. Can we use it in our robot with an eye rolling on a field with a fixed April Tag configuration?

Let's set the terminology. The "eye" is the eye - the camera. The camera is mounted on the robot. So the robot is the "hand" holding the "eye". The robot can move across the field freely and we can track this movement using robot's odometry. So the field is the "base". And the "target", the April Tag, is mounted on the field/base.

This hand-eye calibration function requires two sets of measurements as inputs: Eye-to-target transformations and Base-to-hand transformations. Each transformation is a translation vector and a rotation matrix. In our case "eye-to-target" is camera-to-target which we can obtain from #PhotonVision. And the "base-to-hand" is the robot pose on the field which we can get from the drivetrain odometry. Then the calibration result will be hand-to-eye which is the robot-to-camera transformation.

Should work, right?

#omgRobots