Charuco targets are a merge between checkerboard patterns and aruco targets and it goes without saying that charuco targets are one of many custom targets we provide.

Aruco boards and makers are essential because of their flexibility and ease of detection. However, one of the challenges users face with aruco targets is that the precision of the edges is not as high as desired.

Even if you try subpixel refinement of these corners, the outcome is still not as accurate as required.

Chessboard patterns are easier to fine-tune the corners, given each corner has two black squares. Unfortunately, the versatility of a chessboard target is not as extensive as that of an aruco target. When using the chessboard pattern, no omissions are acceptable, and your board should be entirely visible.

Charuco targets, therefore, combine both patterns to achieve the benefits of both of them.

The aruco areas aid in the interpolation of chessboard corner positions. This way, you can achieve the flexibility of aruco makers as they permit you to use them with some sections blocked from view.

What’s more, given the interpolated corners belong to chessboard patterns, they have high precision when it comes to subpixel accuracy.

So, when calibrating your camera or conducting other applications demanding a high degree of exactness, charuco boards offer a better option than both aruco and chessboard targets.

The most significant advantage of charuco targets lies in the unique coding and identifiability of checker areas. Therefore, even partially obscured and unsuitable camera photos are usable in camera calibration.

For example, immense ring lights can result in the non-uniform exposure of your calibration target. That is, some sections experience semi-specular reflection, something that would fail checkerboard patterns.

When using a charuco target, however, the remaining visible saddle points are still adequate for camera calibration. With charuco targets, you can fine-tune the identification of saddle points with subpixel accuracy, as is the case with checkerboard patterns.

Data at the extreme peripheries of images is a very useful feature. Since you can position charuco targets to be partially visible to your camera, you can record observations from your camera images’ extreme corners and edges.

Access to this information often results in excellent and robust identification of your lens’s distortion parameters. So, when you have access to OpenCV 3.x, charuco targets are the best to use.

Charuco targets are typically applicable when calibrating several cameras, i.e. stereo calibration. During this process, you need to implement a code. This code should help you identify the points that each camera detected and the points of intersection detected by all cameras under calibration.

OpenCV’s aruco module provides a constant function, cv::aruco::CharucoBoard::create(), that eases the charuco board creation process.

Here is a breakdown:

cv::aruco::CharucoBoard board = cv::aruco::CharucoBoard::create(5, 7, 0.04, 0.02, dictionary);

In which,

• 5 and 7 represent the number of squares in the horizontal and vertical axes, respectively
• 0.04 and 0.02 are dimensions of squares and makers
• Dictionary= makers’ dictionary

Though each maker’s ids (unique identifiers) get assigned by default; you can change them from board.ids. After obtaining your charuco board object, the next step is printing it.

Chessboard corners are essentially what you detect when detecting your charuco board. Each corner has its id beginning from zero to the number of corners in your charuco board.

• Detecting charuco boards begins with capturing the input image. This is the original image in which you’ll detect your makers. The images aid in charuco board subpixel corner refinement.
• Next is identifying your calibration parameters, a step that is not vital if you are detecting your charuco corners without calibrating.
• Maker detection followed by maker interpolation comes in next. During maker detection, parameters for detecting without calibration and when calibrating defer slightly. Nevertheless, the charuco corner detection function remains the same.
• cv::aruco::interpolateCornersCharuco().

Where calibration parameters are available, charuco targets' corners get interpolated in two steps.

• Aruco maker rough pose estimation
• Charuco corner reprojection to the input image

Without calibration parameters, the process involves calculating the tallying homography between your charuco image projection and the charuco plane.

The main issue with homography usage is the susceptibility of interpolation to image distortion. Homography only gets performed with closely positioned charuco corner makers to minimize distortion effects.

You should disable maker corner refinement during charuco board detection, especially when applying homography. This way, you prevent the subpixel process from providing corner position deviations given the closeness of checkerboard squares. The variations affect the interpolation of charuco corners resulting in poor outcomes.

Moreover, the only corners returned are those in which the two neighboring makers get found. Failure to detect the neighboring makers around a charuco corner denotes occlusion or poor observation quality within the region.

It’s best to ignore unreturned charuco corners to ensure high accuracy of interpolated ones. A subpixel refinement occurs upon the interpolation of charuco corners.

After charuco corner interpolation, the next step is drawing the corners to verify detection accuracy. You can do so using the drawDetectedCornersCharuco() function.

While pose estimating, position the charuco board coordinate system with its Z-axis pointing outwards and at the board’s bottom left corner.

The function to use is estimatePoseCharucoBoard(). You get true with correct pose estimation, and false if wrong. Leading reasons for a false outcome include:

• Inadequate number of charuco corners to facilitate the estimation process
• The charuco corners are in one line

The deep charuco pose estimation system aims to enable pose estimation in low-light conditions and blurred images. Deep charuco incorporates RefineNet and ChArUcoNet networks while combining them with PnP (Perspective-n-Point) algorithm for 6DoF maker pose estimation.

ChArUcoNet provides 2D point locations and ID-specific classifiers. RefineNet comes in to fine-tune the 2D point location into sub pixel coordinates.

Once you pick charuco targets for calibration, identifying a suitable size is essential. The size of your charuco board relates to the dimensions of the FOV (field of view) of your last application. Considering this is vital as you need to focus your cameras on that particular distance as you calibrate.

Adjusting your focus distance though slightly, still impacts your focal length. As a result, the previous calibrations you performed will be valueless.

Even adjusting your lens aperture detrimentally affects the accuracy of your calibration, and you should therefore refrain from altering the f/stop.

To ensure calibration accuracy, ensure your charuco target occupies most of your camera image. If you decide to use a small charuco plate, multiple variations of camera parameters can account for the taken observations.

In short, the charuco board needs to occupy a minimum of half your camera’s pixel area in a frontal observation.