Twincat Vision Edge Distance Incorrect Results

Introduction

I will say up front, I consider this being more of an edge case than a regular use case. Nevertheless this shows it is very important to sanitize and verify results and that algorithm implementation can have huge effects to end results.

For reference here is the function F_VN_MeasureEdgeDistanceExp.

FUNCTION F_VN_MeasureEdgeDistanceExp : HRESULT
VAR_INPUT
    ipSrcImage             : ITcVnImage;
    fAvgDistance           : Reference To  REAL;
    aStartPoint            : Reference To  TcVnPoint2_REAL;
    aEndPoint              : Reference To  TcVnPoint2_REAL;
    eEdgeDirection         : ETcVnEdgeDirection;
    fMinStrength           : REAL;
    nSearchLines           : UDINT;
    fSearchLineDist        : REAL;
    nMaxThickness          : UDINT;
    bInvertSearchDirection : BOOL;
    fSearchGap             : REAL;
    nSubpixelsIterations   : UDINT;
    fApproxPrecision       : REAL;
    eAlgorithm             : ETcVnEdgeDetectionAlgorithm;
    ipEdgePoints1          : Reference To  ITcVnContainer;
    ipEdgePoints2          : Reference To  ITcVnContainer;
    ipDistances            : Reference To  ITcVnContainer;
    hrPrev                 : HRESULT;
END_VAR

There are four different outputs I will be using in this post for the explanation.

There are a few extra function that are used to calculate sizes and averages of the return values. F_VN_ExportContainerSize is used to find the number of elements in the ipEdgePoints containers. F_VN_ContainerAverage is used for finding the average value from the ipDistances container. F_VN_ContainerAverageElementwise2 is used for finding the average value of [x, y] coordinates from ipEdgePoints containers.

Dimension 1

I configured the function to look for 49 lines (nSearchLines=49). This returned the following.

The number of elements in each ipEdgePoints makes sense as it tries to match up with the defined nSearchLines. If it cannot find an edge point it will be smaller. Same for ipDistances matching the smallest of either ipEdgePoints.

This confirms that fAvgDistance is the average of ipDistances which is the calculated distance between elements of ipEdgePoints. Our first unknown is what and how are elements selected when one edge has fewer points than the other. In this case the difference is only one element or about 2%.

Since the inspected dimension is aligned with the Y axis we can easily calculate the distance using the average Y edge values 1761.62 – 803.67 = 957.95. This, thankfully, matches up with fAvgDistance. If the inspected dimension was not aligned with the X or Y axis, then this would involve a little more math and have different ways of doing it.

Dimension 2

I configured the function to look for 291 lines (nSearchLines=291). This returned the following.

The first thing to notice here is the large difference in elements between the ipEdgePoints. The difference is 272 elements or about 94%. Remember the unknown on how and which elements are used for calculating distances and also what algorithm is used. Now it has a huge impact on the final result.

Since the inspected dimension is aligned with the X axis we can easily calculate the distance using the average X edge values 634.83 – 123.00 = 511.83. This does not match fAvgDistance. Knowing the actual dimensions the value of 511.83 is correct whereas 812.46 is incorrect.

Analysis

The root cause of this discrepancy is the large difference in elements between the two edges. The current algorithm used in F_VN_MeasureEdgeDistanceExp isn't robust enough to deal with this kind of cases. Obviously a difference of 94% of element size is significant, but I would be interested to know at what percentage the current algorithm starts going awry. Is it 25%, 50%, 75%?

The current algorithm likely uses the Euclidean distance between each set of points then averages it all. Let's assume that the length of ipEdgePoints1 and ipEdgePoints2 were equal. This would give the following calculation. ipDistances[i] = sqrt[(ipEdgePoints2[i][x] - ipEdgePoints1[i][x])^2 + (ipEdgePoints2[i][y] - ipEdgePoints1[i][y])^2].

As mentioned before, what happens when they are not the same length. There are a few different approaches. You could sample the longer side, in this case every 16th point from side 1 to every one from side 2. Start from either end and ignore middle ones, or the opposite and start from the middle.

Looking at the results of ipDistances there are 6 values around 512-513 and 12 values around 957-962. What this tells me is that the algorithm starts calculating Euclidean distances from the start of both arrays and ends at the end of the shortest one effectively truncating the longer one. This is what it would visually look like. The 873 = 291 lines * 3 px line space.

Solution

As mentioned before since the measured distance lines up with the X axis, we can use a trivial solution of subtracting the X positions. However this would not work if there was a rotation off X axis.

A solution that would ignore any discrepancy between the number of points would be to best fit two lines to each set of points (linear regression) and then find the distance between the two lines. Using the line with the most points as the base line and thus making the distance tangential to that base line. By doing this you could also easily calculate the angle between both lines. This is actually an important measure when finding distances as you can put thresholds on the angle and if it is more than a few degrees treat it differently or fail the measurement.

Conclusion

I've used metrology software from a few different suppliers and have done this type of measurement with all of them. It does require playing around with parameters, but so far all the others have been able to correctly measure this kind of dimension with their built in tools, while this one from TwinCAT did not do the job.

However, since TwinCAT vision is low level and I have access to a lot of the underlying results, I was able to (easily) investigate and find the root cause. And I am also just as (easily) able to define and create my own measurement algorithm. The other software packages I've worked with are just black boxes and I would essentially be out of luck if it didn't work.

I really like how TwinCAT Vision is directly integrated into the PLC and gives me nearly ultimate control, but it is not a quick and easy metrology or vision solution as compared to using smart cameras or dedicated vision system. However it will likely prove to be cheaper.

If you're currently using Smart Cameras or drag and drop software then I don't think this is a good solution for you. On the other hand if you're already low level vision (for example Matrox Imaging Library) this could an option.

Let me know what you think of TwinCAT Vision!