Trifocal tensor

In computer vision, the trifocal tensor (also tritensor) is a 3×3×3 array of numbers (i.e., a tensor) that incorporates all projective geometric relationships among three views. It relates the coordinates of corresponding points or lines in three views, being independent of the scene structure and depending only on the relative motion (i.e., pose) among the three views and their intrinsic calibration parameters. Hence, the trifocal tensor can be considered as the generalization of the fundamental matrix in three views. It is noted that despite that the tensor is made up of 27 elements, only 18 of them are actually independent. In computer vision, the trifocal tensor (also tritensor) is a 3×3×3 array of numbers (i.e., a tensor) that incorporates all projective geometric relationships among three views. It relates the coordinates of corresponding points or lines in three views, being independent of the scene structure and depending only on the relative motion (i.e., pose) among the three views and their intrinsic calibration parameters. Hence, the trifocal tensor can be considered as the generalization of the fundamental matrix in three views. It is noted that despite that the tensor is made up of 27 elements, only 18 of them are actually independent. There is also a so-called calibrated trifocal tensor, which relates the coordinates of points and lines in three views given their intrinsic parameters and encodes the relative pose of the cameras up to global scale, totalling 11 independent elements or degrees of freedom. The reduced degrees of freedom allow for fewer correspondences to fit the model, at the cost of increased nonlinearity. The tensor can also be seen as a collection of three rank-two 3 x 3 matrices T 1 , T 2 , T 3 {displaystyle {mathbf {T} }_{1},;{mathbf {T} }_{2},;{mathbf {T} }_{3}} known as its correlation slices. Assuming that the projection matrices of three views are P = [ I | 0 ] {displaystyle {mathbf {P} }=} , P ′ = [ A | a 4 ] {displaystyle {mathbf {P} }^{'}=} and P ″ = [ B | b 4 ] {displaystyle {mathbf {P} ^{''}}=} , the correlation slices of the corresponding tensor can be expressed in closed form as T i = a i b 4 t − a 4 b i t , i = 1 … 3 {displaystyle {mathbf {T} }_{i}={mathbf {a} }_{i}{mathbf {b} }_{4}^{t}-{mathbf {a} }_{4}{mathbf {b} }_{i}^{t},;i=1ldots 3} , where a i , b i {displaystyle {mathbf {a} }_{i},;{mathbf {b} }_{i}} are respectively the ith columns of the camera matrices. In practice, however, the tensor is estimated from point and line matches across the three views. One of the most important properties of the trifocal tensor is that it gives rise to linear relationships between lines and points in three images. More specifically, for triplets of corresponding points x ↔ x ′ ↔ x ″ {displaystyle {mathbf {x} };leftrightarrow ;{mathbf {x} }^{'};leftrightarrow ;{mathbf {x} }^{''}} and any corresponding lines l ↔ l ′ ↔ l ″ {displaystyle {mathbf {l} };leftrightarrow ;{mathbf {l} }^{'};leftrightarrow ;{mathbf {l} }^{''}} through them, the following trilinear constraints hold: where [ ⋅ ] × {displaystyle _{ imes }} denotes the skew-symmetric cross product matrix. Given the trifocal tensor of three views and a pair of matched points in two views, it is possible to determine the location of the point in the third view without any further information. This is known as point transfer and a similar result holds for lines and conics. For general curves, the transfer can be realized through a local differential curve model of osculating circles (i.e., curvature), which can then be transferred as conics. The transfer of third-order models reflecting space torsion using calibrated trifocal tensors have been studied, but remains an open problem for uncalibrated trifocal tensors. The classical case is 6 point correspondences giving 3 solutions. The case estimating the trifocal tensor from 9 line correspondences has only recently been solved. Estimating the calibrated trifocal tensor has been cited as notoriously difficult, and requires 4 point correspondences.