In this paper , we provide a perturbation methodology to solve the derivative of 2D transform, including transition and rotation. We denote ( x y θ ) (x\ y\ \theta) (x y θ) as our vehicle 2D-pose, as most of our project used.
In this section, We introduce the coordinate convention about the image point and the basic 2D matrix transformation. Then we derive its derivative by perturbation method. And finally we give a example of sliding window optimization as used in most of SLAM proble.
We denote the homogeneous coordinate ( u v 1 ) (u\ v\ 1) (u v 1) as the point in the image . For convenient, we shift it by the image center ( c x c y ) (c_{x}\ c_{y}) (cx cy) since its more convenient for 2D image transformation,as shown in (1). We use P ( u c v c 1 ) P(u_{c}\ v_{c} 1) P(uc vc1) or P ( p c 1 ) P(p_{c}\ 1) P(pc 1) as 2D Point coordinates in the following derivatives.
(1) u c = u − c x v c = v − c y u_{c} = u - c_{x} \\ v_{c} = v - c_{y} \tag{1} uc=u−cxvc=v−cy(1)
The transformation matrix T ∈ S E ( 2 ) \ T \in SE(2) T∈SE(2) of the pose ξ ( x y θ ) ∈ s e ( 2 ) \xi\ (x\ y\ \theta) \in se(2) ξ (x y θ)∈se(2) is shown as (2),we use it to transform a image point p c ( u c v c 1 ) p_{c}(u_c\ v_c\ 1) pc(uc vc 1) from image coordinate to the wold coordinate.
(2) T = [ c o s ( θ ) − s i n ( θ ) x s i n ( θ ) c o s ( θ ) y 0 0 0 ] = [ R o t ( θ ) t 0 1 ] T = \left[ \begin{matrix} cos(\theta) & -sin(\theta) & x \\ sin(\theta) & cos(\theta) & y \\ 0 & 0 & 0 \end{matrix} \right] = \left[ \begin{matrix} Rot(\theta) & t \\ 0 & 1 \\ \end{matrix} \right] \tag{2} T=⎣⎡cos(θ)sin(θ)0−sin(θ)cos(θ)0xy0⎦⎤=[Rot(θ)0t1](2)
We the right presentation of equation (1) since its more compact.The inverse of T is shown in (3).
(3) T − 1 = [ R o t ( − θ ) − R o t ( − θ ) t 0 1 ] T^{-1} = \left[ \begin{matrix} Rot(-\theta) & -Rot(-\theta) t \\ 0 & 1 \\ \end{matrix} \right] \tag{3} T−1=[Rot(−θ)0−Rot(−θ)t1](3)
In order to solve the derivative of the 2D transformation matrix, we firstly introduce the methodology of perturbation. Let’s first think of the question: To add a small update δ ξ ( δ t δ θ ) \delta \xi\ (\delta t\ \delta\theta) δξ (δt δθ) to the known pose ξ \xi ξ ,what manipulation shall we do to the transformation matrix ? The answer is show in (4).
(4) [ R o t ( θ ) R o t ( δ θ ) t + δ t 0 1 ] = [ R o t ( θ ) t 0 1 ] [ R o t ( δ θ ) R o t ( − θ ) δ t 0 1 ] \left[ \begin{matrix} Rot(\theta)Rot(\delta \theta) & t +\delta t \\ 0 & 1 \\ \end{matrix} \right] = \left[ \begin{matrix} Rot(\theta) & t \\ 0 & 1 \\ \end{matrix} \right] \left[ \begin{matrix} Rot(\delta \theta) & Rot(-\theta)\delta t \\ 0 & 1 \\ \end{matrix} \right] \tag{4} [Rot(θ)Rot(δθ)0t+δt1]=[Rot(θ)0t1][Rot(δθ)0Rot(−θ)δt1](4)
For simplicity ,we use per ( ξ ) ( \xi) (ξ) denote mapping from a perturbation vector in se(2) to the right multiple matrix in SE(2) as shown in (5)
(5) p e r ( δ ξ ) = [ R o t ( δ θ ) R o t ( − θ ) δ t 0 1 ] per(\delta\xi)= \left[ \begin{matrix} Rot( \delta \theta) & Rot(-\theta)\delta t \\ 0 & 1 \end{matrix} \right] \tag{5} per(δξ)=[Rot(δθ)0Rot(−θ)δt1](5)
So the derivative of 2d transformation T can be written as (6).
(6) ∂ T ∂ δ ξ = lim ∂ δ ξ → 0 T p e r ( δ ξ ) P − T P δ ξ \frac{\partial T}{\partial \delta\xi} = \lim_{\partial \delta \xi \to 0} \frac{Tper(\delta\xi)P-TP}{\delta \xi} \tag{6} ∂δξ∂T=∂δξ→0limδξTper(δξ)P−TP(6)
When δ θ \delta\theta δθ is very small , R o t ( δ θ ) Rot(\delta\theta) Rot(δθ) can be approximated to (7).
(7) R o t ( δ θ ) = [ c o s ( δ θ ) − s i n ( δ θ ) s i n ( δ θ ) c o s ( δ θ ) ] = [ 1 − δ θ ) δ θ 1 ] = I + R o t ( π 2 ) δ θ Rot(\delta\theta)=\left[ \begin{matrix} cos(\delta\theta) & -sin(\delta\theta) \\ sin(\delta\theta) & cos(\delta\theta) \end{matrix} \right] \\= \left[ \begin{matrix} 1 & -\delta\theta) \\ \delta\theta & 1 \end{matrix} \right] \\ = I + Rot(\frac{\pi}{2})\delta\theta \tag{7} Rot(δθ)=[cos(δθ)sin(δθ)−sin(δθ)cos(δθ)]=[1δθ−δθ)1]=I+Rot(2π)δθ(7)
Consequently (6) can expand as follow:
(8) ∂ T ∂ δ ξ = lim ∂ δ ξ → 0 T p e r ( δ ξ ) P − T P δ ξ = lim ∂ δ ξ → 0 [ R o t ( θ ) t 0 1 ] [ I + R o t ( π 2 ) δ θ R o t ( − θ ) δ t 0 1 ] [ p c 1 ] − [ R o t ( θ ) t 0 1 ] [ p c 1 ] δ ξ = [ R o t ( π 2 ) δ θ P c δ t ] δ ξ = [ I 2 × 2 R o t ( π 2 ) P c ] \frac{\partial T}{\partial \delta\xi} =\lim_{\partial \delta \xi \to 0} \frac{Tper(\delta\xi)P-TP} {\delta \xi} \\ =\lim_{\partial \delta\xi \to 0} \frac{ \left[ \begin{matrix} Rot(\theta) & t \\ 0 & 1 \\ \end{matrix} \right] \left[ \begin{matrix} I + Rot(\frac{\pi}{2})\delta\theta & Rot(-\theta)\delta t \\ 0& 1\\\end{matrix} \right] \left[ \begin{matrix} p_{c} \\1 \end{matrix} \right]- \left[\begin{matrix} Rot(\theta) & t \\ 0 & 1 \end{matrix}\right] \left[ \begin{matrix}p_{c} \\ 1 \\ \end{matrix}\right] }{\delta\xi} \\= \frac{ \left[\begin{matrix}Rot(\frac{\pi}{2})\delta\theta P_{c} & \delta t \\\end{matrix}\right] }{ \delta\xi } \\= \left[ \begin{matrix} I_{2\times 2} & Rot(\frac{\pi}{2}) P_{c} \end{matrix}\right]\tag{8} ∂δξ∂T=∂δξ→0limδξTper(δξ)P−TP=∂δξ→0limδξ[Rot(θ)0t1][I+Rot(2π)δθ0Rot(−θ)δt1][pc1]−[Rot(θ)0t1][pc1]=δξ[Rot(2π)δθPcδt]=[I2×2Rot(2π)Pc](8)
Note the order in δ ξ \delta\xi δξ is [ δ t δ θ ] [\delta t\ \delta\theta] [δt δθ] corresponding to the transition and rotation perturbation.
Assume we have a serial pose ξ s i i ∈ [ 1 n ] \xi_{si}\ i\in[1\ n] ξsi i∈[1 n] corresponding to sliding window, and current pose ξ c \xi_{c} ξc. Optimization process will adjust current pose making the cost function decrease.The cost function is shown in(9).
1 2 ∑ i = 1 n ∑ j ∣ ∣ e i ∣ ∣ 2 = 1 2 ∑ i = 1 n ∑ j ∣ ∣ I s ( ( T s i w ) − 1 T c w P j c ) − I c ( P j c ) ∣ ∣ 2 = 1 2 ∑ i = 1 n ∑ j ∣ ∣ I s ( ( T c s i ) P j c ) − I c ( P j c ) ∣ ∣ 2 = 1 2 ∑ i = 1 n ∑ j ∣ ∣ e ( ξ c s i ) ∣ ∣ 2 \frac{1}{2}\sum_{i =1}^{n}\sum_{j}||e_{i}||^{2} = \frac{1}{2}\sum_{i=1}^{n}\sum_{j}||I_{s}( (T_{si}^{w})^{-1} T_{c}^{w}P_{j}^{c}) - I_{c}(P_{j}^{c})||^{2} \\= \frac{1}{2}\sum_{i=1}^{n}\sum_{j}|| I_{s}( (T_{c}^{si})P_{j}^{c} )- I_{c}(P_{j}^{c})||^{2} \\= \frac{1}{2}\sum_{i=1}^{n}\sum_{j}||e(\xi_{c}^{si})||^{2} 21i=1∑nj∑∣∣ei∣∣2=21i=1∑nj∑∣∣Is((Tsiw)−1TcwPjc)−Ic(Pjc)∣∣2=21i=1∑nj∑∣∣Is((Tcsi)Pjc)−Ic(Pjc)∣∣2=21i=1∑nj∑∣∣e(ξcsi)∣∣2
Note T c w T_{c}^{w} Tcw denote current pose which is going to be optimize, can also represent as e x p ( ξ c ) exp(\xi_{c}) exp(ξc). And T s i w T_{si}^{w} Tsiw denote the i t h i_{th} ith pose of sliding window. P j c P_{j}^{c} Pjc denote the point in the current image. T c s i T_{c}^{si} Tcsi is a multiple of T s i w − 1 T_{si}^{w -1} Tsiw−1 and T c w T_{c}^{w} Tcw as shown in (9).
(9) T s i w − 1 T c w = [ R o t ( θ c − θ s i ) R o t ( − θ s i ) ( t c − t s i ) 0 1 ] T_{si}^{w -1} T_{c}^{w} = \left[\begin{matrix} Rot(\theta_{c}-\theta_{si}) & Rot(-\theta_{si})(t_{c}-t_{si}) \\ 0 & 1 \end{matrix}\right] \tag{9} Tsiw−1Tcw=[Rot(θc−θsi)0Rot(−θsi)(tc−tsi)1](9)
Now we add perturbation to T c w T_{c}^{w} Tcw and use approximation of first order Taylor expand as shown in (10):
(10) e ( ξ c s i ⊕ δ ξ c ) = I s ( T c s i p e r ( δ ξ c ) P j c ) − I c ( P j c ) ≈ I s ( T c s i P j c ) − I c ( P j c ) + J c ∗ e x p ( δ ξ c ) J c = ∂ I s ∂ u ∗ ∂ u ∂ δ ξ c u = T c s i ∗ p e r ( δ ξ c ) P j c e(\xi^{si}_{c} \oplus \delta\xi_{c} ) = I_{s}( T_{c}^{si}per(\delta\xi_{c})P_{j}^{c} )- I_{c}(P_{j}^{c}) \\\approx I_{s}( T_{c}^{si} P_{j}^{c}) - I_{c}(P_{j}^{c})+ J_{c}* exp(\delta\xi_{c})\tag{10} \\ J_{c} = \frac{\partial I_{s} }{\partial u}* \frac{\partial u}{\partial \delta\xi_{c}} \\ u = T_{c}^{si}*per(\delta\xi_{c})P_{j}^{c} e(ξcsi⊕δξc)=Is(Tcsiper(δξc)Pjc)−Ic(Pjc)≈Is(TcsiPjc)−Ic(Pjc)+Jc∗exp(δξc)Jc=∂u∂Is∗∂δξc∂uu=Tcsi∗per(δξc)Pjc(10)
∂ I s ∂ u \frac{\partial I_{s} }{\partial u} ∂u∂Is is the gradient of the image, which can represent as [ I x s I y s ] [I_{x}^{s} I_{y}^{s}] [IxsIys].
Here we combine (5), (7) and (9) to derive the solution of ∂ u ∂ δ ξ \frac{\partial u}{\partial\delta\xi} ∂δξ∂u ,as shown in (11)
(11) ∂ u ∂ δ ξ c = lim δ ξ → 0 T c s i ∗ p e r ( δ ξ c ) P j c − T c s i P j c δ ξ c = lim δ ξ c → 0 [ R o t ( θ c − θ s i ) R o t ( − θ s i ) ( t c − t s i ) 0 1 ] [ R o t ( δ θ c ) R o t ( − θ c ) δ t 0 1 ] P j c − T c s i P j c δ ξ c = lim δ ξ c → 0 R o t ( − θ s i ) δ t c + R o t ( θ c − θ s i + π ) 2 P c δ θ c δ ξ c = [ R o t ( − θ s i ) R o t ( θ c − θ s i + π ) 2 P c ] \frac{\partial u}{\partial\delta\xi_{c}} = \lim_{\delta\xi\to0} \frac{T_{c}^{si}*per(\delta\xi_{c})P_{j}^{c} - T_{c}^{si}P_{j}^{c}} {\delta\xi_{c}}\\=\\ \lim_{\delta\xi_{c}\to0}\frac{ \left[\begin{matrix} Rot(\theta_{c}-\theta_{si}) & Rot(-\theta_{si})(t_{c}-t_{si}) \\ 0 & 1 \end{matrix}\right] \left[\begin{matrix} Rot(\delta\theta{c}) & Rot(-\theta_{c})\delta t \\ 0 &1 \end{matrix}\right]P_{j}^{c}- T_{c}^{si}P_{j}^{c}}{\delta\xi_{c}}\\= \lim_{\delta\xi_{c}\to0}\frac{Rot(-\theta_{si})\delta t_{c} + Rot(\theta_{c}-\theta_{si}+\frac{\pi)}{2}P_{c}\delta\theta_{c} }{\delta\xi_{c}}\\= \left[\begin{matrix} Rot(-\theta_{si}) & Rot(\theta_{c}-\theta_{si}+\frac{\pi)}{2}P_{c}\end{matrix}\right]\tag{11} ∂δξc∂u=δξ→0limδξcTcsi∗per(δξc)Pjc−TcsiPjc=δξc→0limδξc[Rot(θc−θsi)0Rot(−θsi)(tc−tsi)1][Rot(δθc)0Rot(−θc)δt1]Pjc−TcsiPjc=δξc→0limδξcRot(−θsi)δtc+Rot(θc−θsi+2π)Pcδθc=[Rot(−θsi)Rot(θc−θsi+2π)Pc](11)
Then we derive the final form :
(12) J c = [ I x s I y s ] [ R o t ( − θ s i ) R o t ( θ c − θ s i + π ) 2 P c ] J_{c} = [I_{x}^{s}\ I_{y}^{s}]\left[\begin{matrix} Rot(-\theta_{si}) & Rot(\theta_{c}-\theta_{si}+\frac{\pi)}{2}P_{c}\end{matrix}\right] \tag{12} Jc=[Ixs Iys][Rot(−θsi)Rot(θc−θsi+2π)Pc](12)
Using the similar method we can also derive of jacobian J s i J_{si} Jsicorresponding to pose ξ s i \xi_{si} ξsi ,here we omit the derivation process since its almost the same as mentioned, and directly give the result shown in (13)
(1) J s i = [ I x s I y s ] [ R o t ( − θ s i ) R o t ( π 2 − θ s i ) ( − R o t ( θ c ) P c − t c + t s i ) ] J_{si} = [I_{x}^{s}\ I_{y}^{s} ]\left[\begin{matrix} Rot(-\theta_{si}) & Rot(\frac{\pi}{2}-\theta_{si} )(-Rot( \theta_{c})P_{c} -t_{c}+t_{si})\end{matrix}\right] \tag{1} Jsi=[Ixs Iys][Rot(−θsi)Rot(2π−θsi)(−Rot(θc)Pc−tc+tsi)](1)