“Estimation of Ship-Deck Motion using LIDAR,Gyroscopes and Cameras”
Authors: Hans K.R. Holen, Alexander M. Sjøberg and Olav Egeland,Affiliation: NTNU
Reference: 2021, Vol 42, No 3, pp. 99-112.
Keywords: Vanishing points, sensor fusion, vision, offshore motion compensation
Abstract: This paper presents a system for the estimation of ship deck motion using camera, lidar and gyroscopes. A camera is used in a vision system that is based on the detection of lines as input to a vanishing point detector. This is done under a Manhattan assumption for man-made structures where the majority of lines are along 3 orthogonal axes. Two sets of parallel orthogonal lines are detected for the ship deck, and this is used to estimate the attitude using a complementary filter with input from lidar and gyroscopes. Since the vision algorithm depends on lines rather than points, the system is more resistant to occlusions like vision algorithms based on point tracking. In addition, a lidar is used to measure the distance between the sensor frame and the plane, and gyroscopes are used to improved the accuracy of the estimates. The system is validated in real time in lab experiments on a model of a ship.
PDF (5040 Kb)
DOI: 10.4173/mic.2021.3.1References:
[1] Abdel-Rahman, E., Nayfeh, A., and Masoud, Z. (2003). Dynamics and control of cranes: A review, Journal of Vibration and Control. 9:863--908. doi:10.1177/1077546303009007007
[2] Barnard, S.T. (1983). Interpreting perspective images, Artificial Intelligence. 21(4):435--462. doi:10.1016/S0004-3702(83)80021-6
[3] Bouguet, J.-Y. (1999). Visual methods for three-dimensional modeling, Ph.D. thesis, California Institute of Technology. doi:10.7907/hc2c-sp47
[4] Caprile, B. and Torre, V. (1990). Using vanishing points for camera calibration, International journal of computer vision. 4(2):127--139. doi:10.1007/BF00127813
[5] Coughlan, J.M. and Yuille, A.L. (2003). Manhattan world: Orientation and outlier detection by bayesian inference, Neural Computation. 15(5):1063--1088. doi:10.1162/089976603765202668
[6] Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., and Marín-Jiménez, M.J. (2014). Automatic generation and detection of highly reliable fiducial markers under occlusion, Pattern Recognition. 47(6):2280--2292. doi:10.1016/j.patcog.2014.01.005
[7] Golub, G.H. and VanLoan, C.F. (1996). Matrix Computations (3rd Ed, ). Johns Hopkins University Press, Baltimore, MD, USA.
[8] Hartley, R., Trumpf, J., Dai, Y., and Li, H. (2013). Rotation averaging, International Journal of Computer Vision. 103(3):267--305. doi:10.1007/s11263-012-0601-0
[9] Hartley, R. and Zisserman, A. (2004). Multiple View Geometry in Computer Vision, Cambridge University Press, 2 edition. doi:10.1017/CBO9780511811685
[10] Kok, M., Hol, J., and Schön, T. (2017). Using inertial sensors for position and orientation estimation, Foundation and Trends of Signal Processing. 11. doi:10.1561/2000000094
[11] Lin, S., Garratt, M., and Lambert, A. (2017). Monocular vision-based real-time target recognition and tracking for autonomously landing an uav in a cluttered shipboard environment, Autonomous Robots. 41. doi:10.1007/s10514-016-9564-2
[12] Ma, Y., Soatto, S., Ko secka, J., and Sastry, S.S. (2003). An Invitation to 3-D Vision: From Images to Geometric Models, Springer Verlag. doi:10.1017/S002555720017809X
[13] Magee, M. and Aggarwal, J. (1984). Determining vanishing points from perspective images, Computer Vision, Graphics, and Image Processing. 26(2):256--267. doi:10.1016/0734-189X(84)90188-9
[14] Magri, L. and Fusiello, A. (2014). T-linkage: A continuous relaxation of j-linkage for multi-model fitting, In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 3954--3961. doi:10.1109/CVPR.2014.505
[15] Mahony, R., Hamel, T., and Pflimlin, J.-M. (2008). Nonlinear complementary filters on the special orthogonal group, IEEE Transactions on Automatic Control. 53(5):1203--1218. doi:10.1109/TAC.2008.923738
[16] Manton, J. (2005). A globally convergent numerical algorithm for computing the centre of mass on compact lie groups, volume3. pages 2211 -- 2216. doi:10.1109/ICARCV.2004.1469774
[17] Moakher, M. (2002). Means and averaging in the group of rotations, SIAM Journal on Matrix Analysis and Applications. 24. doi:10.1137/S0895479801383877
[18] Mueller, A. (2019). Modern Robotics: Mechanics, Planning, and Control [Booksh, .
[19] Park, F.C. (1995). Distance metrics on the rigid-body motions with applications to mechanism design, J. Mechanical Design. 117(1):48--54. doi:10.1115/1.2826116
[20] Patruno, C., Nitti, M., Petitti, A., Stella, E., and D'Orazio, T. (2019). A vision-based approach for unmanned aerial vehicle landing, Journal of Intelligent & Robotic Systems. 95. doi:10.1007/s10846-018-0933-2
[21] Polvara, R., Sharma, S., Wan, J., Manning, A., and Sutton, R. (2017). Towards autonomous landing on a moving vessel through fiducial markers, 2017. doi:10.1109/ECMR.2017.8098671
[22] Ramli, L., Mohamed, Z., Abdullahi, A., Jaafar, H.I., and M.Lazim, I. (2017). Control strategies for crane systems: A comprehensive review, Mechanical Systems and Signal Processing. 95C:1--23. doi:10.1016/j.ymssp.2017.03.015
[23] Rauscher, F., Nann, S., and Sawodny, O. (2018). Motion control of an overhead crane using a wireless hook mounted IMU, In Proceedings 2018 American Control Conference (ACC). IEEE, pages 5677--5682. doi:10.23919/ACC.2018.8431170
[24] Tardif, J.-P. (2019). Non-iterative approach for fast and accurate vanishing point detection, In ICCV. IEEE Computer Society, pages 1250--1257. doi:10.1109/ICCV.2009.5459328
[25] Toldo, R. and Fusiello, A. (2008). Robust multiple structures estimation with j-linkage, In European conference on computer vision. Springer, pages 537--547. doi:10.1007/978-3-540-88682-2_41
[26] Tordal, S.S. and Hovland, G. (2019). Ship-to-Ship State Observer Using Sensor Fusion and the Extended Kalman Filter, Journal of Offshore Mechanics and Arctic Engineering. 141(4). doi:10.1115/1.4041643
[27] Tysse, G.O., Cibicik, A., and Egeland, O. (2021). Vision-based control of a knuckle boom crane with online cable length estimation, IEEE Transactions on Mechatronics. 26(1):416--426. doi:10.1109/TMECH.2020.3024637
[28] Umeyama, S. (1991). Least-squares estimation of transformation parameters between two point patterns, IEEE Trans. Pattern Analysis and Machine Intelligence. 13(4):376--380. doi:10.1109/34.88573
[29] Xu, C., Zhang, L., Cheng, L., and Koch, R. (2017). Pose estimation from line correspondences: A complete analysis and a series of solutions, IEEE Transactions on Pattern Analysis and Machine Intelligence. 39(6):1209--1222. doi:10.1109/TPAMI.2016.2582162
[30] Zitnick, C.L. and Dollar, P. (2014). Edge boxes: Locating object proposals from edges, In European conference on computer vision. Springer, pages 391--405. doi:10.1007/978-3-319-10602-1_26
[31] Zuliani, M., Kenney, C., and Manjunath, B. (2005). The multiransac algorithm and its application to detect planar homographies, In IEEE International Conference on Image Processing 2005, volume3. pages III--153. doi:10.1109/ICIP.2005.1530351
BibTeX:
@article{MIC-2021-3-1,
title={{Estimation of Ship-Deck Motion using LIDAR,Gyroscopes and Cameras}},
author={Holen, Hans K.R. and Sjøberg, Alexander M. and Egeland, Olav},
journal={Modeling, Identification and Control},
volume={42},
number={3},
pages={99--112},
year={2021},
doi={10.4173/mic.2021.3.1},
publisher={Norwegian Society of Automatic Control}
};