本論文針對手眼型(hand-eye)視覺機器臂系統探討其深度估測問題。
相對於較早的研究，本論文完整解讀深度的可觀測性問題。
在視覺系統中，特徵點未知深度必須從特徵點影像輸出和攝影機速度中估出來。若令手眼型機器臂上的攝影機速度為系統輸入，則深度估測的可觀測性和攝影機速度相關。我們發現一個充分必要條件，只要攝影機速度滿足特定型式，即可保證深度為可觀測，模擬和實驗結果亦驗證理論。
進一步我們考慮視覺伺服控制上的應用，我們提出一個改良後的 LQ 視覺伺服控制法，藉著改變其加權矩陣，使得深度估測特性得以改善，
然而其控制特性仍可以保持。我們亦延伸線性系統中的估測結果，
提出一個可以評估深度估測性能的準則，這個性能準則只和影像位置及攝影機線性速度有關，並由許多模擬和實驗驗證了所提準則。最後，這個準則被用來發展一套新的視覺伺服控制法，該控制法在深度估測和視覺控制上可同時展現良好的特性，模擬的例子亦驗證了此控制法。

This dissertation deals with the depth estimation problem of a hand-eye robot system. In contrast to the earlier works, this dissertation gives a complete study of such a depth observability problem. In the visual system, the unknown depth of a feature point is estimated from the input of the camera velocity and the output of the image of the feature point. The velocity of the active camera mounted on the end-effector of the robot is considered as the input. The observability of the depth estimation is then related to the velocity of the camera. A necessary and sufficient condition for the type of camera velocities to ensure the observability is found. The theory is also verified by both simulations and experiments.
Furthermore, we consider the application of our result to the visual servo control. A modified linear quadratic (LQ) visual servo control law is proposed to vary the weighting matrices so that the depth estimation is improved while the control performance is still retained. We also proposes a criterion to measure the performance of the depth estimation, which is a heuristic extension from an estimation result of a linear system. The proposed performance criterion is dependent on both the image position and the linear velocity of the camera.
Some simulation and experiment examples demonstrate and verify this performance criterion. Finally, this criterion is used to develop a new visual servo control scheme that has good performance in both the depth estimation and the visual control. The control scheme is also verified by a simulation example.

1. Introduction
2. Depth Estimation
3. Observability of the Depth Estimation
4. Modified LQ Visual Servo Control
5. Performance Index for Depth Estimation
6. Application to Visual Servo Control
7. Conclusion

[1] Allen, P. K., Timcenko, A., Yoshimi, B., and Michelman, P., ``Automated tracking and grasping of a moving object with a robotic hand-eye system'', IEEE Trans. Robotics Automat., vol. 9, no. 2, pp. 152--165, 1993.
[2] Arun, K. S., Huang, T. S., and Blostein, S. D., ``Least-squares fitting of two 3-D point sets'', IEEE Trans. on Pattern Anal. Machine Intell., vol. PAMI--9, no. 5, pp. 698--700, 1987.
[3] Chaumette, F. and Marchand, E., ``A redundancy-based iterative approach for avoiding joint limits: application to visual servoing'', IEEE Trans. Robotics Automat., vol. 17, no. 5, pp. 719--730, 2001.
[4] Chen, H. H. and Huang, T. S., ``Maximal matching of 3-D points for multiple-object motion estimation", Pattern Recognition, vol. 21, no. 2, pp. 75--90, 1988.
[5] Cho, J. S., Kim, H. W., and Kweon, I. S., ``Image-based visual servoing using position and angle of image features'', Electronics Letters, vol. 37, no. 13, pp. 862--864, 2001.
[6] Chui, C. K. and Chen, G., Kalman Filtering with Real-Time Applications, New York, Spring-Verlag, 1991.
[7] Corke, P. I. and Hutchinson, S. A., ``A new partitioned approach to image-based visual servo control'', IEEE Trans. Robotics Automat., vol. 17, no. 4, pp. 507--515, 2001.
[8] Corke, P. I. and Paul, R. P., Video-Rate Visual Servoing for Robots, Technical Report MS-CIS-89-18, GRASP Lab, University of Pennsylvania, 1989.
[9] Dayawansa, W. P., Ghosh, B. K., Martin, C., and Wang, X., ``A necessary and sufficient condition for the perspective observability problem'', System & Control Letters, vol. 25, pp. 159--166, 1995.
[10] Faugeras, O. D. and Toscani, G., ``The calibration problem for stereo'', in Conf. on Computer vision and Pattern Recognition, pp. 15--19, 1986.
[11] Feddema, J. T. and Mitchell, O. R., ``Vision-guided servoing with feature-based trajectory generation'', IEEE Trans. Robotics Automat., vol. 5, no. 5, pp. 691--700, 1989.
[12] Feddema, J. T., Lee, C. S. G., and Mitchell, O.R., ``Weighted selection of image features for resolved rate visual feedback control'', IEEE Trans. Robotics Automat., vol. 7, no. 1, pp. 31--47, 1991.
[13] Friedland, B., Advanced Control System Design, New Jersey, Prentice-Hall, 1996.
[14] Gros, P., ``Matching and Clustering: two steps toward automatic object modeling in computer vision'', Int. J. Robot. Res., vol. 14, no. 6, pp. 633--642, 1995.
[15] Hager, G. D., ``A modular system for robust positioning using feedback from stereo vision'', IEEE Trans. Robotics Automat., vol. 13, no. 4, pp. 582--595, 1997.
[16] Haralick, R. M. and Shapiro, L. G., Computer and Robot Vision II, USA, Addison-Wesley, Inc., 1993.
[17] Hashimoto, K., Ebine, T., and Kimura, H., ``Visual servoing with hand-eye manipulator -- optimal control
approach'', IEEE Trans. Robotics Automat., vol. 12, no. 5, pp. 766--773, 1996.
[18] Hespanha, J., Dodds, Z., Hager, G. D., and Morse, A. S., ``Decidability of robot positioning tasks using stereo vision systems'', in proc. 37th IEEE int. Conf. on Decision and Control, pp. 3736--3741, 1998.
[19] Huang, T. S., and Netravali, A. N., ``Motion and structure from feature correspondences: a review'', Porceedings of the IEEE, vol. 82, no. 2, pp. 251--267, 1994.
[20] Jang, W. and Bein, Z., ``Feature-based visual servoing of an eye-in-hand robot with improved tracking performance'', in proc. IEEE int. Conf. Robotics Automat., pp. 2254--2260, 1991.
[21] Khalil, H. K., Nonlinear Systems, New Jersey, Prentice-Hall, 1996.
[22] Koivo, A. J. and Houshangi, N., ``Real-time vision feedback for servoing robotic manipulator with self-tuning controller'', IEEE Trans. Syst. Man Cybern., vol. 21, no. 1, pp. 134--141, 1991.
[23] Lin, S. K., An Alternative Approach to the Camera Self-Calibration of an Active Vision System, Report of National Science Council Project: NSC89-2213-E-009-130, 2000.
[24] Lin, S. K., Image-Feature Based Tracking Control, Report of National Science Council Project: NSC85-2213-E-009-099, 1996.
[25] Matthies, L. and Kanade, T., ``Kalman filter-based algorithms for estimating depth from image sequences'', Int. J. Computer Vision, vol. 3, pp.
209--236, 1989.
[26] Maybank, S., Theory of Reconstruction from Image Motion, New York, Springer-Verlag, 1993.
[27] Nelson, B. and Khosla, P. K., ``Integrating sensor placement and visual tracking strategies'', in proc. IEEE int. Conf. Robotics Automat., pp.1351--1356, 1994.
[28] Nijmeijer, H. and Schaft, A. J. van der, Nonlinear Dynamical Control Systems, New York, Spring-Verlag, 1990.
[29] Papanikolopoulos, N. P., Khosla, P. K., and Kanade, T., ``Adaptive robotic visual tracking: theory and experiments'', IEEE Trans. Automat. Contr., vol. 38, no. 3, pp. 429--445, 1993.
[30] Papanikolopoulos, N. P., Khosla, P. K., and Kanade, T., ``Visual tracking of a moving target by a camera mounted on a robot: a combination of control and vision'', IEEE Trans. Robotics Automat., vol. 9, no. 1, pp. 14--34,1993.
[31] Rizzi, A. A. and Koditschek, D. E., ``An active visual estimator for dexterous manipulation'', IEEE Trans. Robotics Automat., vol. 12, no. 5, pp. 697--713, 1996.
[32] oderstrom, T. and Stoica, P., System Identification, New York, Prentice-Hall, 1989.
[33] Sharma, R. and Hutchinson, S., ``Motion perceptibility and its application to active vision-based servo control'', IEEE Trans. Robotics Automat., vol. 13, no. 4, pp. 607--617, 1997.
[34] Sharma, R. and Hutchinson, S., ``On the observability of robot motion under active camera control'', in proc. IEEE int. Conf. Robotics Automat., pp. 162--167, 1994.
[35] Sharma, R. and Hutchinson, S., ``Optimizing hand/eye configuration for visual-servo systems'', in proc. IEEE int. Conf. Robotics Automat., pp. 172--177, 1995.
[36] Shekhar, C. and Chellappa, R., ``Passive ranging using a moving camera'', Journal of Robotic Systems, vol. 9, no. 6, pp. 729--752, 1992.
[37] Shih, S. W., Hung, Y. P., and Lin, W. S., ``Accurate linear technique for camera calibration considering lens distortion by solving an eigenvalue problem'', Optical Engineering, vol. 32, no. 1, pp. 138--149, 1993.
[38] Smith, C. E., Brandt, S. A., and Papanikolopoulos, N. P., ``Eye-in-hand robotic tasks in uncalibrated environments'', IEEE Trans. Robotics Automat., vol. 13, no. 6, pp. 903--914, 1997.
[39] Soatto, S., ``3-D structure from visual motion: modeling, representation and observability'', Automatica, vol. 33, no. 7, pp. 1287--1312, 1997.
[40] Soatto, S., Frezza, R., and Perona, P., ``Motion estimation via dynamic vision'', IEEE Trans. Automat. Contr., vol. 41, no. 3, pp. 393--413,
1996.
[41] Sridhar, B., Suorsa, R., Smith, P., and Hussien, B., ``Vision-based obstacle detection for rotorcraft fight'', Journal of Robotic Systems, vol. 9, no. 6, pp. 709--727, 1992.
[42] Sutanto, H. and Sharma, R., ``Global performance evaluation of image features for visual servo control'', Journal of Robotic Systems, vol. 13, no. 4, pp. 243--258, 1996.
[43] Tsai, R. Y. and Huang, T. S., ``Uniqueness and estimation of three-dimensional motion parameters of rigid objects with curved surfaces'', IEEE Trans. on Pattern Anal. Machine Intell., vol. PAMI--6, no. 1, pp. 13--27, 1984.
[44] Vidyasagar, M., Nonlinear System Analysis, Englewood Cliffs, Prentice-Hall, 1993.
[45] Waxman, A. M. and Ullman, S., ``Surface structure and three-dimensional motion from image flow kinematics'', Int. J. Robot. Res., vol. 4, no. 3, pp. 72--94, 1985.
[46] Weiss, L. E., Sanderson, A. C., and Neuman, C. P., ``Dynamic sensor-based control of robots with visual feedback'', IEEE J. Robotics Automat., vol. RA--3, no. 5, pp. 404--417, 1987.
[47] Weng, J., Cohen, P., and Herniou, M., ``Camera calibration with distortion models and accuracy evaluation'', IEEE Trans. on Pattern Anal. Machine Intell., vol. PAMI--14, no. 10, pp. 965--980, 1992.
[48] Wilson, W. J., Hulls, C. C. W, and Bell, G. S., ``Relative end-effector control using cartesian position based visual servoing'', IEEE Trans. Robotics Automat., vol. 12, no. 5, pp. 684--696, 1996.