An investigation of the robustness of the nonlinear least-squares sphere fitting method to small segment angle surfaces

This paper presents an investigation of the nonlinear least-squares sphere fitting algorithm (TLS_A). The work concentrates on investigating the reliability of the TLS_A algorithm when applied to a small segment angle of a sphere. The definition of small segment angle is discussed in the paper and taken to be below 1° (in both x and y directions) of the spherical surface. This application of the TLS_A method is important when it is used on data from optical scanning systems where the measurements are limited by the gauge range and the angular tolerance of the sensor. The TLS_A algorithm has been first compared with the TLS_D algorithm suggested by Forbes for this application. The results show that the TLS_A algorithm can be used in small surface segment angles. The main study is focused on testing the algorithm on a sphere superimposed with surface irregularities (sensor/measurement noise or roughness). Two properties of the TLS_A algorithm are covered: the bias and the uncertainty of the estimated radius. Both simulation and theoretical approaches have been attempted. A new algorithm to estimate the bias of the TLS_A algorithm has been derived in this paper based on Box's method. Together with uncertainty estimation, which can be produced by using either a conventional method or Zhang's error propagation function (EPF), a comprehensive understanding of the TLS_A algorithm in this application is thus achieved, and used to develop a number of recommendations for the precision metrology of spherical and near spherical surfaces.