Main content area

Weighting Formulas for the Least-Squares Analysis of Binding Phenomena Data

Tellinghuisen, Joel, Bolster, Carl H.
Journal of physical chemistry. B 2009 v.113 no.17 pp. 6151
least squares, binding properties, statistical models, equations, sorption, fluorescence, enzyme kinetics, Monte Carlo method, simulation models
The rectangular hyperbola, y ) abx/(1 + bx), is widely used as a fit model in the analysis of data obtained in studies of complexation, sorption, fluorescence quenching, and enzyme kinetics. Frequently, the “independent variable” x is actually a directly measured quantity, and y may be a simply computed function of x, like y ) x0 - x. These circumstances violate one of the fundamental tenets of most least-squares methodssthat the independent variable be error-freesand they lead to fully correlated error in x and y. Using an effective variance approach, we treat this problem to derive weighting formulas for the least-squares analysis of such data by the given equation and by all of its common linearized versions: the double reciprocal, y-reciprocal, and x-reciprocal forms. We verify the correctness of these expressions by computing the nonlinear leastsquares parameter standard errors for exactly fitting data, and we confirm their utility through Monte Carlo simulations. The latter confirm a problem with inversion methods when the inverted data are moderately uncertain (approximately 30%), leading to the recommendation that the reciprocal methods not be used for such data. For benchmark tests, results are presented for specific data sets having error in x alone and in both x and x0. The actual estimates of a and b and their standard errors vary somewhat with the choice of fit model, with one important exception: the Deming-Lybanon algorithm treats multiple uncertain variables equivalently and returns a single set of parameters and standard errors independent of the manner in which the fit model is expressed.