Estimating accuracy of land-cover composition from two-stage cluster sampling

Land-cover maps are often used to compute land-cover composition (i.e., the proportion or percent of area covered by each class), for each unit in a spatial partition of the region mapped. We derive design-based estimators of mean deviation (MD), mean absolute deviation (MAD), root mean square error (RMSE), and correlation (CORR) to quantify accuracy of land-cover composition for a general two-stage cluster sampling design, and for the special case of simple random sampling without replacement (SRSWOR) at each stage. The bias of the estimators for the two-stage SRSWOR design is evaluated via a simulation study. The estimators of RMSE and CORR have small bias except when sample size is small and the land-cover class is rare. The estimator of MAD is biased for both rare and common land-cover classes except when sample size is large. A general recommendation is that rare land-cover classes require large sample sizes to ensure that the accuracy estimators have small bias.     

A priori evaluation of two-stage cluster sampling for accuracy assessment of large-area land-cover maps

Two-stage cluster sampling reduces the cost of collecting accuracy assessment reference data by constraining sample elements to fall within a limited number of geographic domains (clusters). However, because classification error is typically positively spatially correlated, within-cluster correlation may reduce the precision of the accuracy estimates. The detailed population information to quantify a priori the effect of within-cluster correlation on precision is typically unavailable. Consequently, a convenient, practical approach to evaluate the likely performance of a two-stage cluster sample is needed. We describe such an a priori evaluation protocol focusing on the spatial distribution of the sample by land-cover class across different cluster sizes and costs of different sampling options, including options not imposing clustering. This protocol also assesses the two-stage design's adequacy for estimating the precision of accuracy estimates for rare land-cover classes. We illustrate the approach using two large-area, regional accuracy assessments from the National Land-Cover Data (NLCD), and describe how the a priori evaluation was used as a decision-making tool when implementing the NLCD design.     

Basic probability sampling designs for thematic map accuracy assessment

Choosing a sampling design for assessing thematic map accuracy requires the strength of a sampling design to be matched to the objectives and resources available for the accuracy assessment. The criteria to consider when planning the sampling design are that the sample should: (1) satisfy probability sampling protocol; (2) be simple to implement and analyse; (3) result in low variance for the key estimates of the assessment; (4) permit adequate variance estimation; (5) be spatially well distributed; and (6) be cost effective. Several basic probability sampling designs useful for accuracy assessment are reviewed, and recommendations are provided to guide the selection of an appropriate design.     

An empirical assessment of ranking accuracy in ranked set sampling

Ranked set sampling (RSS) involves ranking of potential sampling units on the variable of interest using judgment or an auxiliary variable to aid in sample selection. Its effectiveness depends on the success in this ranking. We provide an empirical assessment of RSS ranking accuracy in estimation of a population proportion.     

On tests of rate ratio under standard inverse sampling

Inverse sampling suggests one continues to sample subjects until a pre-specified number of rare events of interest is observed. It is generally considered to be more appropriate than the usual binomial sampling when the subjects come sequentially, when the response probability is rare, and when maximum likelihood estimators of some epidemiological measures are undefined under binomial sampling. Reliable but conservative exact conditional procedure for the ratio of the response probabilities of subject without the attribute of interest has been studied. However, such a procedure is inapplicable to the risk ratio (i.e., ratio of the response probabilities of subject with the attribute of interest). In this paper, we investigate various test statistics (namely Wald-type, score and likelihood ratio test statistics) for testing non-unity risk ratio under standard inverse sampling scheme, which suggests one continue to sample until the predetermined number of index subjects with the attributes of interest is observed. Both asymptotic and numerical approximate unconditional methods are considered for P-value calculation. Performance of these test procedures are evaluated under different settings by means of Monte Carlo simulation. In general, the Wald-type test statistic is preferable for its satisfactory and stable performance with approximate unconditional procedures. The methodologies are illustrated with a real example from a heart disease study.     

Thematic map accuracy assessment from the perspective of finite population sampling

Accuracy assessment of land-use and land-cover classifications obtained from remotely-sensed data is recognized as a critical step in evaluating thematic maps. In this Letter, thematic accuracy assessment is formulated in terms of a finite population sampling problem. Several examples are given to illustrate how this sampling perspective is applied to problems in accuracy assessment. Finite population sampling methods have the flexibility to provide estimators and variance estimators for a wide variety of sampling designs and parameters of interest in accuracy assessment.     

Estimating probabilities under the three-parameter gamma distribution using composite sampling

Composite sampling may be used in industrial or environmental settings for the purpose of quality monitoring and regulation, particularly if the cost of testing samples is high relative to the cost of collecting samples. In such settings, it is often of interest to estimate the proportion of individual sampling units in the population that are above or below a given threshold value, C. We consider estimation of a proportion of the form p=P(X>C) from composite sample data, assuming that X follows a three-parameter gamma distribution. The gamma distribution is useful for modeling skewed data, which arise in many applications, and adding a shift parameter to the usual two-parameter gamma distribution also allows the analyst to model a minimum or baseline level of the response. We propose an estimator of p that is based on maximum likelihood estimates of the parameters α, β, and γ, and an associated variance estimator based on the observed information matrix. Theoretical properties of the estimator are briefly discussed, and simulation results are given to assess the performance of the estimator. We illustrate the proposed estimator using an example of composite sample data from the meat products industry.     

Estimating quantiles under sampling on two occasions with arbitrary sample designs

A practical problem related to the estimation of quantiles in double sampling with arbitrary sampling designs in each of the two phases is investigated. In practice, this scheme is commonly used for official surveys, in which quantile estimation is often required when the investigation deals with variables such as income or expenditure. A class of estimators for quantiles is proposed and some important properties, such as asymptotic unbiasedness and asymptotic variance, are established. The optimal estimator, in the sense of minimizing the asymptotic variance, is also presented. The proposed class contains several known types of estimators, such as ratio and regression estimators, which are of practical application and are therefore derived. Assuming several populations, the proposed estimators are compared with the direct estimator via an empirical study. Results show that a gain in efficiency can be obtained.     

Estimating quantiles under sampling on two occasions with arbitrary sample designs

A practical problem related to the estimation of quantiles in double sampling with arbitrary sampling designs in each of the two phases is investigated. In practice, this scheme is commonly used for official surveys, in which quantile estimation is often required when the investigation deals with variables such as income or expenditure. A class of estimators for quantiles is proposed and some important properties, such as asymptotic unbiasedness and asymptotic variance, are established. The optimal estimator, in the sense of minimizing the asymptotic variance, is also presented. The proposed class contains several known types of estimators, such as ratio and regression estimators, which are of practical application and are therefore derived. Assuming several populations, the proposed estimators are compared with the direct estimator via an empirical study. Results show that a gain in efficiency can be obtained.     

Estimating errors of certain Gauss quadrature formulae

Using Cauchy's integral formula (as in [8]) we obtain the closed form error-estimates for the general Gauss-Chebyshev quadrature formulas. Some of the estimates of the errors of these quaratdures are excellent in the sense of Chawla [3].     

Estimating area and map accuracy for stratified random sampling when the strata are different from the map classes

The results of an accuracy assessment are typically organized using an error matrix that displays the proportion of area correctly mapped for each class and the proportion of area misclassified. Stratified random sampling is commonly implemented to obtain the reference data used to estimate the error matrix. When the strata correspond exactly to the map classes, the formulas for estimating accuracy and area are well known. Nevertheless, applications arise in which the strata are different from the map classes, as for example when the stratification is based on the map class labels of one map but the sample is subsequently used to assess the accuracy of other maps. In this paper, the estimators required when the stratum label and map label do not match for all pixels are presented for the proportion of area of each class based on the reference classification and for overall, user’s, and producer’s accuracies. Standard error formulas are also presented. A numerical example is provided to illustrate the computations.     

Evaluation of modeling refinements on work sampling statistics

A spreadsheet program is presented as a design tool for work sampling studies. The program incorporates the alternating Poisson process (APP) model for process fluctuation and allows for evaluation of process parameters with, or without, a finite sample size correction.     

Estimating errors of numerical approximation for periodic analytic functions

We give a simple method based on Cauchy's integral formula for estimating the errors of numerical approximation for periodic analytic functions. We then obtain error estimates for the quadrature formulas of Chawla and Ramakrishnan [1] for the numerical evaluation of the Cauchy principal value integral $$I\left( {f;a} \right) = \int\limits_0^{2\pi } {f(x)\cot \left( {\left( {x - a} \right)/2} \right)dx,} $$ and for the quadrature formula of Garrick [2] for the evaluation ofI (f;o), Based on these error estimates, we are led to conclude that for the evaluation ofI (f;o), Garrick's formula has a better error estimate than the formula of Chawla and Ramakrishnan with the same number of function evaluations. Finally, we extend Garrick's formula for the evaluation ofI (f;a) for arbitrarya∈[0,2π); the extended formula has, for alla, the same error estimate as Garrick's formula. While, for a=x_j, the extended formula is identical with the quadrature formula of Wittich [3], fora≠x _j, the extended formula is much better in that it uses only half the number of function evaluations of Wittich's formula for the same accuracy.     

Cluster-wise assessment of cluster stability

Stability in cluster analysis is strongly dependent on the data set, especially on how well separated and how homogeneous the clusters are. In the same clustering, some clusters may be very stable and others may be extremely unstable. The Jaccard coefficient, a similarity measure between sets, is used as a cluster-wise measure of cluster stability, which is assessed by the bootstrap distribution of the Jaccard coefficient for every single cluster of a clustering compared to the most similar cluster in the bootstrapped data sets. This can be applied to very general cluster analysis methods. Some alternative resampling methods are investigated as well, namely subsetting, jittering the data points and replacing some data points by artificial noise points. The different methods are compared by means of a simulation study. A data example illustrates the use of the cluster-wise stability assessment to distinguish between meaningful stable and spurious clusters, but it is also shown that clusters are sometimes only stable because of the inflexibility of certain clustering methods.     

Effect of errors in ground truth on classification accuracy

The effect of errors in ground truth on the estimated thematic accuracy of a classifier is considered. A relationship is derived between the true accuracy of a classifier relative to ground truth without errors, the actual accuracy of the ground truth used, and the measured accuracy of the classifier as a function of the number of classes. We show that if the accuracy of the ground truth is known or can be estimated, the true accuracy of a classifier can be estimated from the measured accuracy. In a series of simulations our method is shown to produce unbiased estimates of the true accuracy of the classifier with an uncertainty that depends on the number of samples and the accuracy of the ground truth. A method for determining the relative performance of two or more classifiers over the same area is then discussed. The results indicate that, as the number of samples increases, the performance of the classifiers can be effectively differentiated using inaccurate ground truth. It is argued that relative accuracies computed using a large number of inaccurate ground truth points are more representative of the true relative performance of the classifiers as they are being evaluated over a larger portion of the scene. An example is presented that uses this method to evaluate the relative performance of two Landsat classifiers.     

Impact of inspection errors on the formulation of a multi-objective optimization process targeting model under inspection sampling plan

The literature on inspection systems has demonstrated that such systems are error prone. Such systems exhibit type I and type II errors. Type I error is classifying a conforming item as non-conforming while type II error is classifying a non-conforming item as conforming. The errors may arise from the measurement system or the inspectors. It is essential to assess the impact of the inspection errors on the optimal parameters and objective functions of process targeting models. The purpose of this paper is to assess the impact of the inspection errors on the optimal parameters and objectives functions values of Duffuaa and El-Ga’aly multi-objectives optimization model recently developed for process targeting (2013a). In order to accomplish the objectives of the paper, Duffuaa and El-Ga’aly multi-objective optimization model is extended by introducing the measurement errors in the inspection system and penalties to mitigate the effect of the errors. The results of the extended model is compared with the previous model and employed to studying the impact of the errors on the values of objective function and the optimal process parameters in a multi-objectives environment. The results indicated that inspection errors have significant impact on the profit and uniformity objectives.     

Estimating accuracy in optimal deconvolution of synthetic AMSR-E observations

Optimal deconvolution (ODC) utilizes the footprint overlap in microwave observations to estimate the earth's brightness temperatures (T_B). This paper examines the accuracy of ODC-estimated T_B compared with a standard averaging technique. Because brightness temperatures cannot be independently verified, we constructed synthetic True T_B for accuracy assessment. We assigned T_B at a high spatial resolution (1 km) grid and computed the True T_B by spatial averaging of the assigned T_B to a lower resolution earth grid (25 km), selected to match the resolution of products generated from the Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E). We used the sensor antenna response function along with the 1-km assigned T_B to generate synthetic observations at AMSR-E footprint locations. These synthetic observations were subsequently deconvolved in the ODC technique to estimate T_B at the lower resolution earth grid. The ODC-estimated T_B and the simple grid cell averages of the synthetic observations were compared with the True T_B allowing us to quantify the efficacy of each technique. In areas of high T_B contrast (such as boundaries of water bodies), ODC performed significantly better than averaging. In other areas, ODC and averaging techniques produced similar results. A technique similar to ODC can be effective in delineating water bodies with significant clarity. That will allow microwave observations to be utilized near the shorelines, a trouble spot for the currently used averaging techniques.     

机械手软抓取的采样信号控制精度研究

在机械手的设计中,下位机硬件电路的设计对采样信号的控制精度起着至关重要的作用。为此,在满足采样定理的条件下,针对信号采样频率对采样信号的精度影响,对带有噪声信号的正弦信号在不同采样频率下恢复信号与原信号的误差、不同强度的噪声在不同采样频率下恢复信号与原信号的误差、不同初相位的不同采样频率下的恢复信号与原信号的误差情况进行了MATLAB仿真分析。该实验结果为硬件电路的设计奠定了坚实的基础。     

Estimating friction errors in MOC analyses of unsteady pipe flows

Errors arising from the numerical treatment of friction in unsteady flows in small pipe networks are assessed for fixed-grid, method of characteristics analyses (MOC) with no interpolation. Although the results of the study are targeted at general unsteady flows, the underpinning analytical development is based on the behaviour of standing waves. This enables quantitative conclusions to be drawn about the sensitivity of MOC solutions to frequency. The development is undertaken first for a single pipe and then for networks, with and without a loop. The performance of MOC solutions is modelled with the aid of polynomial transfer matrices that are analogous to transfer function matrices available directly from partial differential equations of water hammer. It is found that, in a single pipe, the numerical errors tend to increase damping at all frequencies, but that, in networks, either increased or decreased damping may occur. The errors place both higher and lower limits on the frequencies that can propagate along a pipe in a numerical analysis. This contrasts with the physical case where only a lower limit exists. The work presented is part of a wider project with the long-term aim of automating the selection of numerical grid sizes in MOC analyses of unsteady flows.     

Estimating smooth distribution function in the presence of heteroscedastic measurement errors

Measurement error occurs in many biomedical fields. The challenges arise when errors are heteroscedastic since we literally have only one observation for each error distribution. This paper concerns the estimation of smooth distribution function when data are contaminated with heteroscedastic errors. We study two types of methods to recover the unknown distribution function: a Fourier-type deconvolution method and a simulation extrapolation (SIMEX) method. The asymptotics of the two estimators are explored and the asymptotic pointwise confidence bands of the SIMEX estimator are obtained. The finite sample performances of the two estimators are evaluated through a simulation study. Finally, we illustrate the methods with medical rehabilitation data from a neuro-muscular electrical stimulation experiment.