A method is proposed for removing the background by color difference of two images with pests and non-pests. 156 features including color, shape and texture features of each pest are extracted into an support vector machine (SVM) classifier with radial basis kernel function.