In the famous feature contrast model (FCM), the similarity measure is a linear combination of the common (similar) features and the distinctive (dissimilar) features. Because of the combination, FCM is better than other similarity models in explaining human perception similarity. However, the feature of FCM is binary. By defining the fuzzy feature set, FCM is extended into fuzzy feature contrast model (FFCM). In this paper, we adapt FFCM to measure spectral similarity. A spectrum is represented as a set including two subsets. The two subsets are characterized by spectral reflectance and spectral absorption, respectively. Meanwhile, the spectral reflectance and absorption are defined as the common (similar) and distinctive (dissimilar) subset in spectral set, respectively. Our spectral similarity model is expressed as a linear combination of the common subset, distinctive subset and their interaction. The difference between our model and FFCM is interaction of two subsets is defined. Moreover, kernel principal component analysis (KPCA) is used to remove the high correlation among different bands before spectral similarity measure. Experiments show that our model is effective in spectral similarity measure.