Human activity recognition using smartphone has been attracting great interest. Since collecting large amount of labeled data is expensive and time-consuming for conventional machine learning techniques, transfer learning techniques have been proposed for activity recognition. However, existing transfer learning techniques typically rely on feature matching based on global domain shift and lack considering the intra-class knowledge transfer. In this paper, a novel transfer learning technique is proposed for cross-domain activity recognition, which can properly integrate feature matching and instance reweighting across the source and target domain in principled dimensionality reduction. The experiments using three real datasets demonstrate that the proposed scheme can achieve much higher precision (92%), recall (91%), and F1-score (92%), in comparison with the existing schemes.