Orthogonal nonnegative matrix tri-factorization for co-clustering: Multiplicative updates on Stiefel manifolds

Matrix factorization-based methods become popular in dyadic data analysis, where a fundamental problem, for example, is to perform document clustering or co-clustering words and documents given a term-document matrix. Nonnegative matrix tri-factorization (NMTF) emerges as a promising tool for co-clustering, seeking a 3-factor decomposition X≈USV^?$\mathit{X}\approx {\mathit{USV}}^{?}$ with all factor matrices restricted to be nonnegative, i.e., U?0,S?0,V?0.$\mathit{U}?0\text{,}\mathit{S}?0\text{,}\mathit{V}?0\text{.}$ In this paper we develop multiplicative updates for orthogonal NMTF where X≈USV^?$\mathit{X}\approx {\mathit{USV}}^{?}$ is pursued with orthogonality constraints, U^?U=I,${\mathit{U}}^{?}\mathit{U}=\mathit{I}\text{,}$ and V^?V=I${\mathit{V}}^{?}\mathit{V}=\mathit{I}$, exploiting true gradients on Stiefel manifolds. Experiments on various document data sets demonstrate that our method works well for document clustering and is useful in revealing polysemous words via co-clustering words and documents.