Nonnegative Matrix Factorization with Toeplitz Penalty

Authors

  • Matthew Corsetti Department of Biostatistics and Computational Biology, University of Rochester, Rochester, NY 14627
  • Ernest Fokoué School of Mathematical Sciences, Rochester Institute of Technology, Rochester, NY 14623

DOI:

https://doi.org/10.26713/jims.v10i1-2.851

Keywords:

Nonnegative matrix factorization, Auxiliary constraints, Toeplitz matrix, Zellner g-Prior, Image processing, Facial recognition, Subspace methods

Abstract

Nonnegative Matrix Factorization (NMF) is an unsupervised learning algorithm that produces a linear, parts-based approximation of a data matrix. NMF constructs a nonnegative low rank basis matrix and a nonnegative low rank matrix of weights which, when multiplied together, approximate the data matrix of interest using some cost function. The NMF algorithm can be modified to include auxiliary constraints which impose task-specific penalties or restrictions on the cost function of the matrix factorization. In this paper we propose a new NMF algorithm that makes use of non-datadependent auxiliary constraints which incorporate a Toeplitz matrix into the multiplicative updating of the basis and weight matrices. We compare the facial recognition performance of our new Toeplitz Nonnegative Matrix Factorization (TNMF) algorithm to the performance of the Zellner Nonnegative Matrix Factorization (ZNMF) algorithm which makes use of data-dependent auxiliary constraints. We also compare the facial recognition performance of the two aforementioned algorithms with the performance of several preexisting constrained NMF algorithms that have non-data-dependent penalties. The facial recognition performances are evaluated using the Cambridge ORL Database of Faces and the Yale Database of Faces.

Downloads

Download data is not yet available.

References

C. Boutsidis and E. Gallopoulos, Svd based initialization: A head start for nonnegative matrix factorization, Pattern Recognition 41(4) (2008), 1350 – 1362.

Z. Chen and A. Cichocki, Nonnegative matrix factorization with temporal smoothness and/or spatial decorrelation constraints, in Laboratory for Advanced Brain Signal Processing, RIKEN, Tech. Rep (2005).

A. Cichocki, R. Zdunek and S.-I. Amari, Csiszár's divergences for non-negative matrix factorization: Family of new algorithms, in Proceedings of the 6th International Conference on Independent Component Analysis and Blind Signal Separation, ICA'06, pp. 32–39 , Berlin, Heidelberg. Springer-Verlag (2006).

M.A. Corsetti and E.P. Fokoué, Nonnegative matrix factorization with zellner penalty, Open Journal of Statistics 5(7) (2015), 777 – 786.

E.P. Fokoué, D. Sun and P. Goel, Fully bayesian analysis of the relevance vector machine with consistency inducing priors, Technical report, Rochester Institute of Technology, 1 Lomb Memorial Dr, Rochester, NY 14623 (2009).

E. Gonzalez and Y. Zhang, Accelerating the Lee-Seung algorithm for non-negative matrix factorization, in Tech. Rep. TR-05-02, Dept. Comput. & Appl. Math., Rice Univ., Houston, TX, 1 – 13 (2005).

A.B. Hamza and D.J. Brady, Reconstruction of reflectance spectra using robust nonnegative matrix factorization, IEEE Transactions on Signal Processing 54(9) (2006), 3637 – 3642.

P.O. Hoyer, Non-negative sparse coding, in Neural Networks for Signal Processing XII (Proc. IEEE Workshop on Neural Networks for Signal Processing, pp. 557–565 (2002).

P.O. Hoyer, Non-negative matrix factorization with sparseness constraints, Journal of Machine Learning Research 5 (2004), 1457 – 1469.

D.D. Lee and H.S. Seung, Algorithms for non-negative matrix factorization, in Proceedings of the 13th International Conference on Neural Information Processing Systems, NIPS'00, pp. 535–541, Cambridge, MA, USA. MIT Press (2000).

C.-J. Lin, Projected gradient methods for nonnegative matrix factorization, Neural Comput. 19(10) (2007), 2756 – 2779.

P. Paatero and u. Tapper, Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics 5(2) (1994), 111 – 126.

V. Pauca, J. Piper and R. Plemmons, Nonnegative matrix factorization for spectral data analysis, in Linear Algebra and its Applications, Vol. 416 (2005), 29 – 47.

J. Piper, V.P. Pauca, R.J. Plemmons and M. Giffin, Object characterization from spectral data using nonnegative factorization and information theory, in Proceedings of the 2004 AMOS Technical Conference (2004).

R. Sandler and M. Lindenbaum, Nonnegative matrix factorization with earth mover's distance metric for image analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 33(8) (2011), 1590 – 1602.

H.S. Seung and D.D. Lee, Learning the parts of objects by non-negative matrix factorization, Nature 401(6755) (1999), 788 – 791.

S. Sra and I.S. Dhillon, Generalized nonnegative matrix approximations with Bregman divergences, in Y. Weiss, B. Schölkopf and J.C. Platt (editors), Advances in Neural Information Processing Systems 18, pp. 283 – 290, MIT Press (2006).

M.E. Tipping, Sparse bayesian learning and the relevance vector machine, Journal of Machine Learning Research 1 (2001), 211 – 244.

Y. Wang, Y. Jia, C. Hu and M. Turk, Fisher non-negative matrix factorization for learning local features, in Proc. Asian Conf. on Comp. Vision, pp. 27 – 30 (2004).

S. Wild, W.S. Wild, J. Curry, A. Dougherty and M. Betterton, Seeding nonnegative matrix factorization with the spherical k-means clustering, Technical Report, University of Colorado (2002).

S.M. Wild, J.H. Curry and A. Dougherty, Motivating non-negative matrix factorizations, in Proceedings of the Eighth SIAM Conference on Applied Linear Algebra, SIAM (2003).

Y. Xue, C.S. Tong, W.-S. Chen andW. Zhang, A modified non-negative matrix factorization algorithm for face recognition, in 18th International Conference on Pattern Recognition (ICPR'06), Vol. 3, pp. 495 – 498 (2006).

R. Zdunek and A. Cichocki, Non-negative matrix factorization with quasi-Newton optimization, in Eighth International Conference on Artificial Intelligence and Soft Computing (ICAISC), pp. 870 – 879, Springer (2006).

A. Zellner, On assessing prior distributions and Bayesian regression analysis with g-prior distributions, in Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, pp. 233–243, Elsevier Science Publishers (1986).

Downloads

Published

2018-08-09
CITATION

How to Cite

Corsetti, M., & Fokoué, E. (2018). Nonnegative Matrix Factorization with Toeplitz Penalty. Journal of Informatics and Mathematical Sciences, 10(1-2), 201–215. https://doi.org/10.26713/jims.v10i1-2.851

Issue

Vol. 10 No. 1-2 (2018)

Section

Research Articles

License

Authors who publish with this journal agree to the following terms:
  • Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a CCAL that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
  • Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
  • Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.