Target method based on structure sparsity
基于改进SSD的高效目标检测方法
基于改进SSD的高效目标检测方法WANG Wenguang;LI Qiang;LIN Maosong;HE Xianzhen【摘要】为改善一阶段目标检测算法检测精度较差的缺陷,提出一种基于SSD的高效多目标定位检测算法FSD.该算法主要从两个方面对一阶段目标检测算法进行改进:设计了一个更高效的密集残差网络,即R-DenseNet,通过采用一种更窄的密集网络结构形式,在保持特征提取容量的同时降低了计算复杂度,从而提高了算法的检测和收敛性能;改进了损失函数,通过抑制易分样本在损失函数中的权重,提高算法的鲁棒性,改善了目标检测中样本失衡的现象.采用Tensorflow深度学习框架部署网络,并在搭载Nvidia Titan X的Ubuntu上开展实验,实验表明FSD在COCO和PASCAL VOC这两个目标检测数据集上上都取得了最高的检测精度,其中FSD300D的检测精度相比SSD300有3.7%提升,检测相率比SSD有10.87%提升.%In order to improve the defect of poor detection accuracy of the one-stage object detection algorithm, an efficient multi-target location detection algorithm FSD based on SSD is proposed. The algorithm mainly improves the one-stage object detection algorithm from two aspects: on the one hand, it designs a more efficient dense residual network, namely R-DenseNet, by adopting a narrower dense network structure form to maintain feature extraction. The capacity reduces the computational complexity, which improves the detection and convergence performance of the algorithm. On the other hand, the loss function is improved. By suppressing the weight of the easily-divided samples in the loss function, the robustness of the algorithm is improved, and the phenomenon of sample imbalance in object detection is improved. The Tensorflow deeplearning framework is used to deploy the network, and experiments are carried out on Ubuntu equipped with Nvidia Titan X. Experiments show that FSD achieves the highest detection accuracy on both COCO and PASCAL VOC object detection data sets, among which FSD300 detection accuracy compared with the SSD300, there is a 3.7% improvement, and the detection phase rate is 10.87% higher than that of the SSD.【期刊名称】《计算机工程与应用》【年(卷),期】2019(055)013【总页数】8页(P28-35)【关键词】深度学习;目标检测;特征融合;样本失衡;卷积神经网(CNN)【作者】WANG Wenguang;LI Qiang;LIN Maosong;HE Xianzhen【作者单位】School of Information Engineering, Southwest University of Science and Technology, Mianyang ,Sichuan 621010, China;School of Information Engineering, Southwest University of Science and Technology, Mianyang ,Sichuan 621010, China;School of Information Engineering, Southwest University of Science and Technology, Mianyang ,Sichuan 621010, China;School of Information Engineering, Southwest University of Science and Technology, Mianyang ,Sichuan 621010, China【正文语种】中文【中图分类】TP391 引言随着深度卷积神经网络的提出和发展，目标检测算法取得了巨大的进步[1-2]。
谱稀疏ppt
Spectral Compressive Sensing
• Compressive sensing applied to frequency-sparse signals
=
frequency-sparse signal
linear measurements
nonzero DFT coefficients
1
µ( )
0.5 0 0 1
0.02
0.04
0.06
0.08
0.1
0.12
Structured Frequency-Sparse Signals
• If x is K-structured frequency-sparse, then there exists a K-sparse vector such that and the nonzeros in are spaced apart from each other.
Normalized approx. error
1 0.8 0.6 0.4 0.2 0 0 10 20 30 40 Approximation sparsity K 50 Integral frequencies Arbitrary frequencies
Signal is sum of 10 sinusoids
RN
RM
K
K
x1
x2
mK K-dim planes
[Blumensath, Davies; Lu, Do]
Leveraging Structure in Recovery
Many state-of-the-art sparse recovery algorithms (greedy and optimization solvers) rely on [Daubechies, Defrise, and DeMol; thresholding
国际会议英语演讲稿
国际会议英语演讲稿国际会议英语演讲稿篇一：国际会议作报告英语发言稿 Thank you, prof. …. My name is ….. I’m from ….. I am very pleased to be here to join this forum. The topic of my presentation is properties of rapid construction materials for soil pavement of field airfield. As is shon in the picture, the main parts of my research are about soil pavement.My presentation ill include these four parts:First, some background information about this research; second, the main ork e have done; third, some conclusions e have got and the last: innovation and presentation of our published papers.Why I choose this item? I think it can be illustrated from the folloing four parts. First, the existing quantity of airfields is still not sufficient and the airfields have many shortings especially in ar time. Second, the plementary facilities, such as highay runays are far less than airfields, hoever, have more eakness. Third, a certain amount of field airfield is quite necessary considering some emergencies such as rescue and disaster relief. Forth, the field airfield can fill the void of airfield and they can be bined to be airfield netork.The meaning and aim of this research contains three parts. Fast, convenient and validity, fast means the field airfield must be constructedas fast as possible, convenient means the construction should need the minimum equipment, labor and materials considering the actual construction condition, validity means the constructed airfield is able to support the operation of given aircraft in specifically time.Just like many other territories, the situation of the research is that the U.S. Army takes advanced line. The U.S. Army declares that they can reach to anyhere on the earth in 96 hours, the most important method for force projection is though aircraft, thus rapid construction of pavement is the key problem for rapid force transportation.The main ork e have done can be summarized as four parts, materials choosing, scheme making, mechanical properties research and ater-stable properties research.We choose to kinds of soils, hich are got from Xi’an, Shanxi province and Jiuquan, Gansu province separately. The sand from Ba River as considerate to investigate the influence of sand to the properties of stabilized soil. The chosen three kinds of poders are cement, lime and ne-type stabilizer developed by Chang’an University. The principles in considering the function of 4 kings of fibers are referring different length, type and mixing them.On account of the time, I ill make a brief description about the experiment scheme. In summary, three parts ere proposed to distinguish the affecting factors in making experiment scheme.They are poder control, fiber control and other factors. Taking poder control for example, the dosage of cement is respectively 6%, 8% and 10% hen the soil is stabilized only by cement, hile the dosage of cement decrease to 3%, 5% and 7% hen the lime is addicted to stabilized soil. The folloing to factors are stabilizer and sand.Six kinds of experiments ere performed to investigate the influence of above factors to the mechanical properties of stabilized soil. The aim of paction test is to find the maximum dry density and optimum moisture content. The aim of pression strength test is to determine the optimum dosage of cement, lime, poder stabilizer and fiber, meanhile evaluating the performance of stabilized soil. The aim of splitting tension strength test is similar to pression strength test, the left picture is sample stabilized by cement, hile the right picture is the sample stabilized by fiber and cement. The direct sheer is another important parameter in geotechnical engineering. It influences the foundation bearing capacity and many other properties especially for soil base and base course. The left picture shos the course of makingsample and the right picture shos the test process.The CBR test and rebound modulus test are referenced from highay test specification to evaluating the prehensive capacities of each structure level of the pavement. For both the to tests, the left picture shos the course of making sample and the right picture shos the test process. What should be noted is that the number of sample is at least 6, the last result is the average value of these date got from test after eliminating the badresults.Four kinds of experiments ere performed to investigate the influence of above factors to the ater-stable properties of stabilized soil. The scouring test is not the stated experiment in current specification. It is performed by us through looking up large quantity of interrelated literature, and to different ays to carry out. The left picture shos the method of vibration table and the right picture shos the method of fatigue test instrument. Penetrant test refers to the experiment in relating concrete specification. The left picture shos the process of saturation, the right picture shos the test process.Cantabria test and other tests are all original experiments; they are used in stabilized soil for first time, here I ill not develop my narrative.As regards the innovation, I think it throughout the hole research, including materials choosing, scheme making, mechanical and ater-stable experiments. I think it can be draled from the folloing keyords, such as soil choosing, sand, poders, fibers, and so on. Three main parts can be summarized. First, selecting to kinds of soils, three kinds of poders, several binations; second, several kinds of fibers, different length and admixture; third, prehensive experiments, test method and test instrument.篇二：模拟国际会议演讲稿Recsplorer:Remendation Algorithms Based on Precedence Mining 1. IntroductionThank you very much, Dr. Li, for your kind introduction. Ladies and gentlemen, Good morning! I am honored to have been invitedto speak at this conference. Before I start my speech, let me ask a question. Do you think reemdations from others are useful for your internet shopping? Thank you. It is obvious that remendations play an important role in our daily consumption decisions.Today, my topic is about Remendation Algorithms Based on Precedence Mining. I ant to share our interesting research result on remendation algorithms ith you. The content of this presentation is divided into 5 parts: in session 1, I ill intruduce the tradictional remendation and our ne strategy; in session 2, I ill give the formal definition of Precedence Mining; in session 3, I ill talk about the novel remendation algorithms; experimental result ill be shoed in session 4; and finally, I ill make a conclusion.2. BodySession 1: IntroductionThe picture on this slide is an instance of rememdation application on amazon.Remender systems provide advice on products, movies,eb pages, and many other topics, and have bee popular in many sites, such as Amazon. Many systems use collaborative filtering methods. The main process of CF is organized as follo: first, identify users similar to target user; second, remend items based on the similar users. Unfortunately, the order of consumed items is neglect. In our paper, e consider a ne remendation strategy based on precedence patterns. These patterns may enpass user preferences, encode some logical order of options and capture ho interests evolve.Precedence mining model estimate the probability of user future consumption based on past behavior. And these probabilities are used to make remendations. Through our experiment, precedence mining can significantly improve remendation performance. Futhermore, it does not suffer from the sparsity of ratings problem and exploit patterns across all users, not just similar users.This slide demonstrates the differences beteen collaborative filtering and precedence mining. Suppose that the scenario is about course selection. Each quarter/semester a student chooses a course, and rates it from 1 to 5. Figure a) shos five transcripts, a transcript means a list of course. U is our target student ho need remendations. Figure b) illustrates ho CF ork. Assume similar users share at least to mon courses and have similar rating, then u3 and u4 are similar to u, and their mon course h ill be a remendation to u. Figure c) presents ho precedence mining ork. For this example, e consider patterns here one course follos another. Suppose patterns occour at least to transcrips are recognized as significant, then and are found out. And d, h, and f are remendation to u ho has taken a, g and e.No I ill a probabilistic frameork to solve the precedence mining problems. Our target user has selected course a , e ant to pute the probability course x ill follo, i.e., Pr.﹁hoerve, hat e really need to calculate is Pr rather than Pr. Because in our context,e are deciding if x is a good remendation for the target user that has taken a. Thus e kno that our target user’s transcriptdoes not have x before a. For instance, the transcript no. 5 ill be omitted. In more mon situation, our target user has taken a list of courses, T = {a,b,c,…} not﹁just a. Thus, hat really need is Pr. The question is ho to figure out this probability. I illanser it later.Session 2: Precedence MiningWe consider a set D of distinct courses. We use loercase letters to refer to courses in D. A transcript T is a sequence of courses, e.g., a -> b -> c -> d. Then the definition of Top-k Remendation Problem is as follos. Given a set transcripts over D for n users, the extra transcript T of a target user, and a desired number of remendations k, our goal is to:1. Assign a score scoreto every course x ∈ D that reflects ho likely it is the target student ill be interested in taking x. If x ∈ T , then scor e= 0.2. Using the score function, select the top k courses to remend to the target user.To pute scores, e propose to use the folloing statistics, here x, y ∈ D:fand gis 10 and gis 3.We propose a precedence mining model to solve the Top-k Remendation Problem. Here are﹁some notation: xy, hich e have memtioned in session 1, refers to transcript here x occursithout a preceding y; x﹁y refers to transcript here x occurs ithout y folloing it. We use quantities fand gto pte probabilities that encode the precedence information. For instance, from formular 1 to 7. I ould not tell the detail ofall formulars. We just pay attention to﹁formular 5, note that this quantity above is the same as: Pr hich ill be used topute scoreusing the formular 5. The detail is as follos: a student ith a transcrip T of taken courses, for the course y ∈ T, if y and x appear together in transcripts satisfies the ﹁threshold θ, then pute the Pr, reflecting the likelihood the student ill take course x﹁and ignoring the effect of the other courses in T; finally the maximum of Pr is choosen asthe scoreof SignleMC. For example, ith the higer score, d ill be remended.Another ne remendation algorithm named Joint Probabilities algorithm, JointP for short, is proposed. Unlike SingleMC, JointP takes into account the plete set of courses in a transcript. In formular 12, e cannot pute its quantity exactly, Remember this problem e have mentioned. Our solution is to use approximations. This slide is about the first approximating formular. And this the second approximating formular.The system is courseRand, and data set for experiment contains 7,500 transcripts.This slide shos the ne remendation algoritms ith black color and the traditional ones ith blue color.The chart on this slide indicates our ne remendation algorithms beat the traditional ones in precision, because the former ones exploit patterns across all users, hile the latter ones just use the similar users.The chart on this slide points out our ne remendation algorithms also beat the traditional ones in coverage for the same reason.Session 5: Conclusion and SummaryIn conclusion, e proposed a novel precedence mining model, developed a probabilistic frameork for making remendations and implemented a suite of remendation algorithms that use the precedence information. Experimental result shos that our ne algorithms perform better than the traditional ones, and our remendation system can be easily generalized to other scenarios, such as purchases of books, DVDs and electronic equitment.To sum up, first, I introduced the motivation and the outline of ork; second, I gave the definition of precedence mining model; third, I described some ne remendation algorithms using precedence information; forth, I shoed our experimental results to pare the ne algorithms ith traditional ones. Finally, I made a conclusion of our ork..That’s all. Thank you! Are there any questions?篇三：英文国际学术会议开幕词演讲稿 International Conference on Space TechnologyDistinguished guests, distinguished delegates, ladies and gentlemen, and all the friends:At this special time of onderful September, in this grand hall of the beautiful city, our respectable guests are here getting together. Jointly sponsored by the International Astronautical Federation and the International Institute of Space La大家上午好！今天我汇报的主题是：基于改进型LBP算法的运动目标检测系统。
稀疏投影在目标跟踪中的应用
跟踪 算法 能够 实现 对一段 时 间内 目标空 间状 态 的估计 ． 期研究 中 ， 础跟 踪算 法通 常采用 卡 早 基 尔 曼滤 波实 现． 而 由于 卡 尔曼 滤 波 只 能提 供 状 然
态变化服从线性高斯分布的 目标最优值估计 ， 不 符合 现实 中 的 目标 运 动 变 化状 态 ， 得 其 后产 生 使 的基 于非 线性模 型 的粒子 滤波 方法逐 渐成 为更 广
ｓ ａ ｓ ｅ ｒ ｓ ｎ ａｉｎ ｏ ａｇ ｔｆａｕｒ ｓａ ｈ ｅ ｅ ｙ ｓ ｌ ｉｇ ａ ｅ ｕ ａ ｉｅ ｅ ｓ ｑ ｒ ｓｐ ｏ ｌ ｍ ， ｐ ｒｅ ｒ ｐ ｅ ｅ ｔｔｏ ｆｔｒ ｅ ｅ ｔ ｅ ｉ ｃ ｉｖ ｄ ｂ ｏｖｎ ｎ Ｌｌｒ ｇ ｌ ｒｚ ｄ ｌ ａ ｔｓ ｕａ ｅ ｒ ｂ ｅ
第２ ８卷 第 ４期 ２１ ０ ２年 ８月
上
Ｊｕｎ ｏｒ￣
海 电 力 学 院 学
报
Ｖｏ ． ８．Ｎｏ ４ １２ ．
ｏ Ｓ ａ ｇ ａ Ｕｎ ｖ ｒ ｉ ｏ Ｅｌ ｃｒｃ Ｐｏ ｒ ｆ ｈｎ ｈｉ ｉ ｅ ｓｔ ｙ ｆ ｅ ｔｉ ｗｅ
Ａｕ ． ２ ２ ｇ ０１
ｆ ｌ ｔｃ ｏ ｅ，ｈ ｇ ２Ｏ９ ，ｈｎ ｏ Ｅ ｃｉ Ｐｗ ｒＳ ｎｈ ｉ ＯＯＯ Ｃ ｉ ） ｅபைடு நூலகம்ｒ ａ ａ ａ
Ａｂｓｒ ｃ ： ｔａ ｔ
Ｔ ｐ ｌｃ ｔ ｎ ｆ ｓ ａ ｓ ｒ ｐ ｅ ｅ ｔ ｔ ｎ ｉ ｔｒ ｅ ｔａ ｋ ｎ ｓ ｓｅ ｈｅ ａ ｐ ｉａｉ ｏ ｐ ｒｅ ｅ ｒ ｓ ｎ ａ ｉ ｎ ａ ｇ ｔ ｒ ｃ ｉ ｇ ｙ ｔｍ ｉ ｄ ｓ ｕ ｓ ｄ． Ｔ ｅ ｏ ｏ ｓ ｉｃ ｓ ｅ ｈ
基于稀疏贝叶斯正则化的阵列SAR高分辨三维成像算法
基于稀疏贝叶斯正则化的阵列SAR高分辨三维成像算法闫敏;韦顺军;田博坤;张晓玲;师君【摘要】阵列合成孔径雷达(Linear Array Synthetic Aperture Radar,LASAR)3维成像技术是一种具有重要潜在应用价值的雷达成像新体制,但受线阵天线及平台尺寸限制,传统匹配滤波成像算法难以实现LASAR高分辨3维成像.该文利用LASAR回波信号及观测目标的先验分布特性,提出了一种基于快速稀疏贝叶斯正则化重构的LASAR高分辨3维成像算法.该算法先结合贝叶斯估计准则及最大似然估计原理,构造LASAR目标重构的稀疏贝叶斯最小化代价函数;再利用迭代正则化方法求解联合范数最优化问题实现LASAR稀疏目标高分辨3维成像.另外,针对稀疏贝叶斯正则化成像运算量大的问题,结合位置预测快速成像思路,利用阈值分割算法对稀疏粗成像进行强目标提取,进而提升算法运算效率.仿真数据和实测数据验证了该文算法的有效性.【期刊名称】《雷达学报》【年(卷),期】2018(007)006【总页数】12页(P705-716)【关键词】阵列合成孔径雷达;3维成像;压缩感知;稀疏贝叶斯;稀疏重构【作者】闫敏;韦顺军;田博坤;张晓玲;师君【作者单位】电子科技大学信息与通信学院成都 611731;电子科技大学信息与通信学院成都 611731;电子科技大学信息与通信学院成都 611731;电子科技大学信息与通信学院成都 611731;电子科技大学信息与通信学院成都 611731【正文语种】中文【中图分类】TN957.51 引言由于具备全天时、全天候、高分辨3维成像能力，阵列合成孔径雷达(Linear Array Synthetic Aperture Radar， LASAR) 3维成像技术是近几年来被广泛关注的一种新型合成孔径雷达(Synthetic Aperture Radar， SAR)成像技术[1-3]。
LASAR是传统2维SAR成像的扩展，主要通过控制阵列天线在空间中运动形成虚拟2维面阵获得观测目标2维分辨，并结合脉冲压缩技术得到观测目标的第3维分辨，最终实现观测目标的3维成像。
含旋转部件目标稀疏孔径ISAR成像方法
含旋转部件目标稀疏孔径ISAR成像方法徐艺萌;管桦;罗迎;王磊;马赛【摘要】To solve the problem of the sparse-aperture ISAR imaging for the target with rotating parts is poorer,a method based on sub-aperture Chirplet transform and Compressed Sensing(CS) is proposed in this paper.Firstly,the sparse ISAR imaging model of the target with rotating parts is established,then the effect of micro-Doppler of the target with rotating parts in broadband radar is deduced,and then the influence of the sparse of aperture and the micro-Doppler together isanalyzed.Secondly,the radar echoes are decomposed into a series of Chirplet basis,getting rid of the micro-Doppler signal in sub-aperture based on the difference of the Chirplet parameters of the echo signals of main parts and rotating parts.Finally,to recover the imaging from the subaperture based on the CS method of Orthogonal Matching Pursuit (OMP)algorithm,getting the high quality imaging results of the target with rotating parts.Simulations show the method could eliminate the micro-doppler effect and achieve high quality imaging.%针对稀疏孔径条件下含旋转部件目标(ISAR)成像质量较差的问题,提出了一种基于子孔径Chirplet变换和压缩感知(CS)的含旋转部件目标稀疏孔径ISAR成像方法:首先,建立了含旋转部件目标的稀疏ISAR成像模型,推导了宽带雷达条件下含旋转部件目标的微多普勒效应,并分析了孔径的稀疏与微多普勒效应共存时对成像的影响；其次将回波信号投影到Chirplet变换基,利用目标主体回波信号和微动部件回波信号在Chirplet变换投影参数上的差异,有效剔除有效子孔径中的微多普勒调制信号；最后,采用基于正交匹配追踪(OMP)算法的CS方法对有效子孔径进行恢复成像,获得了含旋转部件目标的高质量成像结果.仿真实验表明,该方法可以有效消除微多普勒效应和孔径稀疏的干扰,并实现高质量的ISAR成像.【期刊名称】《空军工程大学学报（自然科学版）》【年(卷),期】2013(014)004【总页数】5页(P57-61)【关键词】ISAR;微多普勒;稀疏孔径;Chirplet变换;压缩感知;正交匹配追踪【作者】徐艺萌;管桦;罗迎;王磊;马赛【作者单位】空军工程大学信息与导航学院,陕西西安,710077;空军工程大学信息与导航学院,陕西西安,710077;空军工程大学信息与导航学院,陕西西安,710077;95853部队,北京,100076;空军工程大学信息与导航学院,陕西西安,710077【正文语种】中文【中图分类】TN957.52逆合成孔径雷达（Inverse Synthetic Aperture Radar，ISAR）可以实现对运动目标的成像，为目标识别提供重要信息。
基于局部特征和稀疏表示的图像目标检测算法
基于局部特征和稀疏表示的图像目标检测算法田元荣;田松;许悦雷;查宇飞【摘要】传统的基于局部特征的图像目标检测算法具有对遮挡和旋转敏感、检测精度不高以及运算速度慢的特点,为了改进该算法的性能,提出了一种将图像局部特征应用于稀疏表示理论的图像目标检测算法.该算法利用随机树的方式有监督地学习样本图像的局部特征形成字典,通过学习好的字典和测试图像的子块来预测图像中目标的中心位置,以此寻求待检测图像稀疏的表示,从而实现对图像中感兴趣目标的检测.实验结果表明,该算法对目标的遮挡、旋转和复杂背景有很好的鲁棒性,而且检测精度和运算速度相对于同类经典算法均有提高.【期刊名称】《计算机应用》【年(卷),期】2013(033)006【总页数】4页(P1670-1673)【关键词】目标检测;稀疏表示;局部特征;随机树;字典学习【作者】田元荣;田松;许悦雷;查宇飞【作者单位】空军工程大学航空航天工程学院,西安710038;空军工程大学航空航天工程学院,西安710038;空军工程大学航空航天工程学院,西安710038;空军工程大学航空航天工程学院,西安710038【正文语种】中文【中图分类】TP391.410 引言图像目标检测是机器视觉和模式识别领域一个基础但非常重要的研究方向，它已被广泛地应用于视频监控、航空制导、战场侦察、医疗诊断等多个领域。
由于光照、遮挡、成像角度等条件对获取的图像有较大的影响，现有的目标检测方法仍然存在很大局限性，寻找高精度、高效和高鲁棒性的目标检测算法仍是机器视觉领域一种迫切的需求。
近年来，神经生理学研究表明在哺乳动物的本原视觉皮层存在稀疏编码策略［1］，人类视觉系统可以将物体分解为许多有意义的小块，并通过这些局部信息进行目标的辨识，加之局部特征相对目标整体而言本身具有旋转不变、对遮挡和姿态变化鲁棒等优良的特性，基于局部特征的目标检测方法很快成为了国内外研究的热点。
该方法一般采用结构与部件(Parts and Structure)模型［2］和词袋(Bag－Of－Words)模型［3］进行目标建模。
Block Coordinate Descent for Sparse NMF
Block Coordinate Descent for Sparse NMFVamsi K.Potluru Department of Computer Science, University of New Mexicoismav@Sergey M.Plis Mind Research Network, splis@Jonathan Le Roux Mitsubishi Electric Research Labs leroux@Barak A.PearlmutterDepartment of Computer Science, National University of Ireland Maynoothbarak@cs.nuim.ieVince D.CalhounElectrical and Computer Engineering,UNM andMind Research Networkvcalhoun@Thomas P.Hayes Department of Computer Science, University of New Mexicohayes@AbstractNonnegative matrix factorization(NMF)has become a ubiquitous tool fordata analysis.An important variant is the sparse NMF problem whicharises when we explicitly require the learnt features to be sparse.A naturalmeasure of sparsity is the L0norm,however its optimization is NP-hard.Mixed norms,such as L1/L2measure,have been shown to model sparsityrobustly,based on intuitive attributes that such measures need to satisfy.This is in contrast to computationally cheaper alternatives such as the plainL1norm.However,present algorithms designed for optimizing the mixednorm L1/L2are slow and other formulations for sparse NMF have been pro-posed such as those based on L1and L0norms.Our proposed algorithmallows us to solve the mixed norm sparsity constraints while not sacrificingcomputation time.We present experimental evidence on real-world datasetsthat shows our new algorithm performs an order of magnitude faster com-pared to the current state-of-the-art solvers optimizing the mixed norm andis suitable for large-scale datasets.1IntroductionMatrix factorization arises in a wide range of application domains and is useful for extracting the latent features in the dataset(Figure1).In particular,we are interested in matrix factorizations which impose the following requirements:•nonnegativity•low-rankedness•sparsityNonnegativity is a natural constraint when modeling data with physical constraints such as chemical concentrations in solutions,pixel intensities in images and radiation dosages for cancer treatment.Low-rankedness is useful for learning a lower dimensionality represen-tation.Sparsity is useful for modeling the conciseness of the representation or that of thelatent features.Imposing all these requirements on our matrix factorization leads to the sparse nonnegative matrix factorization(SNMF)problem.SNMF enjoys quite a few formulations[2,14,13,11,24,17,25,26]with successful applica-tions to single-channel speech separation[27]and micro-array data analysis[17,25]. However,algorithms[14,11]for solving SNMF which utilize the mixed norm of L1/L2as their sparsity measure are slow and do not scale well to large datasets.Thus,we develop an efficient algorithm to solve this problem and has the following ingredients:•A theoretically efficient projection operator(O(m log m))to enforce the user-defined sparsity where m is the dimensionality of the feature vector as opposed to the previous approach[14].•Novel sequential updates which provide the bulk of our speedup compared to the previously employed batch methods[14,11].Figure1:(Left)Features learned from the ORL dataset2with various matrix factorization methods such as principal component analysis(PCA),independent component analysis (ICA),and dictionary learning.The relative merit of the various matrix factorizations depends on both the signal domain and the target application of interest.(Right)Features learned under the sparse NMF formulation where roughly half the features were constrained to lie in the interval[0.2,0.4]and the rest arefixed to sparsity value0.7.This illustrates theflexibility that the user has infine tuning the feature sparsity based on prior domain knowledge.White pixels in thisfigure correspond to the zeros in the features.2Preliminaries and Previous WorkIn this section,we give an introduction to the nonnegative matrix factorization(NMF)and SNMF problems.Also,we discuss some widely used algorithms from the literature to solve them.Both these problems share the following problem and solution structure.At a high-level, given a nonnegative matrix X of size m×n,we want to approximate it with a product of two nonnegative matrices W,H of sizes m×r and r×n,respectively:(1)X≈WH.The nonnegative constraint on matrix H makes the representation a conical combination of features given by the columns of matrix W.In particular,NMF can result in sparse representations,or a parts-based representation,unlike other factorization techniques such as principal component analysis(PCA)and vector quantization(VQ).A common theme in 2Scikit-learn package was used in generating thefigure.the algorithms proposed for solving these problems is the use of alternating updates to the matrix factors,which is natural because the objective function to be minimized is convex in W and in H,separately,but not in both together.Much effort has been focused on optimizing the efficiency of the core step of updating one of W,H while the other stays fixed.2.1Nonnegative Matrix FactorizationFactoring a matrix,all of whose entries are nonnegative,as a product of two low-rank non-negative factors is a fundamental algorithmic challenge.This has arisen naturally in diverse areas such as image analysis[20],micro-array data analysis[17],document clustering[31], chemometrics[19],information retrieval[12]and biology applications[4].For further appli-cations,see the references in the following papers[1,7].We will consider the following version of the NMF problem,which measures the reconstruc-tion error using the Frobenius norm[21]:min W,H 12X−WH 2F s.t.W≥0,H≥0, W j 2=1,∀j∈{1,···,r}(2)where≥is element-wise.We use subscripts to denote column elements.Simple multiplica-tive updates were proposed by Lee and Seung to solve the NMF problem.This is attractive for the following reasons:•Unlike additive gradient descent methods,there is no arbitrary learning rate pa-rameter that needs to be set.•The nonnegativity constraint is satisfied automatically,without any additional pro-jection step.•The objective function converges to a limit point and the values are non-increasing across the updates,as shown by Lee and Seung[21].Algorithm1is an example of the kind of multiplicative update procedure used,for instance, by Lee and Seung[21].The algorithm alternates between updating the matrices W and H (we have only shown the updates for H—those for W are analogous).Algorithm1nnls-mult(X,W,H)1:repeat2:H=H W XW WH .3:until convergence4:Output:Matrix H.Here, indicates element-wise(Hadamard)product and matrix division is also element-wise.To remove the scaling ambiguity,the norm of columns of matrix W are set to unity. Also,a small constant,say10−9,is added to the denominator in the updates to avoid division by zero.Besides multiplicative updates,other algorithms have been proposed to solve the NMF problem based on projected gradient[22],block pivoting[18],sequential constrained opti-mization[6]and greedy coordinate-descent[15].2.2Sparse Nonnegative Matrix FactorizationThe nonnegative decomposition is in general not unique[9].Furthermore,the features may not be parts-based if the data resides well inside the positive orthant.To address these issues,sparseness constraints have been imposed on the NMF problem.Sparse NMF can be formulated in many different ways.From a user point of view,we can split them into two classes of formulations:explicit and implicit.In explicit versions of SNMF[14,11],one can set the sparsities of the matrix factors W,H directly.On the otherhand,in implicit versions of SNMF[17,25],the sparsity is controlled via a regularization parameter and is often hard to tune to specified sparsity values a priori.However,the algorithms for implicit versions tend to be faster compared to the explicit versions of SNMF. In this paper,we consider the explicit sparse NMF formulation proposed by Hoyer[14]. To make the presentation easier to follow,wefirst consider the case where the sparsity is imposed on one of the matrix factors,namely the feature matrix W—the analysis for the symmetric case where the sparsity is instead set on the other matrix factor H is analogous. The case where sparsity requirements are imposed on both the matrix factors is dealt with in the Appendix.The sparse NMF problem formulated by Hoyer[14]with sparsity on matrix W is as follows:min W,H f(W,H)=12X−WH 2F s.t.W≥0,H≥0,W j 2=1,sp(W j)=α,∀j∈{1,···,r}(3)Sparsity measure for a d-dimensional vector x is given by:sp(x)=√d− x 1/ x 2√d−1(4)The sparsity measure(4)defined above has many appealing qualities.Some of which are as follows:•The measure closely models the intuitive notion of sparsity as captured by the L0 norm.So,it easy for the user to specify sparsity constraints from prior knowledge of the application domain.•Simultaneously,it is able to avoid the pitfalls associated with directly optimizing the L0norm.Desirable properties for sparsity measures have been previously ex-plored[16]and it satisfies all of these properties for our problem formulation.The properties can be briefly summarized as:(a)Robin Hood—Spreading the energy from larger coordinates to smaller ones decreases sparsity,(b)Scaling—Sparsity is invariant to scaling,(c)Rising tide—Adding a constant to the coordinates decreases sparsity,(d)Cloning—Sparsity is invariant to cloning,(e)Bill Gates —One big coordinate can increase sparsity,(f)Babies—coordinates with zeros increase sparsity.•The above sparsity measure enables one to limit the sparsity for each feature to lie in a given range by changing the equality constraints in the SNMF formulation(3) to inequality constraints[11].This could be useful in scenarios like fMRI brain analysis,where one would like to model the prior knowledge such as sizes of artifacts are different from that of the brain signals.A sample illustration on a face dataset is shown in Figure1(Right).The features are now evenly split into two groups of local and global features by choosing two different intervals of sparsity.A gradient descent-based algorithm called Nonnegative Matrix Factorization with Sparse-ness Constraints(NMFSC)to solve SNMF was proposed[14].Multiplicative updates were used for optimizing the matrix factor which did not have sparsity constraints spec-ified.Heiler and Schn¨o rr[11]proposed two new algorithms which also solved this prob-lem by sequential cone programming and utilized general purpose solvers like MOSEK ().We will consider the faster one of these called tangent-plane constraint(TPC)algorithm.However,both these algorithms,namely NMFSC and TPC, solve for the whole matrix of coefficients at once.In contrast,we propose a block coordinate-descent strategy which considers a sequence of vector problems where each one can be solved in closed form efficiently.3The Sequential Sparse NMF AlgorithmWe present our algorithm which we call S equential S parse NMF(SSNMF)to solve the SNMF problem as follows:First,we consider a problem of special form which is the building block(Algorithm2)of our SSNMF algorithm and give an efficient,as well as exact,algorithm to solve it.Second,we describe our sequential approach(Algorithm3)to solve the subproblem of SNMF.This uses the routine we developed in the previous step.Finally,we combine our routines developed in the previous two steps along with standard solvers(for instance Algorithm1)to complete the SSNMF Algorithm(Algorithm4).3.1Sparse-optSparse-opt routine solves the following subproblem which arises when solving problem(3):max y≥0b y s.t. y 1=k, y 2=1(5)where vector b is of size m.This problem has been previously considered[14],and an algorithm to solve it was proposed which we will henceforth refer to as the Projection-Hoyer.Similar projection problems have been recently considered in the literature and solved efficiently[10,5].Observation1.For any i,j,we have that if b i≥b j,then y i≥y j.Let usfirst consider the case when the vector b is sorted.Then by the previous observation, we have a transition point p that separates the zeros of the solution vector from the rest. Observation2.By applying the Cauchy-Schwarz inequality on y and the all ones vector, we get p≥k2.The Lagrangian of the problem(5)is:L(y,µ,λ,γ)=b y+µ mi=1y i−k+λ2mi=1y2i−1+γ ySetting the partial derivatives of the Lagrangian to zero,we get by observation1:mi=1y i=k,mi=1y2i=1b i+µ(p)+λ(p)y i=0,∀i∈{1,2,···,p}γi=0,∀i∈{1,···,p}y i=0,∀i∈{p+1,···,m}where we account for the dependence of the Lagrange parametersλ,µ,andγon the tran-sition point p.We compute the objective value of problem(5)for all transition points p in the range from k2to m and select the one with the highest value.In the case,where the vector b is not sorted,we just simply sort it and note down the sorting permutation vector.The complete algorithm is given in Algorithm2.The dominant contribution to the running time of Algorithm2is the sorting of vector b and therefore can be implemented in O(m log m)time3.Contrast this with the running time of Projection-Hoyer whose worst case is O(m2)[14,28].3.2Sequential Approach—Block Coordinate DescentPrevious approaches for solving SNMF[14,11]use batch methods to solve for sparsity constraints.That is,the whole matrix is updated at once and projected to satisfy the con-straints.We take a different approach of updating a column vector at a time.This gives us the benefit of being able to solve the subproblem(column)efficiently and exactly.Subse-quent updates can benefit from the newly updated columns resulting in faster convergence as seen in the experiments.3This can be further reduced to linear time by noting that we do not need to fully sort the input in order tofind p∗.Algorithm 2Sparse-opt(b ,k )1:Set a =sort(b )and p ∗=m .Get a mapping πsuch that a i =b π(i )and a j ≥a j +1for all valid i,j .2:Compute values of µ(p ),λ(p )as follows:3:for p = k 2 to m do4:λ(p )=− p pi =1a 2i −( p i =1a i )2(p −k 2)5:µ(p )=− p i =1a i p −kp λ(p )6:if a (p )<−µ(p )then7:p ∗=p −18:break9:end if10:end for 11:Set x i =−a i+µ(p ∗)λ(p ∗),∀i ∈{1,···,p ∗}and to zero otherwise.12:Output:Solution vector y where y π(i )=x i .In particular,consider the optimization problem (3)for a column j of the matrix W while fixing the rest of the elements of matrices W ,H :min W j ≥0˜f (W j )=12g W j 22+u W j s.t. W j 2=1, W j 1=k where g =H j H j and u =−XH j + i =j W i (HH )ij .This reduces to the problem (5)forwhich we have proposed an exact algorithm (Algorithm 2).We update the columns of the matrix factor W sequentially as shown in Algorithm 3.We call it sequential for we update the columns one at a time.Note that this approach can be seen as an instance of block coordinate descent methods by mapping features to blocks and the Sparse-opt projection operator to a descent step.Algorithm 3sequential-pass(X ,W ,H )1:C =−XH +WHH 2:G =HH 3:repeat 4:for j =1to r (randomly)do 5:U j =C j −W j G jj 6:t =Sparse-opt(−U j ,k ).7:C =C +(t −W j )G j 8:W j =t .9:end for 10:until convergence 11:Output:Matrix W .3.3SSNMF Algorithm for Sparse NMFWe are now in a position to present our complete Sequential Sparse NMF (SSNMF)algo-rithm.By combining Algorithms 1,2and 3,we obtain SSNMF (Algorithm 4).Algorithm 4ssnmf(X ,W ,H )1:repeat2:W =sequential-pass(X ,W ,H )3:H =nnls-mult(X ,W ,H )4:until convergence5:Output:Matrices W ,H .4Implementation IssuesFor clarity of exposition,we presented the plain vanilla version of our SSNMF Algorithm 4.We now describe some of the actual implementation details.•Initialization:Generate a positive random vector v of size m and obtain z =Sparse-opt(v ,k )where k =√m −α√m −1(from equation (4)).Use the solu-tion z and its random permutations to initialize matrix W .Initialize the matrix H to uniform random entries in [0,1].•Incorporating faster solvers:We use multiplicative updates for a fair comparison with NMFSC and TPC.However,we can use other NNLS solvers [22,18,6,15]to solve for matrix H .Empirical results (not reported here)show that this further speeds up the SSNMF algorithm.•Termination:In our experiments,we fix the number of alternate updates or equiv-alently the number of times we update matrix W .Other approaches include spec-ifying total running time,relative change in objective value between iterations or approximate satisfaction of KKT conditions.•Sparsity constraints:We have primarly considered the sparse NMF model as for-mulated by Hoyer [14].This has been generalized by Heiler and Schn¨o rr [11]by relaxing the sparsity constraints to lie in user-defined intervals.Note that,we can handle this formulation [11]by making a trivial change to Algorithm 3.5Experiments and DiscussionIn this section,we compare the performance of our algorithm with the state-of-the-art NMFSC and TPC algorithms [14,11].Running times for the algorithms are presented when applied to one synthetic and three real-world datasets.Experiments report reconstruction error ( X −WH F )instead of objective value for convenience of display.For all experiments on the datasets,we ensure that our final reconstruction error is always better than that of the other two algorithms.Our algorithm was implemented in MATLAB ( )similar to NMFSC and TPC.All of our experiments were run on a 3.2Ghz Intel machine with 24GB of RAM and the number of threads set to one.5.1DatasetsFor comparing the performance of SSNMF with NMFSC and TPC,we consider the following synthetic and three real-world datasets :•Synthetic:200images of size 9×9as provided by Heiler and Schn¨o rr [11]in their code implementation.•CBCL:Face dataset of 2429images of size 19×19and can be obtained at /cbcl/software-datasets/FaceData2.html .•ORL:Face dataset that consists of 400images of size 112×92and can be obtained at /research/dtg/attarchive/facedatabase.html .•sMRI:Structural MRI scans of 269subjects taken at the John Hopkins University were obtained.The scans were taken on a single 1.5T scanner with the imag-ing parameters set to 35mm TR,5ms TE,matrix size of 256×256.We segment these images into gray matter,white matter and cerebral spinal fluid images,using the software program SPM5(/spm/software/spm5/),followed by spatial smoothing with a Gaussian kernel of 10×10×10mm.This results in images which are of size 105×127×46.5.2Comparing Performances of Core UpdatesWe compare our Sparse-opt (Algorithm 2)routine with the competing Projection-Hoyer [14].In particular,we generate 40random problems for each sparsity constraintFigure2:Mean running times for Sparse-opt and the Projection-Hoyer are presented for random problems.The x-axis plots the dimension of the problem while the y-axis has the running time in seconds.Each of the subfigures corresponds to a single sparsity value in {0.2,0.4,0.6,0.8}.Each datapoint corresponds to the mean running time averaged over40 runs for random problems of the samefixed dimension.in{0.2,0.4,0.6,0.8}and afixed problem size.The problems are of size2i×100where i takes integer values from0to12.Input coefficients are generated by drawing samples uniformly at random from[0,1].The mean values of the running times for Sparse-opt and the Projection-Hoyer for each dimension and corresponding sparsity value are plotted in Figure2.We compare SSNMF with SSNMF+Proj on the CBCL dataset.The algorithms were run with rank set to49.The running times are shown in Figure3.We see that in low-dimensionalFigure3:Running times for SSNMF and SSNMF+Proj algorithms for the CBCL face dataset with rank set to49and sparsity values ranging from0.2to0.9datasets,the difference in running times are very small.5.3Comparing Overall PerformancesSSNMF versus NMFSC and TPC:We plot the performance of SSNMF against NMFSC and TPC on the synthetic dataset provided by Heiler and Schn¨o rr[11]in Fig-ure4.We used the settings for both and TPC using the software provided by the authors.Our experience with TPC was not encouraging on bigger datasets and hence we show its performance only on the synthetic dataset.It is possible that the performance of TPC can be improved by changing the default settings but we found it non-trivial to do so.Figure4:Running times for SSNMF and NMFSC and TPC algorithms on the synthetic dataset where the sparsity values range from0.2to0.8and number of features is5.Note that SSNMF and NMFSC are over an order of magnitude faster than TPC.Figure5:Convergence plots for the ORL dataset with sparsity from[0.1,0.8]for the NMFSC and SSNMF algorithms.Note that we are an order of magnitude faster,especially when the sparsity is higher.SSNMF versus NMFSC:To ensure fairness,we removed logging information from NMFSC code[14]and only computed the objective for equivalent number of matrix updates as SSNMF.We do not plot the objective values at thefirst iteration for convenience of display.However,they are the same for both algorithms because of the shared initialization .We ran the SSNMF and NMFSC on the ORL face dataset.The rank wasfixed at25 in both the algorithms.Also,the plots of running times versus objective values are shown in Figure5corresponding to sparsity values ranging from0.1to0.7.Additionally,we ran our SSNMF algorithm and NMFSC algorithm on a large-scale dataset consisting of the structural MRI images by setting the rank to40.The running times are shown in Figure6.5.4Main ResultsWe compared the running times of our Sparse-opt routine versus the Projection-Hoyer and found that on the synthetically generated datasets we are faster on average.Figure 6:Running times for SSNMF and NMFSC algorithms for the sMRI dataset with rank set to 40and sparsity values of αfrom 0.1to 0.8.Note that for higher sparsity values we converged to a lower reconstruction error and are also noticeably faster than the NMFSC algorithm.Our results on switching the Sparse-opt routine with the Projection-Hoyer did not slow down our SSNMF solver significantly for the datasets we considered.So,we conclude that the speedup is mainly due to the sequential nature of the updates (Algorithm 3).Also,we converge faster than NMFSC for fewer number of matrix updates.This can be seen by noting that the plotted points in Figures 5and 6are such that the number of matrix updates are the same for both SSNMF and NMFSC.For some datasets,we noted a speedup of an order of magnitude making our approach attractive for computation purposes.Finally,we note that we recover a parts-based representation as shown by Hoyer [14].An example of the obtained features by NMFSC and ours is shown in Figure 7.(a)sparsity 0.5(b)sparsity 0.6(c)sparsity 0.75Figure 7:Feature sets from NMFSC algorithm (Left)and SSNMF algorithm (Right)using the ORL face dataset for each sparsity value of αin {0.5,0.6,0.75}.Note that SSNMF algorithm gives a parts-based representation similar to the one recovered by NMFSC.6Connections to Related WorkOther SNMF formulations have been considered by Hoyer [13],Mørup et al.[24],Kim and Park [17],Pascual-Montano et al.[25](nsNMF)and Peharz and Pernkopf [26].SNMF formulations using similar sparsity measures as used in this paper have been considered for applications in speech and audio recordings [30,29].We note that our sparsity measure has all the desirable properties,extensively discussed by Hurley and Rickard [16],except for one (“cloning”).Cloning property is satisfied when two vectors of same sparsity when concatenated maintain their sparsity value.Dimensions in our optimization problem are fixed and thus violating the cloning property is not anpare this with the L1norm that satisfies only one of these properties(namely “rising tide”).Rising tide is the property where adding a constant to the elements of a vector decreases the sparsity of the vector.Nevertheless,the measure used in Kim and Park is based on the L1norm.The properties satisfied by the measure in Pascual-Montano et al. are unclear because of the implicit nature of the sparsity formulation.Pascual-Montano et al.[25]claim that the SNMF formulation of Hoyer,as given by prob-lem(3)does not capture the variance in the data.However,some transformation of the sparsity values is required to properly compare the two formulations[14,25].Preliminary results show that the formulation given by Hoyer[14]is able to capture the variance in the data if the sparsity parameters are set appropriately.Peharz and Pernkopf[26]propose to tackle the L0norm constrained NMF directly by projecting from intermediate uncon-strained solutions to the required L0constraint.This leads to the well-known problem of getting stuck in local minima.Indeed,the authors re-initialize their feature matrix with an NNLS solver to recover from the local suboptimum.Our formulation avoids the local minima associated with L0norm by using a smooth surrogate.7ConclusionsWe have proposed a new efficient algorithm to solve the sparse NMF problem.Experiments demonstrate the effectiveness of our approach on real datasets of practical interest.Our algorithm is faster over a range of sparsity values and generally performs better when the sparsity is higher.The speed up is mainly because of the sequential nature of the updates in contrast to the previously employed batch updates of Hoyer.Also,we presented an exact and efficient algorithm to solve the problem of maximizing a linear objective with a sparsity constraint,which is an improvement over the previous approach in Hoyer.Our approach can be extended to other NMF variants[13].Another possible application is the sparse version of nonnegative tensor factorization.A different research direction would be to scale our algorithm to handle large datasets by chunking[23]and/or take advantage of distributed/parallel computational settings[3].AcknowledgementThefirst author would like to acknowledge the support from NIBIB grants1R01EB000840 and1R01EB005846.The second author was supported by NIMH grant1R01MH076282-01.The latter two grants were funded as part of the NSF/NIH Collaborative Research in Computational Neuroscience Program.References[1]Sanjeev Arora,Rong Ge,Ravindran Kannan,and Ankur puting a nonneg-ative matrix factorization–provably.In Proceedings of the44th symposium on Theory of Computing,STOC’12,pages145–162,New York,NY,USA,2012.ACM.[2]Michael W Berry,Murray Browne,Amy N Langville,V Paul Pauca,and Robert JPlemmons.Algorithms and applications for approximate nonnegative matrix putational Statistics&Data Analysis,52(1):155–173,2007.[3]Joseph K.Bradley,Aapo Kyrola,Danny Bickson,and Carlos Guestrin.Parallel coor-dinate descent for L1-regularized loss minimization.In ICML,pages321–328,2011.[4]G.Buchsbaum and O.Bloch.Color categories revealed by non-negative matrix factor-ization of munsell color spectra.Vision research,42(5):559–563,2002.[5]Yunmei Chen and Xiaojing Ye.Projection onto a simplex.arXiv preprintarXiv:1101.6081,2011.[6]A.Cichocki and A.H.Phan.Fast local algorithms for large scale nonnegative matrixand tensor factorizations.IEICE Transactions on Fundamentals of Electronics,92: 708–721,2009.[7]J.E.Cohen and U.G.Rothblum.Nonnegative ranks,decompositions,and factor-izations of nonnegative matrices.Linear Algebra and its Applications,190:149–168, 1993.[8]Chris Ding,Tao Li,Wei Peng,and Haesun Park.Orthogonal nonnegative matrix t-factorizations for clustering.In Proceedings of the12th ACM SIGKDD international conference on Knowledge discovery and data mining,KDD’06,pages126–135,New York,NY,USA,2006.ACM.[9]David Donoho and Victoria Stodden.When does non-negative matrix factorization givea correct decomposition into parts?In Sebastian Thrun,Lawrence Saul,and BernhardSch¨o lkopf,editors,Advances in Neural Information Processing Systems16.MIT Press, Cambridge,MA,2004.[10]John Duchi,Shai Shalev-Shwartz,Yoram Singer,and Tushar Chandra.Efficient pro-jections onto the l1-ball for learning in high dimensions.In Proceedings of the25th international conference on Machine learning,pages272–279,2008.[11]Matthias Heiler and Christoph Schn¨o rr.Learning sparse representations by non-negative matrix factorization and sequential cone programming.The Journal of Ma-chine Learning Research,7:2006,2006.[12]T.Hofmann.Unsupervised learning by probabilistic latent semantic analysis.MachineLearning,42(1):177–196,2001.[13]P.O.Hoyer.Non-negative sparse coding.In Neural Networks for Signal Processing,2002.Proceedings of the200212th IEEE Workshop on,pages557–565,2002.[14]Patrik O.Hoyer.Non-negative matrix factorization with sparseness constraints.J.Mach.Learn.Res.,5:1457–1469,December2004.[15]C.J.Hsieh and I.Dhillon.Fast coordinate descent methods with variable selection fornon-negative matrix factorization.ACM SIGKDD Internation Conference on Knowl-edge Discovery and Data Mining,pages1064–1072,2011.[16]Niall Hurley and Scott paring measures of sparsity.IEEE Trans.Inf.Theor.,55:4723–4741,October2009.[17]Hyunsoo Kim and Haesun Park.Sparse non-negative matrix factorizations via alter-nating non-negativity-constrained least squares for microarray data analysis.Bioinfor-matics,23(12):1495–1502,2007.[18]Jingu Kim and Haesun Park.Toward faster nonnegative matrix factorization:A newalgorithm and comparisons.Data Mining,IEEE International Conference on,0:353–362,2008.[19]wton and E.A.Sylvestre.Self modeling curve resolution.Technometrics,pages617–633,1971.[20]D.D.Lee and H.S.Seung.Learning the parts of objects by non-negative matrixfactorization.Nature,401(6755):788–791,October1999.[21]Daniel D.Lee and Sebastian H.Seung.Algorithms for non-negative matrix factoriza-tion.In NIPS,pages556–562,2000.[22]Chih-Jen Lin.Projected gradient methods for nonnegative matrix factorization.NeuralComp.,19(10):2756–2779,October2007.[23]Julien Mairal,Francis Bach,Jean Ponce,and Guillermo Sapiro.Online learning formatrix factorization and sparse coding.The Journal of Machine Learning Research, 11:19–60,2010.[24]Morten Mørup,Kristoffer Hougaard Madsen,and Lars Kai Hansen.Approximate L0constrained non-negative matrix and tensor factorization.In ISCAS,pages1328–1331, 2008.。