Eigenvectors and Reconstruction

合集下载

Document Clustering Using Locality Preserving Indexing

Xiaofei He Department of Computer Science The University of Chicago 1100 East 58th Street, Chicago, IL 60637, USA Phone: (733) 288-2851 xiaofei@
Jiawei Han Department of Computer Science University of Illinois at Urbana Champaign 2132 Siebel Center, 201 N. Goodwin Ave, Urbana, IL 61801, USA Phone: (217) 333-6903 Fax: (217) 265-6494 hanj@
document clustering [28][27]. They model each cluster as a linear combination of the data points, and each data point as a linear combination of the clusters. And they compute the linear coeﬃcients by minimizing the global reconstruction error of the data points using Non-negative Matrix Factorization. Thus, NMF method still focuses on the global geometrical structure of document space. Moreover, the iterative update method for solving NMF problem is computational expensive. In this paper, we propose a novel document clustering algorithm by using Locality Preserving Indexing (LPI). Diﬀerent from LSI which aims to discover the global Euclidean structure, LPI aims to discover the local geometrical structure. LPI can have more discriminating power. Thus, the documents related to the same semantics are close to each other in the low dimensional representation space. Also, LPI is derived by ﬁnding the optimal linear approximations to the eigenfunctions of the Laplace Beltrami operator on the document manifold. Laplace Beltrami operator takes the second order derivatives of the functions on the manifolds. It evaluates the smoothness of the functions. Therefore, it can discover the non-linear manifold structure to some extent. Some theoretical justiﬁcations can be traced back to [15][14]. The original LPI is not optimal in the sense of computation in that the obtained basis functions might contain a trivial solution. The trivial solution contains no information and thus useless for document indexing. A modiﬁed LPI is proposed to obtain better document representations. In this low dimensional space, we then apply traditional clustering algorithms such as k -means to cluster the documents into semantically diﬀerent classes. The rest of this paper is organized as follows: In Section 2, we give a brief review of LSI and LPI. Section 3 introduces our proposed document clustering algorithm. Some theoretical analysis is provided in Section 4. The experimental results are shown in Section 5. Finally, we give concluding remarks and future work in Section 6.

Eigenvalues,Eigenvectors

18.03 Class 32, May 1Eigenvalues and eigenvectors[1] Prologue on Linear Algebra.Recall [a b ; c d] [x ; y] = x[a ; c] + y[b ; d] :A matrix times a column vector is the linear combination ofthe columns of the matrix weighted by the entries in the column vector. When is this product zero?One way is for x = y = 0. If [a ; c] and [b ; d] point indifferent directions, this is the ONLY way. But if they lie along a single line, we can find x and y so that the sum cancels.Write A = [a b ; c d] and u = [x ; y] , so we have been thinking about A u = 0 as an equation in u . It always has the "trivial" solution u = 0 = [0 ; 0] : 0 is a linear combination of the two columns in a "trivial" way, with 0 coefficients, and we are asking when it is a linear combination of them in a different, "nontrivial" way.We get a nonzero solution [x ; y] exactly when the slopes of thevectors[a ; c] and [b ; d] coincide: combination of the entries in A "determinant" of the matrix:c/a = d/b , or ad - bc = 0. This is so important it's called thedet(A) = ad - bcWe have found:Theorem: Au = 0 has a nontrivial solution exactly when det A = 0 .If A is a larger *square* matrix the same theorem still holds, withthe appropriate definition of the number det A .[2] Solve u' = Au : for example with A = [1 2 ; 2 1] .The "Linear Phase Portraits: Matrix Entry" Mathlet shows that some trajectories seem to be along straight lines. Let's find them first. That is to say, we are going to look for a solution of the form u(t) = r(t) vOne thing for sure: the velocity vector u'(t) also points in the same (or reverse) direction as u(t). So for any vector v on this trajectory,A v = lambda vfor some number lambda. This Greek letter is always used in this context.[3] This is a pure linear algebra problem: A is a square matrix, andwe arelooking for nonzero vectors number v such that A v = lambda v for somelambda. In order to get all theside asv's together, write the right hand lambda v = (lambda I) vwhere I is the identity matrix [1 0 ; 0 1] , and lambda I is the matrix with lambda down the diagonal. Then we can put this on the left:0 = A v - (lambda I) v = (A - lambda I) vDon't forget, we are looking for a nonzero v . We have just found an exact condition for such a solution:det(A - lambda I) = 0This is an equation in lambda ; we will find lambda first, and then set about solving for v (knowing in advance only that there IS a nonzerosolution).In our example, then, we subtract lambda from both diagonal entries and then take the determanent:A - lambda I = [ 1 - lambda , 2 ; 2 , 1 - lambda ]det ( A - lambda I ) = (1-lambda)(1-lambda) - 4= 1 - 2 lambda + lambda^2 - 4= lambda^2 - 2 lambda - 3This is the "characteristic polynomial"p_A(lambda) = det( A - lambda I )of A , and its roots are the "characteristic values" or "eigenvalues" of A .In our case, p_A(lambda) = (lambda + 1)(lambda - 3)and there are two roots, lambda_1 = -1 and lambda_2 = 3 .[4] Now we can find those special directions. There is one line for lambda_1 and another for lambda_2 . We have to find nonzero solution v to(A - lambda I) v = 0eg with lambda = lambda_1 = -1 , A - lambda = [ 2 2 ; 2 2 ]There is a nontrivial linear relation between the columns:A [ 1 ; -1 ] = 0All we are claiming is thatA [ 1 ; -1 ] = - [ 1 ; -1 ]and you can check this directly. Any such v (even zero) is called an "eigenvector" of A.Back to the differential equation. We have found that there is astraight linesolution of the form r(t) v where v = [1;-1] . We have r' v = u' = A u = A rv = r A v = r lambda vso (since v is nonzero)r' = lambda rand solving this goes straight back to Day One:r = c e^{lambda t}so for us r = c e^{-t} and we have found our first straight line solution:u = e^{-t} [1;-1]In fact we've found all solutions which occur along that line: u = c e^{-t} [1;-1]Any one of these solutions is called a "normal mode."General fact: the eigenvalue turns out to play a much more important rolethan it looked like it would: the straight line solutions are*exponential*solutions, e^{lambda t} v , where lambda is an eigenvalue forthe matrix and v is a nonzero eigenvector for this eigenvalue.The second eigenvalue, lambda_2 = 3 , leads toA - lambda I = [ -1 1 ; 1 -1 ]and [ -1 1 ; 1 -1 ] v = 0 has nonzero solution v = [1;1]so [1;1] is a nonzero eigenvector for the eigenvalue lambda = 3 , and there is another straight line solutione^{3t} [1;1][5] The general solution to u' = Au will be a linear combination of the two eigensolutions (as long as there are two distinct eigenvalues). In our example, the general solution isu = c1 e^{-t} [1 ; -1] + c2 e^{3t} [1 ; 1]We can solve for c1 and c2 using an initial condition: say for exampleu(0) = [2 ; 0]. Well,u(0) = c1 [1 ; -1] + c2 [1 ; 1] = [c1+c2 ; -c1+c2]and for this to be [2 ; 0] we must have c1 = c2 = 1:u(t) = e^{-t} [1 ; -1] + e^{3t} [1 ; 1] .When t is very negative, -10, say, the first term is very big and the second tiny: the solution is very near the line through [1 ; -1].As t gets near zero, the two terms become comparable and the solution curves around. As t gets large, 10, say, the second term is very big and the first is tiny: the solution becomes asymptotic to the line through[1 ; 1].The general solution is a combination of the two normal modes.[6] Comments:(1) The characteristic polynomial for the general 2x2 matrix A = [a,b;c,d] isp_A(lambda) = (a-lambda)(d-lambda) - bc= lambda^2 - (a+d) lambda - (ad-bc)The sum of the diagonal terms of a square matrix is the "trace" of A , tr A,sop_A(lambda) = lambda^2 - (tr A) lambda + (det A)In our example, tr A = 2 and det A = 3 , andp_A(lambda) = lambda^2 - 2 lambda - 3 .(2) Any multiple of an eigenvector is another eigenvector for the same eigenvalue; they form a line, an "eigenline."(3) The eigenlines for distinct eigenvalues are not genearally perpendicularto each other; that is a special feature of *symmetric* matrices, those forwhich b = c .Also, generally the eigenvalues, roots of the characteristic polynomial, may be complex, not real. But for a symmetric matrix, all the eigenvaluesare real.Both these facts hold in higher dimensions as well. Most real numbers we know about are eigenvalues of symmetric matrices - the mass of an elementaryparticle, for example.。

正则化方法赫森矩阵 -回复

正则化方法赫森矩阵-回复什么是正则化方法赫森矩阵？在机器学习和统计学中，正则化是一种常用的方法，用于减小学习算法的复杂度并防止过拟合。

而赫森矩阵（Hessian Matrix）是用于描述多元函数的二阶偏导数的矩阵。

正则化方法和赫森矩阵在机器学习中扮演着重要的角色，通过对样本数据进行适当的惩罚，提高了机器学习算法的泛化能力，并减小了模型的复杂度。

本文将一步一步回答关于正则化方法赫森矩阵的问题，以帮助读者更好地理解它们的原理和应用。

第一部分：正则化方法1. 什么是正则化方法？正则化是一种用于减小模型复杂度并防止过拟合的技术。

它通过在损失函数中增加一个正则化项，对模型的参数进行约束，控制参数的大小，以避免模型在训练数据上过多地关注噪声或异常点，从而提高模型的泛化能力。

2. 为什么需要正则化方法？在机器学习中，模型在训练数据上可能会出现过拟合现象，即模型过于复杂，过度拟合了训练数据中的噪声和异常点，导致在新的数据上表现不佳。

正则化方法通过限制模型的参数大小，降低复杂度，提高了模型在新数据上的预测准确性。

3. 常见的正则化方法有哪些？常见的正则化方法包括L1正则化和L2正则化。

L1正则化通过在损失函数中加入模型参数的绝对值之和，并使得部分参数为0，从而实现特征选择和稀疏性；L2正则化则通过在损失函数中加入模型参数的平方和，并使得参数趋于较小的值，从而平滑模型的参数，并避免过拟合。

第二部分：赫森矩阵1. 什么是赫森矩阵？赫森矩阵是一个描述多元函数的二阶偏导数的矩阵。

对于一个具有n个自变量的函数，赫森矩阵是一个n×n的矩阵，其中每个元素是函数的二阶偏导数。

2. 赫森矩阵有什么作用？赫森矩阵提供了有关函数局部曲率变化的信息，可以帮助我们理解函数在某一点附近的趋势和形状。

尤其在求解优化问题时，赫森矩阵可以帮助我们确定函数的极值点以及该点的性质。

3. 如何计算赫森矩阵？为了计算一个多元函数的赫森矩阵，我们需要求解函数的所有一阶和二阶偏导数。

勤劳强国家的英语作文

Diligence is a cornerstone of success,and it is the key to building a strong nation.A nations prosperity and strength are not built overnight they are the result of the collective efforts of its people.Here is an essay on the importance of diligence in building a strong country.Title:The Role of Diligence in Building a Strong NationIn the tapestry of a nations development,diligence is the golden thread that weaves progress and strength.It is the unwavering commitment to hard work and perseverance that propels a country forward,fostering an environment where innovation,growth,and resilience can thrive.This essay delves into the significance of diligence in the construction of a robust and prosperous nation.Cultivating a Diligent Work EthicThe foundation of a strong nation lies in its people.A diligent work ethic is instilled from an early age,encouraging individuals to take pride in their contributions,no matter how small they may seem.Schools,families,and communities play a pivotal role in nurturing this ethos,teaching the value of time,the satisfaction of a job well done,and the importance of striving for excellence.Economic Growth and DevelopmentDiligence is the driving force behind economic growth.It fuels the engine of industry, where workers dedicate long hours to perfecting their crafts and innovating new solutions. This dedication leads to increased productivity,which in turn attracts investment,creates jobs,and raises the standard of living for all citizens.Innovation and Technological AdvancementA nation that values diligence is a nation that values knowledge and innovation.Diligent individuals are more likely to pursue education and research,pushing the boundaries of science and technology.This leads to breakthroughs that can revolutionize industries, improve public services,and enhance the overall quality of life.Social Cohesion and National IdentityDiligence also plays a crucial role in fostering social cohesion.When citizens see themselves as part of a collective effort to build a better future,it strengthens national identity and unity.This shared sense of purpose can overcome social divides and create amore harmonious society.Overcoming ChallengesNo nation is immune to challenges,whether they be economic downturns,natural disasters,or global crises.A diligent nation,however,is better equipped to face these adversities.The resilience born from a culture of hard work allows a country to bounce back stronger and more prepared for future obstacles.Global CompetitivenessIn an increasingly interconnected world,a nations diligence can be its greatest asset in the global marketplace.By maintaining high standards of work and a relentless pursuit of improvement,a country can outcompete its rivals,secure its place on the world stage,and ensure its longterm survival.ConclusionIn conclusion,diligence is the bedrock upon which a strong nation is built.It is the common thread that runs through economic prosperity,social harmony,and national pride.By fostering a culture of diligence,a country can ensure its continued growth, resilience,and success in an everchanging world.It is through the sweat of our collective brow that we pave the way for future generations to inherit a nation that is not only strong but also just,equitable,and prosperous.。

免疫算法基本流程 -回复

免疫算法基本流程 -回复免疫算法（Immune Algorithm，IA）是仿生学领域的一种元启发式算法，它模仿人类免疫系统的功能，用于解决复杂问题的优化问题。

其基本流程包括问题建模、个体编码、种群初始化、克隆操作、变异操作、选择操作等，接下来本文将从这些方面进一步展开详细描述。

一、问题建模在使用免疫算法解决优化问题之前，需要将问题进行合理的建模。

建模过程主要涉及问题的因素、目标和约束条件等问题，例如在TSP（Traveling Salesman Problem）中，需要定义地图中所有城市之间的距离以及行走路线的长度等因素。

建模完成后，将其转化为适合于免疫算法处理的数学表示形式，这有助于优化算法的精度和效率。

二、个体编码从问题建模后，需要将问题的变量转化为适合免疫算法处理的个体编码，即将问题的解转化成一些序列或数值，这样才能进行算法的操作。

对于不同的问题，需要设计合适的编码方式，例如对于TSP问题，可以将城市序列编码成01字符串等。

三、种群初始化在免疫算法中，需要构建一个种群，种群中的每个个体代表了问题的一个解。

种群初始化是在搜索空间中随机生成一组解，并且保证这些解满足约束条件。

种群大小需要根据问题规模和计算能力来合理安排，一般情况下，种群大小越大，搜索空间越大，但是计算成本也越高。

四、克隆操作在免疫算法中，克隆操作是其中一个重要的基因变异操作。

该操作的目的是产生大量近似于当前最优的个体，增加搜索空间的多样性。

克隆操作的流程如下：1.计算适应度函数值，根据适应度函数值进行排序。

2.选择适应度函数值最优的一部分个体进行克隆操作。

3.对克隆个体进行加密操作，增加其多样性。

5、变异操作变异操作是免疫算法中的一个基本操作，其目的是使部分克隆个体产生和原个体不同的搜索方向，增加搜索空间的变异性。

在变异操作中，采用随机、局部搜索或任意搜索等方法来对某些个体进行改变其参数或某些属性，以期望产生一些新的解。

变异操作的流程如下：1.从克隆群体中随机选择一定数量的个体进行变异操作。

reducedimension method

reducedimension method1. IntroductionThe dimensionality reduction technique known as "Reducedimension" refers to a method used to reduce the number of features in a dataset while preserving the most important information. It is commonly used in machine learning and data analysis tasks to overcome the curse of dimensionality and improve computational efficiency. In this article, we will discuss the principles and procedures of the Reducedimension method.2. Principles of ReducedimensionReducedimension is based on the assumption that the data lies on a low-dimensional manifold embedded in a high-dimensional space. The method aims to find a transformation that maps the original high-dimensional space to a lower-dimensional space while preserving the intrinsic structure and relationships of the data. By reducing the dimensionality, it becomes easier to visualize, analyze, and model the data.3. Procedure of ReducedimensionThe Reducedimension method can be performed in several steps:a. Data preprocessing: Before applying Reducedimension, it is necessary to preprocess the data. This includes handling missing values, normalizing or standardizing the features, and dealing with categorical variables. Data preprocessing ensures that the algorithm performs optimally.b. Covariance matrix computation: The covariance matrix is a symmetric positive semi-definite matrix that represents therelationship between the features in the dataset. It is computed to capture the linear dependencies between the variables.c. Eigenvalue decomposition: By decomposing the covariance matrix, we obtain its eigenvalues and eigenvectors. The eigenvalues represent the amount of variance explained by each eigenvector. The eigenvectors are the directions along which the data varies the most.d. Selection of principal components: The principal components are selected based on the eigenvalues. The eigenvectors corresponding to the largest eigenvalues capture most of the variance in the data. These eigenvectors are chosen as the principal components.e. Projection: The original high-dimensional data is projected onto the newly defined lower-dimensional space spanned by the principal components. The projection reduces the dimensionality of the data while preserving its essential properties and minimizing information loss.f. Reconstruction: If needed, the reduced-dimensional data can be reconstructed back into the original high-dimensional space using the inverse projection matrix. This allows for analysis or visualization in the original feature space.4. Advantages of ReducedimensionThe Reducedimension method offers several advantages:a. Dimensionality reduction: The method reduces thedimensionality of the dataset, which helps to overcome the curse of dimensionality. It simplifies the data representation and improves computational efficiency.b. Feature selection: Reducedimension automatically selects the most informative features by calculating the eigenvalues and eigenvectors. It eliminates redundant or irrelevant features, resulting in a more concise and interpretable dataset.c. Intrinsic structure preservation: Reducedimension aims to preserve the intrinsic structure and relationships of the data during the dimensionality reduction process. It ensures that the important information is retained while discarding noise and irrelevant variations.5. Applications of ReducedimensionThe Reducedimension method has various applications in various fields, including:a. Image and signal processing: It is used to reduce the dimensionality of image and signal data, enabling efficient compression, denoising, and feature extraction.b. Pattern recognition: Reducedimension is applied to extract discriminative features from high-dimensional datasets, improving the performance of pattern recognition algorithms.c. Bioinformatics: It is used to analyze and visualize genomic and proteomic data, enabling the identification of key genes or proteins associated with specific diseases or biological processes.d. Financial analysis: Reducedimension is used to analyze and model financial data, identifying the key factors that drive stock prices or predicting market trends.6. ConclusionThe Reducedimension method provides an effective approach for reducing the dimensionality of high-dimensional datasets while preserving the most relevant information. Its principles and procedures, including data preprocessing, covariance matrix computation, eigenvalue decomposition, principal component selection, projection, and reconstruction, enable efficient analysis, modeling, and visualization of complex datasets in various fields.。

A tutorial on Principal Components Analysis

A tutorial on Principal Components AnalysisLindsay I SmithFebruary26,2002Chapter1IntroductionThis tutorial is designed to give the reader an understanding of Principal Components Analysis(PCA).PCA is a useful statistical technique that has found application in ﬁelds such as face recognition and image compression,and is a common technique for ﬁnding patterns in data of high dimension.Before getting to a description of PCA,this tutorialﬁrst introduces mathematical concepts that will be used in PCA.It covers standard deviation,covariance,eigenvec-tors and eigenvalues.This background knowledge is meant to make the PCA section very straightforward,but can be skipped if the concepts are already familiar.There are examples all the way through this tutorial that are meant to illustrate the concepts being discussed.If further information is required,the mathematics textbook “Elementary Linear Algebra5e”by Howard Anton,Publisher John Wiley&Sons Inc, ISBN0-471-85223-6is a good source of information regarding the mathematical back-ground.1Chapter2Background MathematicsThis section will attempt to give some elementary background mathematical skills that will be required to understand the process of Principal Components Analysis.The topics are covered independently of each other,and examples given.It is less important to remember the exact mechanics of a mathematical technique than it is to understand the reason why such a technique may be used,and what the result of the operation tells us about our data.Not all of these techniques are used in PCA,but the ones that are not explicitly required do provide the grounding on which the most important techniques are based.I have included a section on Statistics which looks at distribution measurements, or,how the data is spread out.The other section is on Matrix Algebra and looks at eigenvectors and eigenvalues,important properties of matrices that are fundamental to PCA.2.1StatisticsThe entire subject of statistics is based around the idea that you have this big set of data, and you want to analyse that set in terms of the relationships between the individual points in that data set.I am going to look at a few of the measures you can do on a set of data,and what they tell you about the data itself.2.1.1Standard DeviationTo understand standard deviation,we need a data set.Statisticians are usually con-cerned with taking a sample of a population.To use election polls as an example,the population is all the people in the country,whereas a sample is a subset of the pop-ulation that the statisticians measure.The great thing about statistics is that by only measuring(in this case by doing a phone survey or similar)a sample of the population, you can work out what is most likely to be the measurement if you used the entire pop-ulation.In this statistics section,I am going to assume that our data sets are samples2of some bigger population.There is a reference later in this section pointing to more information about samples and populations.Here’s an example set:I could simply use the symbol to refer to this entire set of numbers.If I want to refer to an individual number in this data set,I will use subscripts on the symbol to indicate a speciﬁc number.Eg.refers to the3rd number in,namely the number4.Note that is theﬁrst number in the sequence,not like you may see in some textbooks.Also,the symbol will be used to refer to the number of elements in the setThere are a number of things that we can calculate about a data set.For example, we can calculate the mean of the sample.I assume that the reader understands what the mean of a sample is,and will only give the formula:Set1:Total208Square Root8.32668-249-1111111224Divided by(n-1) 3.333Table2.1:Calculation of standard deviationdifference between each of the denominators.It also discusses the difference between samples and populations.So,for our two data sets above,the calculations of standard deviation are in Ta-ble2.1.And so,as expected,theﬁrst set has a much larger standard deviation due to the fact that the data is much more spread out from the mean.Just as another example,the data set:also has a mean of10,but its standard deviation is0,because all the numbers are the same.None of them deviate from the mean.2.1.2VarianceVariance is another measure of the spread of data in a data set.In fact it is almost identical to the standard deviation.The formula is this:You will notice that this is simply the standard deviation squared,in both the symbol ()and the formula(there is no square root in the formula for variance).is the usual symbol for variance of a sample.Both these measurements are measures of the spread of the data.Standard deviation is the most common measure,but variance is also used.The reason why I have introduced variance in addition to standard deviation is to provide a solid platform from which the next section,covariance,can launch from. ExercisesFind the mean,standard deviation,and variance for each of these data sets.[12233444597098][12152527328899][15357882909597]协方差2.1.3CovarianceThe last two measures we have looked at are purely1-dimensional.Data sets like this could be:heights of all the people in the room,marks for the last COMP101exam etc. However many data sets have more than one dimension,and the aim of the statistical analysis of these data sets is usually to see if there is any relationship between the dimensions.For example,we might have as our data set both the height of all the students in a class,and the mark they received for that paper.We could then perform statistical analysis to see if the height of a student has any effect on their mark.Standard deviation and variance only operate on1dimension,so that you could only calculate the standard deviation for each dimension of the data set independently of the other dimensions.However,it is useful to have a similar measure toﬁnd out how much the dimensions vary from the mean with respect to each other.Covariance is such a measure.Covariance is always measured between2dimen-sions.If you calculate the covariance between one dimension and itself,you get the variance.So,if you had a3-dimensional data set(,,),then you could measure the covariance between the and dimensions,the and dimensions,and the and dimensions.Measuring the covariance between and,or and,or and would give you the variance of the,and dimensions respectively.The formula for covariance is very similar to the formula for variance.The formula for variance could also be written like this:5includegraphicscovPlot.psFigure2.1:A plot of the covariance data showing positive relationship between the number of hours studied against the mark receivedIt is exactly the same except that in the second set of brackets,the’s are replaced by ’s.This says,in English,“For each data item,multiply the difference between the value and the mean of,by the the difference between the value and the mean of. Add all these up,and divide by”.How does this work?Lets use some example data.Imagine we have gone into the world and collected some2-dimensional data,say,we have asked a bunch of students how many hours in total that they spent studying COSC241,and the mark that they received.So we have two dimensions,theﬁrst is the dimension,the hours studied, and the second is the dimension,the mark received.Figure2.2holds my imaginarydata,and the calculation of,the covariance between the Hours of study done and the Mark received.So what does it tell us?The exact value is not as important as it’s sign(ie.positive or negative).If the value is positive,as it is here,then that indicates that both di-mensions increase together,meaning that,in general,as the number of hours of study increased,so did theﬁnal mark.If the value is negative,then as one dimension increases,the other decreases.If we had ended up with a negative covariance here,then that would have said the opposite, that as the number of hours of study increased the theﬁnal mark decreased.In the last case,if the covariance is zero,it indicates that the two dimensions are independent of each other.The result that mark given increases as the number of hours studied increases can be easily seen by drawing a graph of the data,as in Figure2.1.3.However,the luxury of being able to visualize data is only available at2and3dimensions.Since the co-variance value can be calculated between any2dimensions in a data set,this technique is often used toﬁnd relationships between dimensions in high-dimensional data sets where visualisation is difﬁcult.You might ask“is equal to”?Well,a quick look at the for-mula for covariance tells us that yes,they are exactly the same since the only dif-ference between and is that is replaced by .And since multiplication is commutative,which means that it doesn’t matter which way around I multiply two numbers,I always get the same num-ber,these two equations give the same answer.2.1.4The covariance MatrixRecall that covariance is always measured between2dimensions.If we have a data set with more than2dimensions,there is more than one covariance measurement that can be calculated.For example,from a3dimensional data set(dimensions,,)you could calculate,,and.In fact,for an-dimensional data set,you can calculateHours(H)Mark(M)Totals167749 Covariance:939-23.421.08-6.93259330.580.08-0.111050-12.424.0851.33032-30.422.0846.97542-20.425.0838.511666 3.586.08106.891149.89104.54 Table2.2:2-dimensional data set and covariance calculation7A useful way to get all the possible covariance values between all the different dimensions is to calculate them all and put them in a matrix.I assume in this tutorial that you are familiar with matrices,and how they can be deﬁned.So,the deﬁnition for the covariance matrix for a set of data with dimensions is:where is a matrix with rows and columns,and is the th dimension. All that this ugly looking formula says is that if you have an-dimensional data set, then the matrix has rows and columns(so is square)and each entry in the matrix is the result of calculating the covariance between two separate dimensions.Eg.the entry on row2,column3,is the covariance value calculated between the2nd dimension and the3rd dimension.An example.We’ll make up the covariance matrix for an imaginary3dimensional data set,using the usual dimensions,and.Then,the covariance matrix has3rows and3columns,and the values are this:Some points to note:Down the main diagonal,you see that the covariance value is between one of the dimensions and itself.These are the variances for that dimension. The other point is that since,the matrix is symmetrical about the main diagonal.ExercisesWork out the covariance between the and dimensions in the following2dimen-sional data set,and describe what the result indicates about the data.Item Number:243923433220131411-1Figure2.2:Example of one non-eigenvector and one eigenvectorFigure2.3:Example of how a scaled eigenvector is still and eigenvector2.2.1EigenvectorsAs you know,you can multiply two matrices together,provided they are compatible sizes.Eigenvectors are a special case of this.Consider the two multiplications between a matrix and a vector in Figure2.2.In theﬁrst example,the resulting vector is not an integer multiple of the original vector,whereas in the second example,the example is exactly4times the vector we began with.Why is this?Well,the vector is a vector in2dimensional space.The vector(from the second example multiplication)represents an arrow pointing from the origin,,to the point.The other matrix,the square one,can be thought of as a transformation matrix.If you multiply this matrix on the left of a vector,the answer is another vector that is transformed from it’s original position.It is the nature of the transformation that the eigenvectors arise from.Imagine a transformation matrix that,when multiplied on the left,reﬂected vectors in the line .Then you can see that if there were a vector that lay on the line,it’s reﬂection it itself.This vector(and all multiples of it,because it wouldn’t matter howlong the vector was),would be an eigenvector of that transformation matrix.What properties do these eigenvectors have?You shouldﬁrst know that eigenvec-tors can only be found for square matrices.And,not every square matrix has eigen-vectors.And,given an matrix that does have eigenvectors,there are of them. Given a matrix,there are3eigenvectors.Another property of eigenvectors is that even if I scale the vector by some amount before I multiply it,I still get the same multiple of it as a result,as in Figure2.3.This is because if you scale a vector by some amount,all you are doing is making it longer,9not changing it’s stly,all the eigenvectors of a matrix are perpendicular,ie.at right angles to each other,no matter how many dimensions you have.By the way, another word for perpendicular,in maths talk,is orthogonal.This is important becauseit means that you can express the data in terms of these perpendicular eigenvectors,instead of expressing them in terms of the and axes.We will be doing this later in the section on PCA.Another important thing to know is that when mathematiciansﬁnd eigenvectors, they like toﬁnd the eigenvectors whose length is exactly one.This is because,as you know,the length of a vector doesn’t affect whether it’s an eigenvector or not,whereas the direction does.So,in order to keep eigenvectors standard,whenever weﬁnd an eigenvector we usually scale it to make it have a length of1,so that all eigenvectors have the same length.Here’s a demonstration from our example above.is an eigenvector,and the length of that vector isso we divide the original vector by this much to make it have a length of1.ExercisesFor the following square matrix:Decide which,if any,of the following vectors are eigenvectors of that matrix and give the corresponding eigenvalue.11Chapter3Principal Components Analysis Finally we come to Principal Components Analysis(PCA).What is it?It is a way of identifying patterns in data,and expressing the data in such a way as to highlight their similarities and differences.Since patterns in data can be hard toﬁnd in data of high dimension,where the luxury of graphical representation is not available,PCA is a powerful tool for analysing data.The other main advantage of PCA is that once you have found these patterns in the data,and you compress the data,ie.by reducing the number of dimensions,without much loss of information.This technique used in image compression,as we will see in a later section.This chapter will take you through the steps you needed to perform a Principal Components Analysis on a set of data.I am not going to describe exactly why the technique works,but I will try to provide an explanation of what is happening at each point so that you can make informed decisions when you try to use this technique yourself.3.1MethodStep1:Get some dataIn my simple example,I am going to use my own made-up data set.It’s only got2 dimensions,and the reason why I have chosen this is so that I can provide plots of the data to show what the PCA analysis is doing at each step.The data I have used is found in Figure3.1,along with a plot of that data.Step2:Subtract the meanFor PCA to work properly,you have to subtract the mean from each of the data dimen-sions.The mean subtracted is the average across each dimension.So,all the values have(the mean of the values of all the data points)subtracted,and all the values have subtracted from them.This produces a data set whose mean is zero.12Data =x2.50.72.22.23.12.721.11.50.9DataAdjust =x .69-1.21.39.291.29.79.19-.81-.31-1.01-101234-101234Original PCA data"./PCAdata.dat"Figure 3.1:PCA example data,original data on the left,data with the means subtracted on the right,and a plot of the data13Step3:Calculate the covariance matrixThis is done in exactly the same way as was discussed in section2.1.4.Since the data is2dimensional,the covariance matrix will be.There are no surprises here,so I will just give you the result:So,since the non-diagonal elements in this covariance matrix are positive,we should expect that both the and variable increase together.Step4:Calculate the eigenvectors and eigenvalues of the covariance matrixSince the covariance matrix is square,we can calculate the eigenvectors and eigenval-ues for this matrix.These are rather important,as they tell us useful information about our data.I will show you why soon.In the meantime,here are the eigenvectors and eigenvalues:It is important to notice that these eigenvectors are both unit eigenvectors ie.their lengths are both1.This is very important for PCA,but luckily,most maths packages, when asked for eigenvectors,will give you unit eigenvectors.So what do they mean?If you look at the plot of the data in Figure3.2then you can see how the data has quite a strong pattern.As expected from the covariance matrix, they two variables do indeed increase together.On top of the data I have plotted both the eigenvectors as well.They appear as diagonal dotted lines on the plot.As stated in the eigenvector section,they are perpendicular to each other.But,more importantly, they provide us with information about the patterns in the data.See how one of the eigenvectors goes through the middle of the points,like drawing a line of bestﬁt?That eigenvector is showing us how these two data sets are related along that line.The second eigenvector gives us the other,less important,pattern in the data,that all the points follow the main line,but are off to the side of the main line by some amount.So,by this process of taking the eigenvectors of the covariance matrix,we have been able to extract lines that characterise the data.The rest of the steps involve trans-forming the data so that it is expressed in terms of them lines.Step5:Choosing components and forming a feature vectorHere is where the notion of data compression and reduced dimensionality comes into it.If you look at the eigenvectors and eigenvalues from the previous section,you14-2-1.5-1-0.50.511.52-2-1.5-1-0.500.51 1.52Mean adjusted data with eigenvectors overlayed"PCAdataadjust.dat"(-.740682469/.671855252)*x (-.671855252/-.740682469)*x Figure 3.2:A plot of the normalised data (mean subtracted)with the eigenvectors of the covariance matrix overlayed on top.15will notice that the eigenvalues are quite different values.In fact,it turns out that the eigenvector with the highest eigenvalue is the principle component of the data set. In our example,the eigenvector with the larges eigenvalue was the one that pointed down the middle of the data.It is the most signiﬁcant relationship between the data dimensions.In general,once eigenvectors are found from the covariance matrix,the next step is to order them by eigenvalue,highest to lowest.This gives you the components in order of signiﬁcance.Now,if you like,you can decide to ignore the components of lesser signiﬁcance.You do lose some information,but if the eigenvalues are small,you don’t lose much.If you leave out some components,theﬁnal data set will have less dimensions than the original.To be precise,if you originally have dimensions in your data,and so you calculate eigenvectors and eigenvalues,and then you choose only theﬁrst eigenvectors,then theﬁnal data set has only dimensions.What needs to be done now is you need to form a feature vector,which is just a fancy name for a matrix of vectors.This is constructed by taking the eigenvectors that you want to keep from the list of eigenvectors,and forming a matrix with these eigenvectors in the columns.Given our example set of data,and the fact that we have2eigenvectors,we have two choices.We can either form a feature vector with both of the eigenvectors:or,we can choose to leave out the smaller,less signiﬁcant component and only have a single column:We shall see the result of each of these in the next section.Step5:Deriving the new data setThis theﬁnal step in PCA,and is also the easiest.Once we have chosen the components (eigenvectors)that we wish to keep in our data and formed a feature vector,we simply take the transpose of the vector and multiply it on the left of the original data set, transposed.where is the matrix with the eigenvectors in the columns trans-posed so that the eigenvectors are now in the rows,with the most signiﬁcant eigenvec-tor at the top,and is the mean-adjusted data transposed,ie.the data items are in each column,with each row holding a separate dimension.I’m sorry if this sudden transpose of all our data confuses you,but the equations from here on are16easier if we take the transpose of the feature vector and the dataﬁrst,rather that having a little T symbol above their names from now on.is theﬁnal data set,with data items in columns,and dimensions along rows.What will this give us?It will give us the original data solely in terms of the vectors we chose.Our original data set had two axes,and,so our data was in terms of them.It is possible to express data in terms of any two axes that you like.If these axes are perpendicular,then the expression is the most efﬁcient.This was why it was important that eigenvectors are always perpendicular to each other.We have changed our data from being in terms of the axes and,and now they are in terms of our2 eigenvectors.In the case of when the new data set has reduced dimensionality,ie.we have left some of the eigenvectors out,the new data is only in terms of the vectors that we decided to keep.To show this on our data,I have done theﬁnal transformation with each of the possible feature vectors.I have taken the transpose of the result in each case to bring the data back to the nice table-like format.I have also plotted theﬁnal points to show how they relate to the components.In the case of keeping both eigenvectors for the transformation,we get the data and the plot found in Figure3.3.This plot is basically the original data,rotated so that the eigenvectors are the axes.This is understandable since we have lost no information in this decomposition.The other transformation we can make is by taking only the eigenvector with the largest eigenvalue.The table of data resulting from that is found in Figure3.4.As expected,it only has a single dimension.If you compare this data set with the one resulting from using both eigenvectors,you will notice that this data set is exactly the ﬁrst column of the other.So,if you were to plot this data,it would be1dimensional, and would be points on a line in exactly the positions of the points in the plot in Figure3.3.We have effectively thrown away the whole other axis,which is the other eigenvector.So what have we done here?Basically we have transformed our data so that is expressed in terms of the patterns between them,where the patterns are the lines that most closely describe the relationships between the data.This is helpful because we have now classiﬁed our data point as a combination of the contributions from each of those lines.Initially we had the simple and axes.This isﬁne,but the and values of each data point don’t really tell us exactly how that point relates to the rest of the data.Now,the values of the data points tell us exactly where(ie.above/below)the trend lines the data point sits.In the case of the transformation using both eigenvectors, we have simply altered the data so that it is in terms of those eigenvectors instead of the usual axes.But the single-eigenvector decomposition has removed the contribution due to the smaller eigenvector and left us with data that is only in terms of the other.3.1.1Getting the old data backWanting to get the original data back is obviously of great concern if you are using the PCA transform for data compression(an example of which to will see in the next section).This content is taken fromhttp://www.vision.auc.dk/sig/Teaching/Flerdim/Current/hotelling/hotelling.html17Transformed Data=-.827970186.142857227-.992197494.130417207-1.67580142.175282444.0991094375.0464172582.438046137-.162675287-2-1.5-1-0.50.511.52-2-1.5-1-0.500.51 1.52Data transformed with 2 eigenvectors"./doublevecfinal.dat"Figure 3.3:The table of data by applying the PCA analysis using both eigenvectors,and a plot of the new data points.18Transformed Data(Single eigenvector)-101234-101234Original data restored using only a single eigenvector"./lossyplusmean.dat"Figure 3.5:The reconstruction from the data that was derived using only a single eigen-vectorit to the original data plot in Figure 3.1and you will notice how,while the variationalong the principle eigenvector (see Figure 3.2for the eigenvector overlayed on top ofthe mean-adjusted data)has been kept,the variation along the other component (theother eigenvector that we left out)has gone.ExercisesWhat do the eigenvectors of the covariance matrix give us?At what point in the PCA process can we decide to compress the data?Whateffect does this have?For an example of PCA and a graphical representation of the principal eigenvec-tors,research the topic ’Eigenfaces’,which uses PCA to do facial recognition20Chapter4Application to Computer VisionThis chapter will outline the way that PCA is used in computer vision,ﬁrst showing how images are usually represented,and then showing what PCA can allow us to dowith those images.The information in this section regarding facial recognition comes from“Face Recognition:Eigenface,Elastic Matching,and Neural Nets”,Jun Zhang etal.Proceedings of the IEEE,V ol.85,No.9,September1997.The representation infor-mation,is taken from“Digital Image Processing”Rafael C.Gonzalez and Paul Wintz, Addison-Wesley Publishing Company,1987.It is also an excellent reference for further information on the K-L transform in general.The image compression information is taken from http://www.vision.auc.dk/sig/Teaching/Flerdim/Current/hotelling/hotelling.html, which also provides examples of image reconstruction using a varying amount of eigen-vectors.4.1RepresentationWhen using these sort of matrix techniques in computer vision,we must consider repre-sentation of images.A square,by image can be expressed as an-dimensional vectorwhere the rows of pixels in the image are placed one after the other to form a one-dimensional image.E.g.Theﬁrst elements(will be theﬁrst row of the image,the next elements are the next row,and so on.The values in the vector arethe intensity values of the image,possibly a single greyscale value.4.2PCA toﬁnd patternsSay we have20images.Each image is pixels high by pixels wide.For each image we can create an image vector as described in the representation section.Wecan then put all the images together in one big image-matrix like this:21which gives us a starting point for our PCA analysis.Once we have performed PCA, we have our original data in terms of the eigenvectors we found from the covariance matrix.Why is this useful?Say we want to do facial recognition,and so our original images were of peoples faces.Then,the problem is,given a new image,whose face from the original set is it?(Note that the new image is not one of the20we started with.)The way this is done is computer vision is to measure the difference between the new image and the original images,but not along the original axes,along the new axes derived from the PCA analysis.It turns out that these axes works much better for recognising faces,because the PCA analysis has given us the original images in terms of the differences and simi-larities between them.The PCA analysis has identiﬁed the statistical patterns in the data.Since all the vectors are dimensional,we will get eigenvectors.In practice, we are able to leave out some of the less signiﬁcant eigenvectors,and the recognition still performs well.4.3PCA for image compressionUsing PCA for image compression also know as the Hotelling,or Karhunen and Leove (KL),transform.If we have20images,each with pixels,we can form vectors, each with20dimensions.Each vector consists of all the intensity values from the same pixel from each picture.This is different from the previous example because before we had a vector for image,and each item in that vector was a different pixel,whereas now we have a vector for each pixel,and each item in the vector is from a different image.Now we perform the PCA on this set of data.We will get20eigenvectors because each vector is20-dimensional.To compress the data,we can then choose to transform the data only using,say15of the eigenvectors.This gives us aﬁnal data set with only15dimensions,which has saved us of the space.However,when the original data is reproduced,the images have lost some of the information.This compression technique is said to be lossy because the decompressed image is not exactly the same as the original,generally worse.22。

全模型迭代重建在头颈部CT血管成像中的应用

国际医学放射学杂志ＩｎｔＪＭｅｄＲａｄｉｏｌ２０19July 鸦42穴4雪：418-421全模型迭代重建在头颈部CT 血管成像中的应用吴晓玲黄飚*【摘要】全模型迭代重建（IMR ）在重建过程中建立图像统计模型和数据统计模型，通过反复进行扫描模型与采集数据之间的差异对比校正，实现影像最优化。

头颈部CT 血管成像采用IMR 技术能显著降低影像噪声、提高影像信噪比和对比噪声比，可增强对脑动脉硬化和动脉瘤的诊断信心；还能提高对早期缺血性卒中动脉致密征检出的敏感度、大幅降低CT 灌注的辐射剂量、减少CT 血管成像对比剂用量以及降低病人的碘负荷。

综述IMR 技术在头颈部CT 血管成像中的应用进展。

【关键词】全模型迭代重建技术；CT 血管成像；辐射剂量中图分类号：R445.3；R814.42文献标志码：AApplication of knowledge-based iterative model reconstruction in head-neck CT angiography WU Xiaoling,HUANG Biao.Department of Radiology,Guangdong General Hospital,School of Medicine,South China University of Technology,Guangzhou 510080,China【Abstract 】Iterative model reconstruction (IMR)establishes image statistics and data statistics models during itsiterative cycle and achieves image optimization by repeatedly correcting the differences between scan model and acquired data.IMR can significantly reduce image noise,improve image signal-to-noise ratio and contrast-to-noise ratio for head and neck CTA,and increase the diagnostic confidence of atherosclerosis and aneurysm.IMR can also increase sensitivity in detecting hyperdense artery sign in the early ischemic stroke,significantly reduce the radiation dose in CT perfusion and the iodine load of patients.In this paper,the progress of application of iterative model reconstruction technology in head and neck CT angiography was reviewed.【Keywords 】Iterative model reconstruction;CT angiography;Radiation doseＩｎｔＪＭｅｄＲａｄｉｏｌ，２０１9，42（4）：418-421作者单位：华南理工大学附属广东省人民医院放射科，广州510080通信作者：黄飚，E-mail:cjr.huangbiao@ *审校者DOI:10.19300/j.2019.Z6654CT 血管成像（CT angiography ，CTA ）具有无创、简便、快速等优点，对于脑血管疾病的诊断和评估具有重要价值。

1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。

Abstract In this paper, we study the simple eigenvectors of two hypomorphic matrices using linear algebra. We also give new proofs of results of Godsil and McKay.
1
Introduction
We start by ﬁxing some notations ( [HE1]). Let A be a n × n real symmetric matrix. Let Ai be the matrix obtaining by deleting the i-th row and i-th column of A. We say that two symmetric matrices A and B are hypomorphic if, for each i, Bi can be obtained by simultaneously permuting the rows and columns of Ai . Let Σ be the set of permutations. We write B = Σ(A). If M is a symmetric real matrix, then the eigenvalues of M are real. We write eigen(M ) = (λ1 (M ) ≥ λ2 (M ) ≥ . . . ≥ λn (M )). If α is an eigenvalue of M , we denote the corresponding eigenspace by eigenα (M ). Let 1 be the n-dimensional vector (1, 1, . . . , 1). Put J = 1t 1. In [HE1], we proved the following theorem. Theorem 1 ( [HE1]) Let B and A be two real n × n symmetric matrices. Let Σ be a hypomorphism such that B = Σ(A). Let t be a real number. Then there exists an open interval T such that for t ∈ T we have 1. λn (A + tJ) = λn (B + tJ); 2. eigenλn (A + tJ) and eigenλn (B + tJ) are both one dimensional;
பைடு நூலகம்
Deleting the m-th row and m-th column, we obtain λ1 − λ i · · · p1,1 · · · p1,i · · · p1,n . .. . . . .. . .. . . . . . . . . . . pm,1 · · · pm,i · · · pm,n 0 ··· . . . . . . .. . .. . .. . . . . . . . pn,1 · · · pn,i · · · pn,n 0 ··· p1,1 · · · .. . . . . p1,i · · · . .. . . . p1,n · · · pm,1 · · · . .. . . .
j =i
(λj − λi )pj pt j.
the electronic journal of combinatorics 14 (2007), #N14
2
which equals p1,1 p2,1 . . . pn,1 ··· ··· .. . p1,i · · · p2,i · · · . .. . . . · · · pn,i · · · λ1 − λ i · · · p1,n . .. . . . p2,n 0 ··· . . . . .. . . . pn,n 0 ··· p2,1 · · · . .. . . . p2,i · · · . .. . . . p2,n 0 . . . λi − λ i . . . 0 0 . . . 0 . . . · · · λ n − λi ··· .. . ··· .. .
p1,1 . . . p1,i . . . p1,n
pn,1 . . . pn,i . . . . · · · pn,n 0 . . . λi − λ i . . . 0
pm,i . . . pm,n
This is Am − λi In−1 . Notice that P is orthogonal. Let Pm,i be the matrix obtained by 2 deleting the m-th row and i-th column. Then det Pm,i = p2 m,i where pm,i is the (m, i)-th entry of P . Taking the determinant, we have det(Am − λi In−1 ) = p2 m,i
2
Reconstruction of Square Functions
Theorem 2 Let A be a n × n real symmetric matrix. Let (λ1 ≥ λ2 ≥ · · · ≥ λn ) be the eigenvalues of A. Suppose λi is a simple eigenvalue of A. Let pi = (p1,i , p2,i , . . . , pn,i )t be a unit vector in eigenλi (A). Then for every m, p2 m,i can be expressed as a function of eigen(A) and eigen(Am ). Proof: Let λi be a simple eigenvalue of A. Let pi = (p1,i , p2,i , . . . , pn,i )t be a unit vector in eigenλi (A). There exists an orthogonal matrix P such that P = (p1 , p2 , · · · , pn ) and A = P DP t where λ1 0 · · · 0 0 λ2 · · · 0 D= . . . . . . . . . . . . . 0 0 · · · λn Then A − λi I = P DP t − λi I = P (D − λi I )P t =
Eigenvectors and Reconstruction
Hongyu He
∗
Department of Mathematics Louisiana State University, Baton Rouge, USA hongyu@ Submitted: Jul 6, 2006; Accepted: Jun 14, 2007; Published: Jul 5, 2007 Mathematics Subject Classiﬁcation: 05C88
∗
I would like to thank the referee for his valuable comments. 1
the electronic journal of combinatorics 14 (2007), #N14
3. eigenλn (A + tJ) = eigenλn (B + tJ). As proved in [HE1], our result implies Tutte’s theorem which says that eigen(A + tJ ) = eigen(B + tJ ). So det(A + tJ − λI ) = det(B + tJ − λI ). In this paper, we shall study the eigenvectors of A and B . Most of the results in this paper are not new. Our approach is new. We apply Theorem 1 to derive several wellknown results. We ﬁrst prove that the squares of the entries of simple unit eigenvectors of A can be reconstructed as functions of eigen(A) and eigen(Ai ). This yields a proof of a Theorem of Godsil-McKay. We then study how the eigenvectors of A change after a perturbation of rank 1 symmetric matrices. Combined with Theorem 1, we prove another result of Godsil-McKay which states that the simple eigenvectors that are perpendicular to 1 are reconstructible. We further show that the orthogonal projection of 1 onto higher dimensional eigenspaces is reconstructible. Our investigation indicates that the following conjecture could be true. Conjecture 1 Let A be a real n × n symmetric matrix. Then there exists a subgroup G(A) ⊆ O (n) such that a real symmetric matrix B satisﬁes the properties that eigen(B ) = eigen(A) and eigen(Bi ) = eigen(Ai ) for each i if and only if B = U AU t for some U ∈ G(A). This conjecture is clearly true if rank (A) = 1. For rank (A) = 1, the group G(A) can be chosen as Zn 2 , all in the form of diagonal matrices. In some other cases, G(A) can be a subgroup of the permutation group Sn .