人工神经网络外文翻译

附录二英文参考文献原文

Artificial Neural Networks

Artificial Neural Networks - Basic Features

Composed of a large number of processing units connected by a nonlinear, adaptive information processing system. It is the basis for modern neuroscience research findings presented, trying to simulate a large neural network processing, memory, information processing way of information. Artificial neural network has four basic characteristics:

(1) non-linear non-linear relationship is the general characteristics of the natural world. The wisdom of the brain is a nonlinear phenomenon. Artificial neural activation or inhibition in two different states, this behavior mathematically expressed as a linear relationship. Threshold neurons have a network with better performance, can improve fault tolerance and storage capacity.

(2) non-limitation of a neural network is usually more extensive neuronal connections made. The overall behavior of a system depends not only on the characteristics of single neurons, and may primarily by interaction between units, connected by the decision. By a large number of connections between the cells of non-simulated brain limitations. Associative memory limitations of a typical example of non-

(3) characterization of artificial neural network is adaptive, self-organizing, self-learning ability. Neural networks can not only deal with the changes of information, but also process information the same time, nonlinear dynamic system itself is also changing. Iterative process is frequently used in describing the evolution of dynamical systems.

(4) Non-convexity of the direction of the evolution of a system, under certain conditions, will depend on a particular state function. Such as energy function, and its extreme value corresponding to the state of the system more stable. Non-convexity of this function is more than one extremum, this system has multiple stable equilibrium, which will cause the system to the evolution of diversity.

Artificial neural network, neural processing unit can be expressed in different objects, such as features, letters, concepts, or some interesting abstract patterns. The type of network processing unit is divided into three categories: input units, output units and hidden units. Input unit receiving the signal and data outside world; output unit for processing the results to achieve the output; hidden unit is in between the input and output units can not be observed from outside the system unit. Neurons and the connection weights reflect the strength of the connections between elements of information representation and processing reflected in the network processing unit

connected relationships. Artificial neural network is a non-procedural, adaptability, the brain's information processing style, its essence is transformation through the network and dynamic behavior is a parallel distributed information processing, and to varying degrees and levels mimic brain information processing system. It is involved in neural science, thinking, science and artificial intelligence, computer science and other interdisciplinary fields. Artificial neural networks are parallel distributed systems, using traditional artificial intelligence and information processing technology is completely different mechanism to overcome the traditional symbol of artificial intelligence-based logic in dealing with intuition, unstructured information deficiencies, adaptive, Self-organization and the characteristics of real-time learning.

Artificial Neural Network – History

In 1943, psychologist WSMcCulloch mathematical logician W. Pitts neural network and the establishment of a mathematical model, called the MP model. They put forward by MP model neurons and network structure of formal mathematical description of methods, that a single neuron can perform logic functions, thus creating the era of artificial neural network. In 1949, psychologists proposed the idea of synaptic strength variable. 60 years, artificial neural network to the further development of improved neural network models have been proposed, including the sensors and the adaptive linear element, etc.. M. Minsky and so careful analysis of the sensor represented by the neural network system capabilities and limitations, the in 1969 published a "Perceptron" book, pointed out that the sensor can not solve the issue of higher order predicate. Their argument has greatly influenced research in neural networks, combined with serial computers and artificial intelligence at the achievements made to cover up the development of new computer and artificial intelligence, new ways of necessity and urgency to the research of artificial neural networks at a low ebb . In the meantime, some artificial neural network remains committed to the study, researchers proposed to adapt resonance theory (ART Wang), Zi Zuzhiyingshe, Ren Zhi machine network, while for the neural network Shuxue research. More research and development of neural network research foundation. In 1982, California Institute of Technology physicist JJHopfield proposed Hopfield neural grid model, the concept of "computational energy" concept gives the network stability of the judge. In 1984, he made continuous time Hopfield neural network model for the neural computer research done pioneering work to create a neural network for associative memory and optimization of computing new ways to effectively promote the study of neural networks, In 1985, there are scholars of the Boltzmann model, the use of statistical thermodynamics in the study simulated annealing technology to ensure the overall stability of the whole system tends to point. 1986 to study the microstructure of cognition, proposed the theory of parallel distributed processing. Artificial neural networks in various countries of the importance attached by the U.S. Congress passed a resolution to January 5, 1990 began with a decade as the "brain decade", the international research organization called on its members to "brain of the 10 in "to the global behavior. In Japan's

"real-world computing (RWC)" project, the artificial intelligence research has become an important part.

The main consideration of artificial neural network model topology of the network connection, the characteristics of neurons, learning rules. Currently, nearly 40 kinds of neural network model, including back propagation network, perceptron, self organizing maps, Hopfield networks, Boltzmann machines, meet the resonance theory. According to the connection topology, neural network model can be divided into:

(1) before the network to the network each neuron for the former level of input and output to the next level, network, no feedback, you can use a directed loop-free graph. This network signal from the input space and output space transform its information processing capabilities from a simple nonlinear function of several complex. Network structure is simple, easy to implement. Back-propagation network is a typical feedforward network.

(2) feedback network network neurons have feedback, you can use an undirected complete graph. This neural network information processing is the state of transformation, you can deal with dynamic systems theory. Stability of the system is closely related with the associative memory function. Hopfield networks, Boltzmann machines belong to this category.

Neural network learning is an important part of its adaptability is achieved through learning. According to the environmental changes in the value of the right to make adjustments to improve the system's behavior. Hebb proposed by the Hebb learning rule neural network learning algorithm for the foundation. Hebb learning rule that eventually occurred in the synapses between neurons, synaptic contacts with the synaptic strength of neuronal activity before and after the change. On this basis, it proposed a variety of learning rules and algorithms to meet the needs of different network models. Effective learning algorithm, making neural network connection weights through the adjustment, construction of the objective world of the intrinsic representation, forming a unique information processing method, information Cunchu and processing reflected in the network connection.

According to different learning environment, neural network learning methods can be divided into supervised learning and unsupervised learning. In monitoring the study, the training sample data added to the network input side, while the corresponding desired output and network output compared Dedao error signal, thereby Kongzhiquanzhi connection strength adjustments, Jing Hou several training convergence to a determine the weights. When the sample situation changes, the weights can be modified by learning to adapt to the new environment. Use supervised learning back propagation neural network model has the network, and HC. Non-supervised learning, the prior is not a given standard sample placed directly into the network environment, learning stage and become one session. At this point, learn

to obey the law of change in the evolution equation of connection weights. Non-supervised learning the most simple example is the Hebb learning rule. Competitive learning rule is a more complex example of non-supervised learning, which is based on the established clustering weight adjustment. Self-organizing map, resonance theory network to adapt to competition learn about all the typical model.

Study of nonlinear dynamics of neural networks, the main use of dynamic systems theory, nonlinear programming theory and statistical theory to analyze the evolutionary neural network attractor nature of the process and to explore the neural network of collaborative behavior and collective computing to understand neural information processing mechanism. To investigate the neural networks and fuzzy in the integrity of information in terms of dealing with the possibility of chaos theory concepts and methods could be useful. Chaos is a very difficult to precisely defined mathematical concept. Generally speaking, the "chaos" refers to the dynamic system described by deterministic equations demonstrated the non-deterministic behavior, or call to determine the randomness. "Certainty" because it causes from within and not outside noise or interference generated, and "random" refers to the irregular, unpredictable behavior, can only be described using statistical methods. The main features of chaotic dynamic system is its state of sensitive dependence on initial conditions, reflect the inherent randomness of chaos. Chaos theory is the description of chaotic behavior with the basic theory of nonlinear dynamical systems, concepts, methods, and it is to understand the dynamic complexity of the system for their own actions with the outside world and its material, energy and information exchange process in the inner structured, and not external and accidental behavior, chaotic state is a steady state. Steady state chaotic dynamical systems, including: static, steady volume, periodicity, quasi-period and chaotic solution. Lorenz is the overall stability and local instability in the results of the combination, called the strange attractor. A strange attractor has the following characteristics: (1) strange attractor is an attractor, but it is neither fixed points nor periodic solutions; (2) strange attractor is indivisible, that can not be divided into two and two or more attractors; (3) is very sensitive to its initial value, different initial values will lead to very different behavior.

Artificial neural network - advantages

Artificial neural network features and advantages mainly in three aspects:

First, self-learning function. For example, in pattern recognition, only the first of many different images in the model and the corresponding results should be identified input of artificial neural network, the network will be through self-learning function, and slowly learn to identify similar images. Self-learning function for the forecasts of particular importance. Expected future human artificial neural network computer will provide economic forecast, market forecasting, prediction efficiency, the application of the future is very bright.

Second, with the association storage. Artificial neural network feedback network can achieve this association.

Third, find the optimal solution with high capacity. Find the optimal solution of a complex problem, often requires a great amount of computation, the use of a problem for the design of a feedback type artificial neural network, play a computer's high-speed computing power, may soon find the optimal solution.

Artificial neural network - research

Artificial Neural Networks

Neural network research can be divided into theoretical and applied research on two areas.

Theory can be divided into the following categories:

1, using neuropsychological and cognitive science of human thinking and intelligence mechanism.

2, the neural basis of theoretical research, using mathematical methods to explore a more complete functional performance, superior of the neural network model, in-depth study and performance of network algorithms, such as: stability, convergence, fault tolerance, robustness, etc.; Development mathematical theory of the new network, such as: neural network dynamics, nonlinear neural farms.

Applied research can be divided into the following categories:

1, neural network software simulation and hardware implementation of the study.

2, neural network applications in all areas of research. These areas include:

Pattern recognition, signal processing, knowledge engineering, expert system, optimize, robot control. With the neural network theory and the theory itself, the continuous development of relevant technology, application of neural networks will be more in-depth.

Artificial Neural Networks - Application

Neural networks more and more attention recently because it was solved by the complexity of the problem provides a relatively simple and effective way. Neural networks can easily solve problems with hundreds of parameters (of course the actual existence of the neural network organisms than the procedures described here simulated neural networks are much more complex). Neural network used in two problems: classification and regression. In the structure, a neural network can be divided into input layer, output layer and hidden layer (see Figure 1). Each input layer

node corresponds to a forecast of a variable. Output layer of nodes corresponding to the target variable can have more than one. In the input layer and output layer is hidden layer (the neural network is not visible for the user), number of layers and each layer of hidden layer nodes determines the number of neural network complexity.

Figure 1 A neural network

In addition to the input layer nodes, neural network, each node in front of it with a lot of nodes (called the node of input nodes) connected together, each connection corresponds to a weight Wxy, the value of this node is the node through which all input The value of the weights with the corresponding product and as a function of the input and get our activities in this function is called function or squeezing function. Figure 2, the output node 4 to node 6, the value can be calculated as follows:

W14 * Node 1, the value of the + W24 * value of node 2

Neural network each node can be expressed as predictor variables (nodes 1,2) value or the value of the portfolio (nodes 3-6). Attention to the value of the node 6 is no longer a linear combination of node 1,2, because the data passed in the hidden layer using the activity function. In fact, if there are no active function, then neural network is equivalent to a linear regression function, if the activity of a particular function is a nonlinear function, neural network that it is equivalent to logistic regression.

Adjust the weights of connections between nodes is established (also called training) neural network to do work. The first and most basic method of weight adjustment feedback law is wrong, and now there are more changes in the new gradient method, Newton method, Levenberg-Marquardt method, and genetic algorithms. Regardless of the kind of training methods, we need to have some parameters to control the training process, such as training to prevent overtraining and control the pace.

Figure 2 weighted Wxy the neural network

Decided to neural network topology (or architecture) is contained in the hidden layer and the number of nodes and connections between nodes. To design a neural network from scratch, have to decide the number of hidden layers and nodes, activities in the form of the function, as well as the right to redo that limit, of course, if using sophisticated software tools, he will help you decide these things. In many types of neural networks, the most commonly used to spread the former type neural network, which is ahead of us the kind depicted in the icon. People talk in detail below, in order to facilitate the discussion assumes that only contain a layer of hidden nodes. That the error feedback type training method is the change in slope of the simplified method, the process is as follows: prior to the dissemination of: data from input to output of the process is a front to back of the transmission process, after the value of a node connected to the node through which pass in front of come, then the values of the

weights in accordance with the size of all the weighted input event function and then get the new value, and further spread to the next node.

Feedback: When the output value of the node we expect the value of the difference, that is, an error occurs, the neural network would "learn" (learning from mistakes). We can weight between nodes connected as a node after the previous node on the "trust" level (of his own down the output of a node in front of him which is more vulnerable to the impact of input nodes). Learning are punitive approach, the process is as follows: If a node output error, then he see his error by which (some) input node of the impact caused, is not his most trusted node (the highest weight of node) in framing him (make him wrong), if the trust will have to lower his value (reducing weight), punish them, while those who make the right recommendations increase the trust value of nodes. Nodes who received punishment, he also needs the same way to further punish it in front of the node. Move forward one step on the way to spread until the punishment until the input nodes.

Focus on the training must be repeated for each record this step, using the former to the spread of the output value, if an error occurs, then use the feedback method to learn. When the training set a record for each run over again, people said the completion of a training cycle. To complete the training of neural networks may take many months training period, often several hundred. After training by the neural network training set is found by the model, describes the training focused on response variables affected by the changes of predictor variables.

As the hidden layer neural networks are too many variable parameters, if the training time long enough, then neural network training set is possible to all the details of information "in mind" down, Er not overdo the detail to establish a regular model only has We call this situation as overtraining. Obviously this "model" for training has a high accuracy of assembly, and once left the training set applied to other data, it is possible accuracy declined sharply. To prevent this training over the situation, one must know when to stop training. In some software implementations of the same training courses will use a test set to computational neuroscience network in this test set the correct rate, once the correct rate Buzaishenggao even started to decline, it feels that the neural network has achieved good The state has to stop training.

The curve in Figure 3 can be used to help people understand why the test set to prevent the emergence of excessive training. As can be seen in Figure training set and test set error rate increased with training in the beginning, continue to lower the increase of cycle, the test set error rate reached a trough Instead, it began Shang Sheng, people think it started to rise the moment Jiu Shi should stop training the moment.

Neural network training cycle increases the accuracy of the changes

Neural networks and statistical methods, in essence, there are many differences. Neural Network for a lot more than the statistical methods. Figure 4, there are 13 parameters (9 and 4 weight restrictions). Because so many parameters, through various combinations of parameters to affect the output, so difficult that a neural network model to make visual interpretation. Neural networks is in fact as a "black box" to use, not to manage "box" inside what is simply used on the line. In most cases, this restriction is acceptable. Such as banks may need a handwriting recognition software, but he did not need to know why these lines together is a person's signature, while a similar not. In many complex problems such as high degree of chemical testing, robotics, simulation of financial markets, and language image recognition, neural networks and other fields have achieved very good results.

Another advantage of neural network is easily implemented on parallel computers, can he be assigned to different CPU nodes in parallel computing.

When using the neural network points to note: first, the neural network is difficult to explain, has yet to make a clear interpretation of the neural network methodology.

Second, the neural network will learn too, to train the neural network must be appropriate to use a neural network can be critically evaluated the methods, such as the previously mentioned methods and cross-validation test set method. This is mainly due to neural network is too flexible, too many variable parameters, if given enough time, he almost can "remember" anything.

Third, unless the problem is very simple to train a neural network may need considerable time to complete. Of course, once the neural network was good, and the forecasts do with it, or will soon be running.

Fourth, the establishment of neural networks need a lot of work to do data preparation.

A very misleading myth is that no matter what data neural networks can work well and make accurate predictions. This is inaccurate, in order to obtain accurate models of the data must be careful cleaning, sorting, transformation, selection, etc., of any data mining is that the neural network with particular emphasis on this point. Such as neural networks require that all input variables must be 0-1 (or -1 - +1) between the real number, so as the "area" like the text data need to be done only after the necessary processing for neural networks input.

译文

人工神经网络

人工神经网络-基本特征

由大量处理单元互联组成的非线性、自适应信息处理系统。它是在现代神经科学研究成果的基础上提出的，试图通过模拟大脑神经网络处理、记忆信息的方式进行信息处理。人工神经网络具有四个基本特征：

（1）非线性非线性关系是自然界的普遍特性。大脑的智慧就是一种非线性现象。人工神经元处于激活或抑制二种不同的状态，这种行为在数学上表现为一种非线性关系。具有阈值的神经元构成的网络具有更好的性能，可以提高容错性和存储容量。

（2）非局限性一个神经网络通常由多个神经元广泛连接而成。一个系统的整体行为不仅取决于单个神经元的特征，而且可能主要由单元之间的相互作用、相互连接所决定。通过单元之间的大量连接模拟大脑的非局限性。联想记忆是非局限性的典型例子

（3）非常定性人工神经网络具有自适应、自组织、自学习能力。神经网络不但处理的信息可以有各种变化，而且在处理信息的同时，非线性动力系统本身也在不断变化。经常采用迭代过程描写动力系统的演化过程。

（4）非凸性一个系统的演化方向，在一定条件下将取决于某个特定的状态函数。例如能量函数，它的极值相应于系统比较稳定的状态。非凸性是指这种函数有多个极值，故系统具有多个较稳定的平衡态，这将导致系统演化的多样性。

人工神经网络中，神经元处理单元可表示不同的对象，例如特征、字母、概念，或者一些有意义的抽象模式。网络中处理单元的类型分为三类：输入单元、输出单元和隐单元。输入单元接受外部世界的信号与数据；输出单元实现系统处理结果的输出；隐单元是处在输入和输出单元之间，不能由系统外部观察的单元。神经元间的连接权值反映了单元间的连接强度，信息的表示和处理体现在网络处理单元的连接关系中。人工神经网络是一种非程序化、适应性、大脑风格的信息处理，其本质是通过网络的变换和动力学行为得到一种并行分布式的信息处理功能，并在不同程度和层次上模仿人脑神经系统的信息处理功能。它是涉及神经科学、思维科学、人工智能、计算机科学等多个领域的交叉学科。人工神经网络是并行分布式系统，采用了与传统人工智能和信息处理技术完全不同的机理，克服了传统的基于逻辑符号的人工智能在处理直觉、非结构化信息方面的缺陷，具有自适应、自组织和实时学习的特点

人工神经网络-历史沿革

1943年，心理学家W.S.McCulloch和数理逻辑学家W.Pitts建立了神经网络和数学模型，称为MP模型。他们通过MP模型提出了神经元的形式化数学描述和网络结构方法，证明了单个神经元能执行逻辑功能，从而开创了人工神经网络研究的时代。1949年，心理学家提出了突触联系强度可变的设想。60年代，人工神经网络的到了进一步发展，更完善的神经网络模型被提出，其中包括感知器和自适应线性元件等。M.Minsky等仔细分析了以感知器为

代表的神经网络系统的功能及局限后，于1969年出版了《Perceptron》一书，指出感知器不能解决高阶谓词问题。他们的论点极大地影响了神经网络的研究，加之当时串行计算机和人工智能所取得的成就，掩盖了发展新型计算机和人工智能新途径的必要性和迫切性，使人工神经网络的研究处于低潮。在此期间，一些人工神经网络的研究者仍然致力于这一研究，提出了适应谐振理论（ART网）、自组织映射、认知机网络，同时进行了神经网络数学理论的研究。以上研究为神经网络的研究和发展奠定了基础。1982年，美国加州工学院物理学家J.J.Hopfield提出了Hopfield神经网格模型，引入了“计算能量”概念，给出了网络稳定性判断。1984年，他又提出了连续时间Hopfield神经网络模型，为神经计算机的研究做了开拓性的工作，开创了神经网络用于联想记忆和优化计算的新途径，有力地推动了神经网络的研究，1985年，又有学者提出了波耳兹曼模型，在学习中采用统计热力学模拟退火技术，保证整个系统趋于全局稳定点。1986年进行认知微观结构地研究，提出了并行分布处理的理论。人工神经网络的研究受到了各个发达国家的重视，美国国会通过决议将1990年1月5日开始的十年定为“脑的十年”，国际研究组织号召它的成员国将“脑的十年”变为全球行为。在日本的“真实世界计算（RWC）”项目中，人工智能的研究成了一个重要的组成部分。

人工神经网络-基本内容

人工神经网络模型主要考虑网络连接的拓扑结构、神经元的特征、学习规则等。目前，已有近40种神经网络模型，其中有反传网络、感知器、自组织映射、Hopfield网络、波耳兹曼机、适应谐振理论等。根据连接的拓扑结构，神经网络模型可以分为：

（1）前向网络网络中各个神经元接受前一级的输入，并输出到下一级，网络中没有反馈，可以用一个有向无环路图表示。这种网络实现信号从输入空间到输出空间的变换，它的信息处理能力来自于简单非线性函数的多次复合。网络结构简单，易于实现。反传网络是一种典型的前向网络。

（2）反馈网络网络内神经元间有反馈，可以用一个无向的完备图表示。这种神经网络的信息处理是状态的变换，可以用动力学系统理论处理。系统的稳定性与联想记忆功能有密切关系。Hopfield网络、波耳兹曼机均属于这种类型。

学习是神经网络研究的一个重要内容，它的适应性是通过学习实现的。根据环境的变化，对权值进行调整，改善系统的行为。由Hebb提出的Hebb学习规则为神经网络的学习算法奠定了基础。Hebb规则认为学习过程最终发生在神经元之间的突触部位，突触的联系强度随着突触前后神经元的活动而变化。在此基础上，人们提出了各种学习规则和算法，以适应不同网络模型的需要。有效的学习算法，使得神经网络能够通过连接权值的调整，构造客观世界的内在表示，形成具有特色的信息处理方法，信息存储和处理体现在网络的连接中。

根据学习环境不同，神经网络的学习方式可分为监督学习和非监督学习。在监督学习中，将训练样本的数据加到网络输入端，同时将相应的期望输出与网络输出相比较，得到误差信号，以此控制权值连接强度的调整，经多次训练后收敛到一个确定的权值。当样本情况发生变化时，经学习可以修改权值以适应新的环境。使用监督学习的神经网络模型有反传网络、感知器等。非监督学习时，事先不给定标准样本，直接将网络置于环境之中，学习阶段与工作阶段成为一体。此时，学习规律的变化服从连接权值的演变方程。非监督学习最简单的例子是

Hebb学习规则。竞争学习规则是一个更复杂的非监督学习的例子，它是根据已建立的聚类进行权值调整。自组织映射、适应谐振理论网络等都是与竞争学习有关的典型模型。

研究神经网络的非线性动力学性质，主要采用动力学系统理论、非线性规划理论和统计理论，来分析神经网络的演化过程和吸引子的性质，探索神经网络的协同行为和集体计算功能，了解神经信息处理机制。为了探讨神经网络在整体性和模糊性方面处理信息的可能，混沌理论的概念和方法将会发挥作用。混沌是一个相当难以精确定义的数学概念。一般而言，“混沌”是指由确定性方程描述的动力学系统中表现出的非确定性行为，或称之为确定的随机性。“确定性”是因为它由内在的原因而不是外来的噪声或干扰所产生，而“随机性”是指其不规则的、不能预测的行为，只可能用统计的方法描述。混沌动力学系统的主要特征是其状态对初始条件的灵敏依赖性，混沌反映其内在的随机性。混沌理论是指描述具有混沌行为的非线性动力学系统的基本理论、概念、方法，它把动力学系统的复杂行为理解为其自身与其在同外界进行物质、能量和信息交换过程中内在的有结构的行为，而不是外来的和偶然的行为，混沌状态是一种定态。混沌动力学系统的定态包括：静止、平稳量、周期性、准同期性和混沌解。混沌轨线是整体上稳定与局部不稳定相结合的结果，称之为奇异吸引子。一个奇异吸引子有如下一些特征：（1）奇异吸引子是一个吸引子，但它既不是不动点，也不是周期解；（2）奇异吸引子是不可分割的，即不能分为两个以及两个以上的吸引子；（3）它对初始值十分敏感，不同的初始值会导致极不相同的行为。

人工神经网络-优越性

人工神经网络的特点和优越性，主要表现在三个方面：

第一，具有自学习功能。例如实现图像识别时，只在先把许多不同的图像样板和对应的应识别的结果输入人工神经网络，网络就会通过自学习功能，慢慢学会识别类似的图像。自学习功能对于预测有特别重要的意义。预期未来的人工神经网络计算机将为人类提供经济预测、市场预测、效益预测，其应用前途是很远大的。

第二，具有联想存储功能。用人工神经网络的反馈网络就可以实现这种联想。

第三，具有高速寻找优化解的能力。寻找一个复杂问题的优化解，往往需要很大的计算量，利用一个针对某问题而设计的反馈型人工神经网络，发挥计算机的高速运算能力，可能很快找到优化解。

人工神经网络-研究方向

神经网络的研究可以分为理论研究和应用研究两大方面。

理论研究可分为以下两类：

1、利用神经生理与认知科学研究人类思维以及智能机理。

2、利用神经基础理论的研究成果，用数理方法探索功能更加完善、性能更加优越的神经网络模型，深入研究网络算法和性能，如：稳定性、收敛性、容错性、鲁棒性等；开发新的网

络数理理论，如：神经网络动力学、非线性神经场等。

应用研究可分为以下两类：

1、神经网络的软件模拟和硬件实现的研究。

2、神经网络在各个领域中应用的研究。这些领域主要包括：

模式识别、信号处理、知识工程、专家系统、优化组合、机器人控制等。随着神经网络理论本身以及相关理论、相关技术的不断发展，神经网络的应用定将更加深入。

人工神经网络-应用分析

神经网络近来越来越受到人们的关注，因为它为解决大复杂度问题提供了一种相对来说比较有效的简单方法。神经网络可以很容易的解决具有上百个参数的问题（当然实际生物体中存在的神经网络要比我们这里所说的程序模拟的神经网络要复杂的多）。神经网络常用于两类问题：分类和回归。在结构上，可以把一个神经网络划分为输入层、输出层和隐含层（见图1）。输入层的每个节点对应一个个的预测变量。输出层的节点对应目标变量，可有多个。在输入层和输出层之间是隐含层（对神经网络使用者来说不可见），隐含层的层数和每层节点的个数决定了神经网络的复杂度。

图1一个神经元网络

除了输入层的节点，神经网络的每个节点都与很多它前面的节点（称为此节点的输入节点）连接在一起，每个连接对应一个权重Wxy，此节点的值就是通过它所有输入节点的值与对应连接权重乘积的和作为一个函数的输入而得到，我们把这个函数称为活动函数或挤压函数。如图2中节点4输出到节点6的值可通过如下计算得到：

W14*节点1的值+W24*节点2的值

神经网络的每个节点都可表示成预测变量（节点1，2）的值或值的组合（节点3-6）。注意节点6的值已经不再是节点1、2的线性组合，因为数据在隐含层中传递时使用了活动函数。实际上如果没有活动函数的话，神经元网络就等价于一个线性回归函数，如果此活动函数是某种特定的非线性函数，那神经网络又等价于逻辑回归。

调整节点间连接的权重就是在建立（也称训练）神经网络时要做的工作。最早的也是最基本的权重调整方法是错误回馈法，现在较新的有变化坡度法、类牛顿法、Levenberg-Marquardt 法、和遗传算法等。无论采用那种训练方法，都需要有一些参数来控制训练的过程，如防止训练过度和控制训练的速度。

图2带权重Wxy的神经元网络

决定神经网络拓扑结构（或体系结构）的是隐含层及其所含节点的个数，以及节点之间的连

接方式。要从头开始设计一个神经网络，必须要决定隐含层和节点的数目，活动函数的形式，以及对权重做那些限制等，当然如果采用成熟软件工具的话，他会帮你决定这些事情。在诸多类型的神经网络中，最常用的是前向传播式神经网络，也就是我们前面图示中所描绘的那种。人们下面详细讨论一下，为讨论方便假定只含有一层隐含节点。可以认为错误回馈式训练法是变化坡度法的简化，其过程如下：前向传播：数据从输入到输出的过程是一个从前向后的传播过程，后一节点的值通过它前面相连的节点传过来，然后把值按照各个连接权重的大小加权输入活动函数再得到新的值，进一步传播到下一个节点。

回馈：当节点的输出值与我们预期的值不同，也就是发生错误时，神经网络就要“学习”（从错误中学习）。我们可以把节点间连接的权重看成后一节点对前一节点的“信任”程度（他自己向下一节点的输出更容易受他前面哪个节点输入的影响）。学习的方法是采用惩罚的方法，过程如下：如果一节点输出发生错误，那么他看他的错误是受哪个（些）输入节点的影响而造成的，是不是他最信任的节点（权重最高的节点）陷害了他（使他出错），如果是则要降低对他的信任值（降低权重），惩罚他们，同时升高那些做出正确建议节点的信任值。对那些收到惩罚的节点来说，他也需要用同样的方法来进一步惩罚它前面的节点。就这样把惩罚一步步向前传播直到输入节点为止。

对训练集中的每一条记录都要重复这个步骤，用前向传播得到输出值，如果发生错误，则用回馈法进行学习。当把训练集中的每一条记录都运行过一遍之后，人们称完成一个训练周期。要完成神经网络的训练可能需要很多个训练周期，经常是几百个。训练完成之后得到的神经网络就是在通过训练集发现的模型，描述了训练集中响应变量受预测变量影响的变化规律。

由于神经网络隐含层中的可变参数太多，如果训练时间足够长的话，神经网络很可能把训练集的所有细节信息都“记”下来，而不是建立一个忽略细节只具有规律性的模型，我们称这种情况为训练过度。显然这种“模型”对训练集会有很高的准确率，而一旦离开训练集应用到其他数据，很可能准确度急剧下降。为了防止这种训练过度的情况，人们必须知道在什么时候要停止训练。在有些软件实现中会在训练的同时用一个测试集来计算神经网络在此测试集上的正确率，一旦这个正确率不再升高甚至开始下降时，那么就认为现在神经网络已经达到做好的状态了可以停止训练。

图3中的曲线可以帮人们理解为什么利用测试集能防止训练过度的出现。在图中可以看到训练集和测试集的错误率在一开始都随着训练周期的增加不断降低，而测试集的错误率在达到一个谷底后反而开始上升，人们认为这个开始上升的时刻就是应该停止训练的时刻。

神经网络在训练周期增加时准确度的变化情况

神经元网络和统计方法在本质上有很多差别。神经网络的参数可以比统计方法多很多。如图4中就有13个参数（9个权重和4个限制条件）。由于参数如此之多，参数通过各种各样的组合方式来影响输出结果，以至于很难对一个神经网络表示的模型做出直观的解释。实际上神经网络也正是当作“黑盒”来用的，不用去管“盒子”里面是什么，只管用就行了。在大部分情况下，这种限制条件是可以接受的。比如银行可能需要一个笔迹识别软件，但他没必要知道为什么这些线条组合在一起就是一个人的签名，而另外一个相似的则不是。在很多复杂度很高的问题如化学试验、机器人、金融市场的模拟、和语言图像的识别，等领域神经网络都取得了很好的效果。

神经网络的另一个优点是很容易在并行计算机上实现，可以把他的节点分配到不同的CPU 上并行计算。

在使用神经网络时有几点需要注意：第一，神经网络很难解释，目前还没有能对神经网络做出显而易见解释的方法学。

第二，神经网络会学习过度，在训练神经网络时一定要恰当的使用一些能严格衡量神经网络的方法，如前面提到的测试集方法和交叉验证法等。这主要是由于神经网络太灵活、可变参数太多，如果给足够的时间，他几乎可以“记住”任何事情。

第三，除非问题非常简单，训练一个神经网络可能需要相当可观的时间才能完成。当然，一旦神经网络建立好了，在用它做预测时运行时还是很快得。

第四，建立神经网络需要做的数据准备工作量很大。一个很有误导性的神话就是不管用什么数据神经网络都能很好的工作并做出准确的预测。这是不确切的，要想得到准确度高的模型必须认真的进行数据清洗、整理、转换、选择等工作，对任何数据挖掘技术都是这样，神经网络尤其注重这一点。比如神经网络要求所有的输入变量都必须是0-1（或-1--+1）之间的实数，因此像“地区”之类文本数据必须先做必要的处理之后才能用作神经网络的输入。