Session 3-permutaiton combination probability 排列组合和概率

1. PERMUTATIONSuppose n objects are to be ordered from 1st to nth, each order is called a permutation. Apply the multiplication principle to count the number of permutations of n objects, i.e. n(n-1)(n-2)(n-3)....(3)(2)(1), or n!, called n factorial.e.g. Suppose that 10 students are going on a bus trip, and each of the students will be assigned to one of the 10 available seats. What is the number of possible different seating arrangements of the students on the bus?Notice: n objects should be distinguishable and they are always ordered in a line. If the objects are not ordered in a line but in other shapes, such as a circle or a square, how to calculate the number of permutations?e.g. Five students are going to sit around a table, how many arrangement can there be? (If the relative position of two students is the same, then we view it as one arrangement.)formula: (n-1)!If there are some objects are exactly the same, the number of permutations should be calculated in another way.e.g. How many different five-letter words can be formed when all letters in the word ENTER are used each time.formula: n!/(number of repeated objects)!Suppose that k objects will be selected from a set of n objects, where k<=n, and the k objects will be placed in order from 1st to kth. The number of permutation is n(n-1)(n-2)....(n-k+1).e.g. How many different five-digit positive integers can be formed using the digits 1, 2, 3, 4, 5, 6, and 7 if none of the digits can occur more than once in the integer?2. CombinationGiven the five letters A, B, C, D, and E, determine the number of ways of selecting 3 of the 5 letters, but unlike before, you do not want to count different orders for the 3 letters. i.e.Note that n choose k is always equal to n choose n-ke.g. You should choose 3-person committee from a group of 9 students. How many ways are there to do this?The difference between permutation and combination is whether order is considered. If different order makes different arrangement, it is permutation, and if different order makes no difference, it is combination. Most problems require us to combine these two.e.g. Suppose you want to choose 5 members from a group of 8 to watering 5 different gardens. How many ways to do this?3. Permutation& combinationMultiplication Principle:If we need N steps to do a job and in the first step we have M1 different ways, second M2, third M3...., then we apply multiplication principle to count the total ways of doing this job, i.e. N=M1*M2*M3....*MnAddition Principle:If we have N different ways to do a job and in the first way we have M1 different ways, second M2, third M3..., then we apply addition principle to count the total ways of doing this job, i.e. N=M1+M2+M3...+MnNotice: In multiplication principle steps are interdependent, while in addition principle each way can independently make the job done.General method to solve permutation & combination problems:1. Understand what needs to be done2. Decide whether to take steps or divide into different ways, or both. When taking steps, how many steps are there; when dividing into different ways, how many ways in total.4. Make sure of the requirements in each step or way, permutation or combination, and calculate the number of arrangements in each step or each way.5. Count the total ways using multiplication principle or addition principle.3.1 Give priorities to special elements and placese.g. How many different 5-digit odd numbers can be formed when choosing from 0, 1, 2, 3, 4, 5 and each number can be used only once? (key: 288)3.2 Bundling Strategye.g. There are 7 students to stand in a straight line. If student A and B should always stand side by side, so should student D and G, how many ways to line those 7 students? (key: 480)Remember to make permutation between bundling elements3.3 Slot strategye.g. There is going to be 2 dances, 2 cross talks, and 1 solo in a party. If the 2 dances cannot be performed in a row, how many different program lists can we make for this party? (key: 72)Permutation first and then plug in the special elements3.4 More than one line problemse.g. Suppose you are going to arrange 8 students into two rows with four students in each row. Student A and B must be in the first row and Student C must in the second row. How many ways to arrange? (key:5760)Turn it into one straight line problem3.5 Mixture problemse.g. Suppose we need to put 5 different balls into 4 different boxes and each box should contain at least one ball. How many ways to make this done? (key: 240) First choose and then order3.6 Plank strategye.g. An elementary school has 10 athlete quota to be assigned to 7 classes. If every class should have at least one quota, how many ways are there to assign? (key: 84)Pay attention to the differences between 3.5 and 3.64. ProbabilityProbability is a way of describing uncertainty in numerical terms.Probability experiment (random experiment)The set of all possible outcomes of a random experiment is called the sample space, and any particular set of outcomes is called an event.P(E)=the number of outcomes in the event/the number of possible outcomes in the experimentGeneral facts about probability:●If an event E is certain to occur, then P(E)=1●If an even E is certain not to occur, then P(E)=0●If an even E is possible but not certain to occur, then 0<P(E)<1●The probability that an event E will not occur is equal to 1-P(E)●If E is an event, then the probability of E is the sum of the probabilities ofoutcomes in E.●The sum of the probabilities of all possible outcomes of an experiment is 1. If E and F are two events of an experiment, we consider two other events relatedto E and F.●The event that both E and F occur, that is, all outcomes in the set●The event that E or F, or both, occur, that is , all outcomes in the set●P(E or F)=P(E)+P(F)-P(both E and F)●P(E or F)=P(E)+p(F) if E and F are mutually exclusive●P(E and F)=P(E)P(F) if E and F are independent●the probability of one event occurring k times during n independentexperiments P(E)=e.g. A fair 6-sided die is rolled nine times and what's the probability that two of the nine are odd number?If a fair 6-sided die is rolled once, let E be the event of rolling a 3 and let F be the event of rolling an odd number. What is the probability of both E and F occurring?A 12-sided die, with faces numbered 1 to 12, is to be rolled once, and each of the12 possible outcomes is equally likely to occur. What is the probability of rollinga number that is either a multiple of 5 or an odd number?Consider an experiment with events A, B, and C for which P(A)=0.23, P(B)=0.40, and P(C)=0.85. Suppose that events A and B are mutually exclusive and events B and C are independent. What are the probabilities P(A or B) and P(B or C)?Suppose that there is a 6-sided die that is weighted in such a way that each time the die is rolled, the probabilities of rolling any of the numbers from 1 to 5 are all equal, but the probability of rolling a 6 is twice the probability of rolling a 1. When you roll the die once, the 6 outcomes are not equally likely. What are the probabilities of the 6 outcomes?Suppose that you roll the weighted 6-sided die from above example twice. What is the probability that the first roll will be an odd number and the second roll will be an even number?A box contains 5 orange disks, 4 red disks, and 1 blue disk. You are to select two disks at random and without replacement from the box. What is the probability that the first disk you select will be red and the second disk you select will be orange?There are going to be 5 different songs and 3 different dances in a particular party. How many ways to arrange the programs as follows?(1) three dances should be shown in a row(2) three dances should be separatedHow many ways to divide 6 different books into three piles?How many ways to assign 6 different books to 3 persons and each has exactly two books?How many different 5-digit even numbers can be formed from 0, 2, 3, 6, and 9? (Each number should be used only once.)A. 120B. 72C. 60D, 48E. 106Xavier, Yvonne, and Zelda each try independently to solve a problem. If their individual probabilities for success are 1/4, 1/2 and 5/8, respectively, what is the probability that Xavier and Yvonne, but not Zelda, will solve the problem?(A)11/8(B)7/8(C)9/64(D)5/64(E)3/64Among a group of 2,500 people, 35 percent invest inmunicipal bonds, 18 percent invest in oil stocks, and7 percent invest in both municipal bonds and oilstocks. If 1 person is to be randomly selected fromthe 2,500 people, what is the probabilitythat theperson selected will be one who invests in municipalbonds but NOT in oil stocks?A certain club has 10 members, including Harry. Oneof the 10 members is to be chosen at random to bethe president, one of the remaining 9 members is tobe chosen at random to be the secretary, and one ofthe remaining 8 members is to be chosen at randomto be the treasurer. What is the probability that Harrywill be either the memberchosento be the secretaryor the member chosen to be the treasurer?A={2, 3, 4, 5}, B={4, 5, 6, 7, 8}Two integers will be randomly selected from the setsabove, one integer from set Aand one integer fromset B. What is the probability that the sum of the twointegers will equal 9 ?。

合集下载

泊松融合原理和python代码

泊松融合原理和python代码【原创版】目录1.泊松融合原理概述2.Python 代码实现泊松融合原理3.泊松融合原理的应用正文1.泊松融合原理概述泊松融合原理是一种概率论中的经典理论，由法国数学家泊松提出。

泊松融合原理描述了一个事件在特定时间间隔内发生的概率与另一个事件在相同时间间隔内发生的次数之间的关系。

具体而言，泊松融合原理表明，一个事件在时间间隔Δt 内发生的次数服从泊松分布，即P(X=k)=e^(-λΔt) * (λΔt)^k / k!，其中λ是事件的平均发生率，X 是事件在时间间隔Δt 内发生的次数，k 是事件发生的次数。

2.Python 代码实现泊松融合原理为了验证泊松融合原理，我们可以使用 Python 编写代码模拟事件的发生过程。

以下是一个简单的 Python 代码示例：```pythonimport randomimport mathdef poisson_fusion(lambda_value, dt, num_trials):"""泊松融合原理模拟:param lambda_value: 事件的平均发生率:param dt: 时间间隔:param num_trials: 模拟次数:return: 事件在时间间隔内发生的次数"""count = 0for _ in range(num_trials):# 随机生成一个 0 到 dt 之间的时间间隔t = random.uniform(0, dt)# 计算在时间间隔内事件发生的次数k = int(lambda_value * t)# 计算泊松分布的概率poisson_prob = math.exp(-lambda_value * t) * (lambda_value * t) ** k / k# 根据泊松分布概率随机生成事件发生的次数count += random.choice([0, 1, 2, 3, 4],p=poisson_prob)return count# 示例lambda_value = 1 # 事件的平均发生率为 1dt = 1 # 时间间隔为 1um_trials = 1000 # 模拟次数为 1000# 模拟事件在时间间隔内发生的次数counts = [poisson_fusion(lambda_value, dt, num_trials) for _in range(num_trials)]# 计算平均值和方差mean = sum(counts) / num_trialsvariance = sum((count - mean) ** 2 for count in counts) / num_trialsprint("平均值:", mean)print("方差:", variance)```3.泊松融合原理的应用泊松融合原理在实际应用中有很多场景，例如统计学、保险、生物学等领域。

孟德尔随机化extract clump access代码 -回复

孟德尔随机化extract clump access代码-回复【孟德尔随机化extract clump access代码】是指利用孟德尔随机化方法来提取并分析数据集中的束状访问（clump access）模式的一种编程代码。

在这篇文章中，我将一步一步回答有关这个主题的问题，包括介绍孟德尔随机化方法、解释什么是束状访问模式，以及如何使用代码来提取和分析这种模式。

第一部分：孟德尔随机化方法孟德尔随机化方法是一种在实验设计中用来产生随机对照组的统计学方法。

它的主要原理是通过将研究对象随机分为实验组和对照组，来控制潜在的干扰因素，进而评估因果关系。

在这种方法中，随机分组是非常关键的，因为它可以确保实验组和对照组在其他特征上的相似性，从而降低实验结果的偏见。

第二部分：什么是束状访问模式假设我们有一个数据集，其中包含多个变量。

束状访问模式是指在数据集中出现的一种特殊模式，即某些变量在数据中具有聚集的趋势。

这种模式的存在可能有助于我们理解数据中的内在结构和关系。

第三部分：提取束状访问模式的代码下面是一个用于提取和分析束状访问模式的代码示例：import numpy as npimport pandas as pddef extract_clump_access(data):# 计算每个变量的均值means = data.mean(axis=0)# 根据均值将数据分为两组data_group1 = data[data >= means]data_group2 = data[data < means]# 分别计算每组中变量的方差variances_group1 = data_group1.var(axis=0)variances_group2 = data_group2.var(axis=0)# 根据方差的大小判断是否存在束状访问模式clump_access = []for i in range(len(data.columns)):if variances_group1[i] > variances_group2[i]:clump_access.append(data.columns[i])return clump_access在这段代码中，我们首先导入了numpy和pandas库，以便进行数组和数据框的操作。

残差半循环神经网络[发明专利]

专利名称：残差半循环神经网络专利类型：发明专利
发明人：汤琦,祁褎然
申请号：CN202080036830.3申请日：20200323
公开号：CN114175052A
公开日：
20220311
专利内容由知识产权出版社提供
摘要：残差半循环神经网络(RSNN)可以被配置成接收时不变输入和时变输入数据以生成一个或多个时间序列预测。

所述时不变输入可以由所述RSNN的多层感知器处理。

所述多层感知器的输出可以用作所述RSNN的循环神经网络单元的初始状态。

循环神经网络单元还可以接收时不变输入，并且利用所述时不变输入处理所述时不变输入以生成输出。

所述多层感知器和所述循环神经网络单元的输出可以被组合以生成所述一个或多个时间序列预测。

申请人：赛诺菲
地址：法国巴黎
国籍：FR
代理机构：北京坤瑞律师事务所
代理人：封新琴
更多信息请下载全文后查看。

ABSTRACT Progressive Simplicial Complexes

Progressive Simplicial Complexes Jovan Popovi´c Hugues HoppeCarnegie Mellon University Microsoft ResearchABSTRACTIn this paper,we introduce the progressive simplicial complex(PSC) representation,a new format for storing and transmitting triangu-lated geometric models.Like the earlier progressive mesh(PM) representation,it captures a given model as a coarse base model together with a sequence of reﬁnement transformations that pro-gressively recover detail.The PSC representation makes use of a more general reﬁnement transformation,allowing the given model to be an arbitrary triangulation(e.g.any dimension,non-orientable, non-manifold,non-regular),and the base model to always consist of a single vertex.Indeed,the sequence of reﬁnement transforma-tions encodes both the geometry and the topology of the model in a uniﬁed multiresolution framework.The PSC representation retains the advantages of PM’s.It deﬁnes a continuous sequence of approx-imating models for runtime level-of-detail control,allows smooth transitions between any pair of models in the sequence,supports progressive transmission,and offers a space-efﬁcient representa-tion.Moreover,by allowing changes to topology,the PSC sequence of approximations achieves betterﬁdelity than the corresponding PM sequence.We develop an optimization algorithm for constructing PSC representations for graphics surface models,and demonstrate the framework on models that are both geometrically and topologically complex.CR Categories:I.3.5[Computer Graphics]:Computational Geometry and Object Modeling-surfaces and object representations.Additional Keywords:model simpliﬁcation,level-of-detail representa-tions,multiresolution,progressive transmission,geometry compression.1INTRODUCTIONModeling and3D scanning systems commonly give rise to triangle meshes of high complexity.Such meshes are notoriously difﬁcult to render,store,and transmit.One approach to speed up rendering is to replace a complex mesh by a set of level-of-detail(LOD) approximations;a detailed mesh is used when the object is close to the viewer,and coarser approximations are substituted as the object recedes[6,8].These LOD approximations can be precomputed Work performed while at Microsoft Research.Email:jovan@,hhoppe@Web:/jovan/Web:/hoppe/automatically using mesh simpliﬁcation methods(e.g.[2,10,14,20,21,22,24,27]).For efﬁcient storage and transmission,meshcompression schemes[7,26]have also been developed.The recently introduced progressive mesh(PM)representa-tion[13]provides a uniﬁed solution to these problems.In PM form,an arbitrary mesh M is stored as a coarse base mesh M0together witha sequence of n detail records that indicate how to incrementally re-ﬁne M0into M n=M(see Figure7).Each detail record encodes theinformation associated with a vertex split,an elementary transfor-mation that adds one vertex to the mesh.In addition to deﬁninga continuous sequence of approximations M0M n,the PM rep-resentation supports smooth visual transitions(geomorphs),allowsprogressive transmission,and makes an effective mesh compressionscheme.The PM representation has two restrictions,however.First,it canonly represent meshes:triangulations that correspond to orientable12-dimensional manifolds.Triangulated2models that cannot be rep-resented include1-d manifolds(open and closed curves),higherdimensional polyhedra(e.g.triangulated volumes),non-orientablesurfaces(e.g.M¨o bius strips),non-manifolds(e.g.two cubes joinedalong an edge),and non-regular models(i.e.models of mixed di-mensionality).Second,the expressiveness of the PM vertex splittransformations constrains all meshes M0M n to have the same topological type.Therefore,when M is topologically complex,the simpliﬁed base mesh M0may still have numerous triangles(Fig-ure7).In contrast,a number of existing simpliﬁcation methods allowtopological changes as the model is simpliﬁed(Section6).Ourwork is inspired by vertex uniﬁcation schemes[21,22],whichmerge vertices of the model based on geometric proximity,therebyallowing genus modiﬁcation and component merging.In this paper,we introduce the progressive simplicial complex(PSC)representation,a generalization of the PM representation thatpermits topological changes.The key element of our approach isthe introduction of a more general reﬁnement transformation,thegeneralized vertex split,that encodes changes to both the geometryand topology of the model.The PSC representation expresses anarbitrary triangulated model M(e.g.any dimension,non-orientable,non-manifold,non-regular)as the result of successive reﬁnementsapplied to a base model M1that always consists of a single vertex (Figure8).Thus both geometric and topological complexity are recovered progressively.Moreover,the PSC representation retains the advantages of PM’s,including continuous LOD,geomorphs, progressive transmission,and model compression.In addition,we develop an optimization algorithm for construct-ing a PSC representation from a given model,as described in Sec-tion4.1The particular parametrization of vertex splits in[13]assumes that mesh triangles are consistently oriented.2Throughout this paper,we use the words“triangulated”and“triangula-tion”in the general dimension-independent sense.Figure 1:Illustration of a simplicial complex K and some of its subsets.2BACKGROUND2.1Concepts from algebraic topologyTo precisely deﬁne both triangulated models and their PSC repre-sentations,we ﬁnd it useful to introduce some elegant abstractions from algebraic topology (e.g.[15,25]).The geometry of a triangulated model is denoted as a tuple (K V )where the abstract simplicial complex K is a combinatorial structure specifying the adjacency of vertices,edges,triangles,etc.,and V is a set of vertex positions specifying the shape of the model in 3.More precisely,an abstract simplicial complex K consists of a set of vertices 1m together with a set of non-empty subsets of the vertices,called the simplices of K ,such that any set consisting of exactly one vertex is a simplex in K ,and every non-empty subset of a simplex in K is also a simplex in K .A simplex containing exactly d +1vertices has dimension d and is called a d -simplex.As illustrated pictorially in Figure 1,the faces of a simplex s ,denoted s ,is the set of non-empty subsets of s .The star of s ,denoted star(s ),is the set of simplices of which s is a face.The children of a d -simplex s are the (d 1)-simplices of s ,and its parents are the (d +1)-simplices of star(s ).A simplex with exactly one parent is said to be a boundary simplex ,and one with no parents a principal simplex .The dimension of K is the maximum dimension of its simplices;K is said to be regular if all its principal simplices have the same dimension.To form a triangulation from K ,identify its vertices 1m with the standard basis vectors 1m ofm.For each simplex s ,let the open simplex smdenote the interior of the convex hull of its vertices:s =m:jmj =1j=1jjsThe topological realization K is deﬁned as K =K =s K s .The geometric realization of K is the image V (K )where V :m 3is the linear map that sends the j -th standard basis vector jm to j 3.Only a restricted set of vertex positions V =1m lead to an embedding of V (K )3,that is,prevent self-intersections.The geometric realization V (K )is often called a simplicial complex or polyhedron ;it is formed by an arbitrary union of points,segments,triangles,tetrahedra,etc.Note that there generally exist many triangulations (K V )for a given polyhedron.(Some of the vertices V may lie in the polyhedron’s interior.)Two sets are said to be homeomorphic (denoted =)if there ex-ists a continuous one-to-one mapping between them.Equivalently,they are said to have the same topological type .The topological realization K is a d-dimensional manifold without boundary if for each vertex j ,star(j )=d .It is a d-dimensional manifold if each star(v )is homeomorphic to either d or d +,where d +=d:10.Two simplices s 1and s 2are d-adjacent if they have a common d -dimensional face.Two d -adjacent (d +1)-simplices s 1and s 2are manifold-adjacent if star(s 1s 2)=d +1.Figure 2:Illustration of the edge collapse transformation and its inverse,the vertex split.Transitive closure of 0-adjacency partitions K into connected com-ponents .Similarly,transitive closure of manifold-adjacency parti-tions K into manifold components .2.2Review of progressive meshesIn the PM representation [13],a mesh with appearance attributes is represented as a tuple M =(K V D S ),where the abstract simpli-cial complex K is restricted to deﬁne an orientable 2-dimensional manifold,the vertex positions V =1m determine its ge-ometric realization V (K )in3,D is the set of discrete material attributes d f associated with 2-simplices f K ,and S is the set of scalar attributes s (v f )(e.g.normals,texture coordinates)associated with corners (vertex-face tuples)of K .An initial mesh M =M n is simpliﬁed into a coarser base mesh M 0by applying a sequence of n successive edge collapse transforma-tions:(M =M n )ecol n 1ecol 1M 1ecol 0M 0As shown in Figure 2,each ecol uniﬁes the two vertices of an edgea b ,thereby removing one or two triangles.The position of the resulting uniﬁed vertex can be arbitrary.Because the edge collapse transformation has an inverse,called the vertex split transformation (Figure 2),the process can be reversed,so that an arbitrary mesh M may be represented as a simple mesh M 0together with a sequence of n vsplit records:M 0vsplit 0M 1vsplit 1vsplit n 1(M n =M )The tuple (M 0vsplit 0vsplit n 1)forms a progressive mesh (PM)representation of M .The PM representation thus captures a continuous sequence of approximations M 0M n that can be quickly traversed for interac-tive level-of-detail control.Moreover,there exists a correspondence between the vertices of any two meshes M c and M f (0c f n )within this sequence,allowing for the construction of smooth vi-sual transitions (geomorphs)between them.A sequence of such geomorphs can be precomputed for smooth runtime LOD.In addi-tion,PM’s support progressive transmission,since the base mesh M 0can be quickly transmitted ﬁrst,followed the vsplit sequence.Finally,the vsplit records can be encoded concisely,making the PM representation an effective scheme for mesh compression.Topological constraints Because the deﬁnitions of ecol and vsplit are such that they preserve the topological type of the mesh (i.e.all K i are homeomorphic),there is a constraint on the min-imum complexity that K 0may achieve.For instance,it is known that the minimal number of vertices for a closed genus g mesh (ori-entable 2-manifold)is (7+(48g +1)12)2if g =2(10if g =2)[16].Also,the presence of boundary components may further constrain the complexity of K 0.Most importantly,K may consist of a number of components,and each is required to appear in the base mesh.For example,the meshes in Figure 7each have 117components.As evident from the ﬁgure,the geometry of PM meshes may deteriorate severely as they approach topological lower bound.M 1;100;(1)M 10;511;(7)M 50;4656;(12)M 200;1552277;(28)M 500;3968690;(58)M 2000;14253219;(108)M 5000;029010;(176)M n =34794;0068776;(207)Figure 3:Example of a PSC representation.The image captions indicate the number of principal 012-simplices respectively and the number of connected components (in parenthesis).3PSC REPRESENTATION 3.1Triangulated modelsThe ﬁrst step towards generalizing PM’s is to let the PSC repre-sentation encode more general triangulated models,instead of just meshes.We denote a triangulated model as a tuple M =(K V D A ).The abstract simplicial complex K is not restricted to 2-manifolds,but may in fact be arbitrary.To represent K in memory,we encode the incidence graph of the simplices using the following linked structures (in C++notation):struct Simplex int dim;//0=vertex,1=edge,2=triangle,...int id;Simplex*children[MAXDIM+1];//[0..dim]List<Simplex*>parents;;To render the model,we draw only the principal simplices ofK ,denoted (K )(i.e.vertices not adjacent to edges,edges not adjacent to triangles,etc.).The discrete attributes D associate amaterial identiﬁer d s with each simplex s(K ).For the sake of simplicity,we avoid explicitly storing surface normals at “corners”(using a set S )as done in [13].Instead we let the material identiﬁer d s contain a smoothing group ﬁeld [28],and let a normal discontinuity (crease )form between any pair of adjacent triangles with different smoothing groups.Previous vertex uniﬁcation schemes [21,22]render principal simplices of dimension 0and 1(denoted 01(K ))as points and lines respectively with ﬁxed,device-dependent screen widths.To better approximate the model,we instead deﬁne a set A that associates an area a s A with each simplex s 01(K ).We think of a 0-simplex s 00(K )as approximating a sphere with area a s 0,and a 1-simplex s 1=j k 1(K )as approximating a cylinder (with axis (j k ))of area a s 1.To render a simplex s 01(K ),we determine the radius r model of the corresponding sphere or cylinder in modeling space,and project the length r model to obtain the radius r screen in screen pixels.Depending on r screen ,we render the simplex as a polygonal sphere or cylinder with radius r model ,a 2D point or line with thickness 2r screen ,or do not render it at all.This choice based on r screen can be adjusted to mitigate the overhead of introducing polygonal representations of spheres and cylinders.As an example,Figure 3shows an initial model M of 68,776triangles.One of its approximations M 500is a triangulated model with 3968690principal 012-simplices respectively.3.2Level-of-detail sequenceAs in progressive meshes,from a given triangulated model M =M n ,we deﬁne a sequence of approximations M i :M 1op 1M 2op 2M n1op n 1M nHere each model M i has exactly i vertices.The simpliﬁcation op-erator M ivunify iM i +1is the vertex uniﬁcation transformation,whichmerges two vertices (Section 3.3),and its inverse M igvspl iM i +1is the generalized vertex split transformation (Section 3.4).Thetuple (M 1gvspl 1gvspl n 1)forms a progressive simplicial complex (PSC)representation of M .To construct a PSC representation,we ﬁrst determine a sequence of vunify transformations simplifying M down to a single vertex,as described in Section 4.After reversing these transformations,we renumber the simplices in the order that they are created,so thateach gvspl i (a i)splits the vertex a i K i into two vertices a i i +1K i +1.As vertices may have different positions in the different models,we denote the position of j in M i as i j .To better approximate a surface model M at lower complexity levels,we initially associate with each (principal)2-simplex s an area a s equal to its triangle area in M .Then,as the model is simpliﬁed,wekeep constant the sum of areas a s associated with principal simplices within each manifold component.When2-simplices are eventually reduced to principal1-simplices and0-simplices,their associated areas will provide good estimates of the original component areas.3.3Vertex uniﬁcation transformationThe transformation vunify(a i b i midp i):M i M i+1takes an arbitrary pair of vertices a i b i K i+1(simplex a i b i need not be present in K i+1)and merges them into a single vertex a i K i. Model M i is created from M i+1by updating each member of the tuple(K V D A)as follows:K:References to b i in all simplices of K are replaced by refer-ences to a i.More precisely,each simplex s in star(b i)K i+1is replaced by simplex(s b i)a i,which we call the ancestor simplex of s.If this ancestor simplex already exists,s is deleted.V:Vertex b is deleted.For simplicity,the position of the re-maining(uniﬁed)vertex is set to either the midpoint or is left unchanged.That is,i a=(i+1a+i+1b)2if the boolean parameter midp i is true,or i a=i+1a otherwise.D:Materials are carried through as expected.So,if after the vertex uniﬁcation an ancestor simplex(s b i)a i K i is a new principal simplex,it receives its material from s K i+1if s is a principal simplex,or else from the single parent s a i K i+1 of s.A:To maintain the initial areas of manifold components,the areasa s of deleted principal simplices are redistributed to manifold-adjacent neighbors.More concretely,the area of each princi-pal d-simplex s deleted during the K update is distributed toa manifold-adjacent d-simplex not in star(a ib i).If no suchneighbor exists and the ancestor of s is a principal simplex,the area a s is distributed to that ancestor simplex.Otherwise,the manifold component(star(a i b i))of s is being squashed be-tween two other manifold components,and a s is discarded. 3.4Generalized vertex split transformation Constructing the PSC representation involves recording the infor-mation necessary to perform the inverse of each vunify i.This inverse is the generalized vertex split gvspl i,which splits a0-simplex a i to introduce an additional0-simplex b i.(As mentioned previously, renumbering of simplices implies b i i+1,so index b i need not be stored explicitly.)Each gvspl i record has the formgvspl i(a i C K i midp i()i C D i C A i)and constructs model M i+1from M i by updating the tuple (K V D A)as follows:K:As illustrated in Figure4,any simplex adjacent to a i in K i can be the vunify result of one of four conﬁgurations in K i+1.To construct K i+1,we therefore replace each ancestor simplex s star(a i)in K i by either(1)s,(2)(s a i)i+1,(3)s and(s a i)i+1,or(4)s,(s a i)i+1and s i+1.The choice is determined by a split code associated with s.Thesesplit codes are stored as a code string C Ki ,in which the simplicesstar(a i)are sortedﬁrst in order of increasing dimension,and then in order of increasing simplex id,as shown in Figure5. V:The new vertex is assigned position i+1i+1=i ai+()i.Theother vertex is given position i+1ai =i ai()i if the boolean pa-rameter midp i is true;otherwise its position remains unchanged.D:The string C Di is used to assign materials d s for each newprincipal simplex.Simplices in C Di ,as well as in C Aibelow,are sorted by simplex dimension and simplex id as in C Ki. A:During reconstruction,we are only interested in the areas a s fors01(K).The string C Ai tracks changes in these areas.Figure4:Effects of split codes on simplices of various dimensions.code string:41422312{}Figure5:Example of split code encoding.3.5PropertiesLevels of detail A graphics application can efﬁciently transitionbetween models M1M n at runtime by performing a sequence ofvunify or gvspl transformations.Our current research prototype wasnot designed for efﬁciency;it attains simpliﬁcation rates of about6000vunify/sec and reﬁnement rates of about5000gvspl/sec.Weexpect that a careful redesign using more efﬁcient data structureswould signiﬁcantly improve these rates.Geomorphs As in the PM representation,there exists a corre-spondence between the vertices of the models M1M n.Given acoarser model M c and aﬁner model M f,1c f n,each vertexj K f corresponds to a unique ancestor vertex f c(j)K cfound by recursively traversing the ancestor simplex relations:f c(j)=j j cf c(a j1)j cThis correspondence allows the creation of a smooth visual transi-tion(geomorph)M G()such that M G(1)equals M f and M G(0)looksidentical to M c.The geomorph is deﬁned as the modelM G()=(K f V G()D f A G())in which each vertex position is interpolated between its originalposition in V f and the position of its ancestor in V c:Gj()=()fj+(1)c f c(j)However,we must account for the special rendering of principalsimplices of dimension0and1(Section3.1).For each simplexs01(K f),we interpolate its area usinga G s()=()a f s+(1)a c swhere a c s=0if s01(K c).In addition,we render each simplexs01(K c)01(K f)using area a G s()=(1)a c s.The resultinggeomorph is visually smooth even as principal simplices are intro-duced,removed,or change dimension.The accompanying video demonstrates a sequence of such geomorphs.Progressive transmission As with PM’s,the PSC representa-tion can be progressively transmitted by ﬁrst sending M 1,followed by the gvspl records.Unlike the base mesh of the PM,M 1always consists of a single vertex,and can therefore be sent in a ﬁxed-size record.The rendering of lower-dimensional simplices as spheres and cylinders helps to quickly convey the overall shape of the model in the early stages of transmission.Model compression Although PSC gvspl are more general than PM vsplit transformations,they offer a surprisingly concise representation of M .Table 1lists the average number of bits re-quired to encode each ﬁeld of the gvspl records.Using arithmetic coding [30],the vertex id ﬁeld a i requires log 2i bits,and the boolean parameter midp i requires 0.6–0.9bits for our models.The ()i delta vector is quantized to 16bitsper coordinate (48bits per),and stored as a variable-length ﬁeld [7,13],requiring about 31bits on average.At ﬁrst glance,each split code in the code string C K i seems to have 4possible outcomes (except for the split code for 0-simplex a i which has only 2possible outcomes).However,there exist constraints between these split codes.For example,in Figure 5,the code 1for 1-simplex id 1implies that 2-simplex id 1also has code 1.This in turn implies that 1-simplex id 2cannot have code 2.Similarly,code 2for 1-simplex id 3implies a code 2for 2-simplex id 2,which in turn implies that 1-simplex id 4cannot have code 1.These constraints,illustrated in the “scoreboard”of Figure 6,can be summarized using the following two rules:(1)If a simplex has split code c12,all of its parents havesplit code c .(2)If a simplex has split code 3,none of its parents have splitcode 4.As we encode split codes in C K i left to right,we apply these two rules (and their contrapositives)transitively to constrain the possible outcomes for split codes yet to be ing arithmetic coding with uniform outcome probabilities,these constraints reduce the code string length in Figure 6from 15bits to 102bits.In our models,the constraints reduce the code string from 30bits to 14bits on average.The code string is further reduced using a non-uniform probability model.We create an array T [0dim ][015]of encoding tables,indexed by simplex dimension (0..dim)and by the set of possible (constrained)split codes (a 4-bit mask).For each simplex s ,we encode its split code c using the probability distribution found in T [s dim ][s codes mask ].For 2-dimensional models,only 10of the 48tables are non-trivial,and each table contains at most 4probabilities,so the total size of the probability model is small.These encoding tables reduce the code strings to approximately 8bits as shown in Table 1.By comparison,the PM representation requires approximately 5bits for the same information,but of course it disallows topological changes.To provide more intuition for the efﬁciency of the PSC repre-sentation,we note that capturing the connectivity of an average 2-manifold simplicial complex (n vertices,3n edges,and 2n trian-gles)requires ni =1(log 2i +8)n (log 2n +7)bits with PSC encoding,versus n (12log 2n +95)bits with a traditional one-way incidence graph representation.For improved compression,it would be best to use a hybrid PM +PSC representation,in which the more concise PM vertex split encoding is used when the local neighborhood is an orientableFigure 6:Constraints on the split codes for the simplices in the example of Figure 5.Table 1:Compression results and construction times.Object#verts Space required (bits/n )Trad.Con.n K V D Arepr.time a i C K i midp i (v )i C D i C Ai bits/n hrs.drumset 34,79412.28.20.928.1 4.10.453.9146.1 4.3destroyer 83,79913.38.30.723.1 2.10.347.8154.114.1chandelier 36,62712.47.60.828.6 3.40.853.6143.6 3.6schooner 119,73413.48.60.727.2 2.5 1.353.7148.722.2sandal 4,6289.28.00.733.4 1.50.052.8123.20.4castle 15,08211.0 1.20.630.70.0-43.5-0.5cessna 6,7959.67.60.632.2 2.50.152.6132.10.5harley 28,84711.97.90.930.5 1.40.453.0135.7 3.52-dimensional manifold (this occurs on average 93%of the time in our examples).To compress C D i ,we predict the material for each new principalsimplex sstar(a i )star(b i )K i +1by constructing an ordered set D s of materials found in star(a i )K i .To improve the coding model,the ﬁrst materials in D s are those of principal simplices in star(s )K i where s is the ancestor of s ;the remainingmaterials in star(a i )K i are appended to D s .The entry in C D i associated with s is the index of its material in D s ,encoded arithmetically.If the material of s is not present in D s ,it is speciﬁed explicitly as a global index in D .We encode C A i by specifying the area a s for each new principalsimplex s 01(star(a i )star(b i ))K i +1.To account for this redistribution of area,we identify the principal simplex from which s receives its area by specifying its index in 01(star(a i ))K i .The column labeled in Table 1sums the bits of each ﬁeld of the gvspl records.Multiplying by the number n of vertices in M gives the total number of bits for the PSC representation of the model (e.g.500KB for the destroyer).By way of compari-son,the next column shows the number of bits per vertex required in a traditional “IndexedFaceSet”representation,with quantization of 16bits per coordinate and arithmetic coding of face materials (3n 16+2n 3log 2n +materials).4PSC CONSTRUCTIONIn this section,we describe a scheme for iteratively choosing pairs of vertices to unify,in order to construct a PSC representation.Our algorithm,a generalization of [13],is time-intensive,seeking high quality approximations.It should be emphasized that many quality metrics are possible.For instance,the quadric error metric recently introduced by Garland and Heckbert [9]provides a different trade-off of execution speed and visual quality.As in [13,20],we ﬁrst compute a cost E for each candidate vunify transformation,and enter the candidates into a priority queueordered by ascending cost.Then,in each iteration i =n 11,we perform the vunify at the front of the queue and update the costs of affected candidates.4.1Forming set of candidate vertex pairs In principle,we could enter all possible pairs of vertices from M into the priority queue,but this would be prohibitively expensive since simpliﬁcation would then require at least O(n2log n)time.Instead, we would like to consider only a smaller set of candidate vertex pairs.Naturally,should include the1-simplices of K.Additional pairs should also be included in to allow distinct connected com-ponents of M to merge and to facilitate topological changes.We considered several schemes for forming these additional pairs,in-cluding binning,octrees,and k-closest neighbor graphs,but opted for the Delaunay triangulation because of its adaptability on models containing components at different scales.We compute the Delaunay triangulation of the vertices of M, represented as a3-dimensional simplicial complex K DT.We deﬁne the initial set to contain both the1-simplices of K and the subset of1-simplices of K DT that connect vertices in different connected components of K.During the simpliﬁcation process,we apply each vertex uniﬁcation performed on M to as well in order to keep consistent the set of candidate pairs.For models in3,star(a i)has constant size in the average case,and the overall simpliﬁcation algorithm requires O(n log n) time.(In the worst case,it could require O(n2log n)time.)4.2Selecting vertex uniﬁcations fromFor each candidate vertex pair(a b),the associated vunify(a b):M i M i+1is assigned the costE=E dist+E disc+E area+E foldAs in[13],theﬁrst term is E dist=E dist(M i)E dist(M i+1),where E dist(M)measures the geometric accuracy of the approximate model M.Conceptually,E dist(M)approximates the continuous integralMd2(M)where d(M)is the Euclidean distance of the point to the closest point on M.We discretize this integral by deﬁning E dist(M)as the sum of squared distances to M from a dense set of points X sampled from the original model M.We sample X from the set of principal simplices in K—a strategy that generalizes to arbitrary triangulated models.In[13],E disc(M)measures the geometric accuracy of disconti-nuity curves formed by a set of sharp edges in the mesh.For the PSC representation,we generalize the concept of sharp edges to that of sharp simplices in K—a simplex is sharp either if it is a boundary simplex or if two of its parents are principal simplices with different material identiﬁers.The energy E disc is deﬁned as the sum of squared distances from a set X disc of points sampled from sharp simplices to the discontinuity components from which they were sampled.Minimization of E disc therefore preserves the geom-etry of material boundaries,normal discontinuities(creases),and triangulation boundaries(including boundary curves of a surface and endpoints of a curve).We have found it useful to introduce a term E area that penalizes surface stretching(a more sophisticated version of the regularizing E spring term of[13]).Let A i+1N be the sum of triangle areas in the neighborhood star(a i)star(b i)K i+1,and A i N the sum of triangle areas in star(a i)K i.The mean squared displacement over the neighborhood N due to the change in area can be approx-imated as disp2=12(A i+1NA iN)2.We let E area=X N disp2,where X N is the number of points X projecting in the neighborhood. To prevent model self-intersections,the last term E fold penalizes surface folding.We compute the rotation of each oriented triangle in the neighborhood due to the vertex uniﬁcation(as in[10,20]).If any rotation exceeds a threshold angle value,we set E fold to a large constant.Unlike[13],we do not optimize over the vertex position i a, but simply evaluate E for i a i+1a i+1b(i+1a+i+1b)2and choose the best one.This speeds up the optimization,improves model compression,and allows us to introduce non-quadratic energy terms like E area.5RESULTSTable1gives quantitative results for the examples in theﬁgures and in the video.Simpliﬁcation times for our prototype are measured on an SGI Indigo2Extreme(150MHz R4400).Although these times may appear prohibitive,PSC construction is an off-line task that only needs to be performed once per model.Figure9highlights some of the beneﬁts of the PSC representa-tion.The pearls in the chandelier model are initially disconnected tetrahedra;these tetrahedra merge and collapse into1-d curves in lower-complexity approximations.Similarly,the numerous polyg-onal ropes in the schooner model are simpliﬁed into curves which can be rendered as line segments.The straps of the sandal model initially have some thickness;the top and bottom sides of these straps merge in the simpliﬁcation.Also note the disappearance of the holes on the sandal straps.The castle example demonstrates that the original model need not be a mesh;here M is a1-dimensional non-manifold obtained by extracting edges from an image.6RELATED WORKThere are numerous schemes for representing and simplifying tri-angulations in computer graphics.A common special case is that of subdivided2-manifolds(meshes).Garland and Heckbert[12] provide a recent survey of mesh simpliﬁcation techniques.Several methods simplify a given model through a sequence of edge col-lapse transformations[10,13,14,20].With the exception of[20], these methods constrain edge collapses to preserve the topological type of the model(e.g.disallow the collapse of a tetrahedron into a triangle).Our work is closely related to several schemes that generalize the notion of edge collapse to that of vertex uniﬁcation,whereby separate connected components of the model are allowed to merge and triangles may be collapsed into lower dimensional simplices. Rossignac and Borrel[21]overlay a uniform cubical lattice on the object,and merge together vertices that lie in the same cubes. Schauﬂer and St¨u rzlinger[22]develop a similar scheme in which vertices are merged using a hierarchical clustering algorithm.Lue-bke[18]introduces a scheme for locally adapting the complexity of a scene at runtime using a clustering octree.In these schemes, the approximating models correspond to simplicial complexes that would result from a set of vunify transformations(Section3.3).Our approach differs in that we order the vunify in a carefully optimized sequence.More importantly,we deﬁne not only a simpliﬁcation process,but also a new representation for the model using an en-coding of gvspl=vunify1transformations.Recent,independent work by Schmalstieg and Schauﬂer[23]de-velops a similar strategy of encoding a model using a sequence of vertex split transformations.Their scheme differs in that it tracks only triangles,and therefore requires regular,2-dimensional trian-gulations.Hence,it does not allow lower-dimensional simplices in the model approximations,and does not generalize to higher dimensions.Some simpliﬁcation schemes make use of an intermediate vol-umetric representation to allow topological changes to the model. He et al.[11]convert a mesh into a binary inside/outside function discretized on a three-dimensional grid,low-passﬁlter this function,。

Efficient RANSAC for Point-Cloud Shape Detection

Volume0(1981),Number0pp.1–12Efﬁcient RANSAC for Point-Cloud Shape DetectionRuwen Schnabel Roland Wahl Reinhard Klein†Universität Bonn,Computer Graphics GroupAbstractIn this work we present an automatic algorithm to detect basic shapes in unorganized point clouds.The algorithm decomposes the point cloud into a concise,hybrid structure of inherent shapes and a set of remaining points.Each detected shape serves as a proxy for a set of corresponding points.Our method is based on random sampling and detects planes,spheres,cylinders,cones and tori.For models with surfaces composed of these basic shapes only,e.g.CAD models,we automatically obtain a representation solely consisting of shape proxies.We demonstratethat the algorithm is robust even in the presence of many outliers and a high degree of noise.The proposed method scales well with respect to the size of the input point cloud and the number and size of the shapes within the data.Even point sets with several millions of samples are robustly decomposed within less than a minute.Moreover the algorithm is conceptually simple and easy to implement.Application areas include measurement of physical parameters,scan registration,surface compression,hybrid rendering,shape classiﬁcation,meshing, simpliﬁcation,approximation and reverse engineering.Categories and Subject Descriptors(according to ACM CCS):I.4.8[Image Processing and Computer Vision]:Scene AnalysisShape;Surface Fitting;I.3.5[Computer Graphics]:Computational Geometry and Object ModelingCurve, surface,solid,and object representations1.IntroductionDue to the increasing size and complexity of geometric data sets there is an ever-growing demand for concise and mean-ingful abstractions of this data.Especially when dealing with digitized geometry,e.g.acquired with a laser scanner,no handles for modiﬁcation of the data are available to the user other than the digitized points themselves.However,in or-der to be able to make use of the data effectively,the raw digitized data has to be enriched with abstractions and pos-sibly semantic information,providing the user with higher-level interaction possibilities.Only such handles can pro-vide the interaction required for involved editing processes, such as deleting,moving or resizing certain parts and hence can make the data more readily usable for modeling pur-poses.Of course,traditional reverse engineering approaches can provide some of the abstractions that we seek,but usu-ally reverse engineering focuses onﬁnding a reconstruction of the underlying geometry and typically involves quite te-dious user interaction.This is not justiﬁed in a setting where †e-mail:{schnabel,wahl,rk}@cs.uni-bonn.de a complete and detailed reconstruction is not required at all, or shall take place only after some basic editing operations have been applied to the data.On the other hand,detecting instances of a set of primitive geometric shapes in the point sampled data is a means to quickly derive higher levels of ab-straction.For example in Fig.1patches of primitive shapes provide a coarse approximation of the geometry that could be used to compress the point-cloud very effectively. Another problem arising when dealing with digitized geom-etry is the often huge size of the datasets.Therefore the efﬁciency of algorithms inferring abstractions of the data is of utmost importance,especially in interactive settings. Thus,in this paper we focus especially onﬁnding an efﬁ-cient algorithm for point-cloud shape detection,in order to be able to deal even with large point-clouds.Our work is a high performance RANSAC[FB81]algorithm that is capa-ble to extract a variety of different types of primitive shapes, while retaining such favorable properties of the RANSAC paradigm as robustness,generality and simplicity.At the heart of our algorithm are a novel,hierarchically structured sampling strategy for candidate shape generation as well as a novel,lazy cost function evaluation scheme,which signif-c The Eurographics Association and Blackwell Publishing2007.Published by Blackwell Publishing,9600Garsington Road,Oxford OX42DQ,UK and350Main Street,Malden, MA02148,USA.(a)Original(b)ApproximationFigure1:The372detected shapes in the choir screen deﬁne a coarse approximation of the surface.icantly reduces overall computational cost.Our method de-tects planes,spheres,cylinders,cones and tori,but additional primitives are possible.The goal of our algorithm is to reli-ably extract these shapes from the data,even under adverse conditions such as heavy noise.As has been indicated above,our method is especially well suited in situations where geometric data is automatically acquired and users refrain from applying surface reconstruc-tion methods,either due to the data’s low quality or due to processing time constraints.Such constraints are typical for areas where high level model interaction is required,as is the case when measuring physical parameters or in interactive, semi-automatic segmentation and postprocessing.Further applications are,for instance,registering many scans of an object,where detecting corresponding primitive shapes in multiple scans can provide good initial matches.High compression rates for point clouds can be achieved if prim-itive shapes are used to represent a large number of points with a small set of parameters.Other areas that can beneﬁt from primitive shape information include hybrid rendering and shape classiﬁcation.Additionally,a fast shape extraction method as ours can serve as building block in applications such as meshing,simpliﬁcation,approximation and reverse engineering and bears the potential of signiﬁcant speed up.2.Previous workThe detection of primitive shapes is a common problem en-countered in many areas of geometry related computer sci-ence.Over the years a vast number of methods have been proposed which cannot all be discussed here in depth.In-stead,here we give a short overview of some of the most important algorithms developed in the differentﬁelds.We treat the previous work on RANSAC algorithms separately in section2.1as it is of special relevance to our work. Vision In computer vision,the two most widely known methodologies for shape extraction are the RANSAC paradigm[FB81]and the Hough transform[Hou62].Both have been proven to successfully detect shapes in2D as well as3D.RANSAC and the Hough transform are reliable even in the presence of a high proportion of outliers,but lack of efﬁciency or high memory consumption remains their ma-jor drawback[IK88].For both schemes,many acceleration techniques have been proposed,but no one on its own,or combinations thereof,have been shown to be able to provide an algorithm as efﬁcient as ours for the3D primitive shape extraction problem.The Hough transform maps,for a given type of parameter-ized primitive,every point in the data to a manifold in the pa-rameter space.The manifold describes all possible variants of the primitive that contain the original point,i.e.in practice each point casts votes for many cells in a discretized param-eter space.Shapes are extracted by selecting those parame-ter vectors that have received a signiﬁcant amount of votes. If the parameter space is discretized naively using a simple grid,the memory requirements quickly become prohibitive even for primitives with a moderate number of parameters, such as,for instance,cones.Although several methods have been suggested to alleviate this problem[IK87][XO93]its major application area remains the2D domain where the number of parameters typically is quite small.A notable ex-ception is[VGSR04]where the Hough transform is used to detect planes in3D datasets,as3D planes still have only a small number of parameters.They also propose a two-step procedure for the Hough based detection of cylinders that uses estimated normals in the data points.In the vision community many approaches have been pro-posed for segmentation of range images with primitive shapes.When working on range images these algorithms usually efﬁciently exploit the implicitly given connectiv-ity information of the image grid in some kind of region growing or region merging step[FEF97][GBS03].This is a fundamental difference to our case,where we are given only an unstructured cloud of points that lacks any explicit connectivity information.In[LGB95]and[LJS97]shapes are found by concurrently growing different seed primi-tives from which a suitable subset is selected according to an MDL criterion(coined the recover-and-select paradigm). [GBS03]detect shapes using a genetic algorithm to optimize a robust MSACﬁtness function(see also sec.2.1).[MLM01]c The Eurographics Association and Blackwell Publishing2007.introduce involved non-linearﬁtting functions for primitive shapes that are able to handle geometric degeneracy in the context of recover-and-select segmentation.Another robust method frequently employed in the vision community is the tensor voting framework[MLT00]which has been applied to successfully reconstruct surface geome-try from extremely cluttered scenes.While tensor voting can compete with RANSAC in terms of robustness,it is,how-ever,inherently model-free and therefore cannot be applied to the detection of predeﬁned types of primitive shapes. Reverse engineering In reverse engineering,surface re-covery techniques are usually based on either a separate seg-mentation step or on a variety of region growing algorithms [VMC97][SB95][BGV∗02].Most methods call for some kind of connectivity information and are not well equipped to deal with a large amount of outliers[VMC97].Also these approaches try toﬁnd a shape proxy for every part of the pro-cessed surface with the intent of loading the reconstructed geometry information into a CAD application.[BMV01]de-scribe a system which reconstructs a boundary representa-tion that can be imported into a CAD application from an unorganized point-cloud.However,their method is based on ﬁnding a triangulation for the point-set,whereas the method presented in this work is able to operate directly on the input points.This is advantageous as computing a suitable tessela-tion may be extremely costly and becomes very intricate or even ill-deﬁned when there is heavy noise in the data.We do not,however,intend to present a method implementing all stages of a typical reverse engineering process.Graphics In computer graphics,[CSAD04]have recently proposed a general variational framework for approximation of surfaces by planes,which was extended to a set of more elaborate shape proxies by[WK05].Their aim is not only to extract certain shapes in the data,but toﬁnd a globally optimal representation of the object by a given number of primitives.However,these methods require connectivity in-formation and are,due to their exclusive use of least squares ﬁtting,susceptible to errors induced by outliers.Also,the optimization procedure is computationally expensive,which makes the method less suitable for large data sets.The out-put of our algorithm,however,could be used to initialize the set of shape proxies used by these methods,potentially accelerating the convergence of the optimization procedure. While the Hough transform and the RANSAC paradigm have been mainly used in computer vision some applica-tions have also been proposed in the computer graphics com-munity.[DDSD03]employ the Hough transform to identify planes for billboard clouds for triangle data.They propose an extension of the standard Hough transform to include a compactness criterion,but due to the high computational de-mand of the Hough transform,the method exhibits poor run-time performance on large or complex geometry.[WGK05] proposed a RANSAC-based plane detection method for hy-brid rendering of point clouds.To facilitate an efﬁcient plane detection,planes are detected only in the cells of a hier-archical space decomposition and therefore what is essen-tially one plane on the surface is approximated by several planar patches.While this is acceptable for their hybrid ren-dering technique,our methodﬁnds maximal surface patches in order to yield a more concise representation of the ob-ject.Moreover,higher order primitives are not considered in their approach.[GG04]detect so-called slippable shapes which is a superset of the shapes recognized by our method. They use the eigenvalues of a symmetric matrix derived from the points and their normals to determine the slippability of a point-set.Their detection is a bottom-up approach that merges small initial slippable surfaces to obtain a global de-composition of the model.However the computation of the eigenvalues is costly for large models,the method is sen-sitive to noise and it is hard to determine the correct size of the initial surface patches.A related approach is taken by [HOP∗05].They also use the eigenvalues of a matrix derived from line element geometry to classify surfaces.A RANSAC based segmentation algorithm is employed to detect several shapes in a point-cloud.The method is aimed mainly at mod-els containing small numbers of points and shapes as no opti-mizations or extensions to the general RANSAC framework are adopted.2.1.RANSACThe RANSAC paradigm extracts shapes by randomly draw-ing minimal sets from the point data and constructing cor-responding shape primitives.A minimal set is the smallest number of points required to uniquely deﬁne a given type of geometric primitive.The resulting candidate shapes are tested against all points in the data to determine how many of the points are well approximated by the primitive(called the score of the shape).After a given number of trials,the shape which approximates the most points is extracted and the algorithm continues on the remaining data.RANSAC ex-hibits the following,desirable properties:•It is conceptually simple,which makes it easily extensible and straightforward to implement•It is very general,allowing its application in a wide range of settings•It can robustly deal with data containing more than50% of outliers[RL93]Its major deﬁciency is the considerable computational de-mand if no further optimizations are applied.[BF81]apply RANSAC to extract cylinders from range data,[CG01]use RANSAC and the gaussian image toﬁnd cylinders in3D point clouds.Both methods,though,do not consider a larger number of different classes of shape prim-itives.[RL93]describe an algorithm that uses RANSAC to detect a set of different types of simple shapes.However, their method was adjusted to work in the image domain orc The Eurographics Association and Blackwell Publishing2007.on range images,and they did not provide the optimization necessary for processing large unstructured3D data sets.A vast number of extensions to the general RANSAC scheme have been proposed.Among the more recent ad-vances,methods such as MLESAC[TZ00]or MSAC[TZ98] improve the robustness of RANSAC with a modiﬁed score function,but do not provide any enhancement in the perfor-mance of the algorithm,which is the main focus of our work. Nonetheless the integration of a MLESAC scoring function is among the directions of our future work.[Nis05]pro-poses an acceleration technique for the case that the num-ber of candidates isﬁxed in advance.As it is a fundamen-tal property of our setup that an unknown large number of possibly very small shapes has to be detected in huge point-clouds,the amount of necessary candidates cannot,however, be speciﬁed in advance.3.OverviewGiven a point-cloud P={p1,...,p N}with associated nor-mals{n1,...,n N}the output of our algorithm is a set of primitive shapesΨ={ψ1,...,ψn}with corresponding dis-joint sets of points Pψ1⊂P,...,Pψn⊂P and a set of re-maining points R=P\{Pψ1,...,Pψn}.Similar to[RL93]and[DDSD03],we frame the shape extraction problem as an optimization problem deﬁned by a score function.The overall structure of our method is outlined in pseudo-code in algorithm1.In each iteration of the algorithm,the prim-itive with maximal score is searched using the RANSAC paradigm.New shape candidates are generated by randomly sampling minimal subsets of P using our novel sampling strategy(see sec.4.3).Candidates of all considered shape types are generated for every minimal set and all candidates are collected in the set C.Thus no special ordering has to be imposed on the detection of different types of shapes.After new candidates have been generated the one with the high-est score m is computed employing the efﬁcient lazy score evaluation scheme presented in sec.4.5.The best candidate is only accepted if,given the size|m|(in number of points) of the candidate and the number of drawn candidates|C|, the probability P(|m|,|C|)that no better candidate was over-looked during sampling is high enough(see sec.4.2.1).We provide an analysis of our sampling strategy to derive a suit-able probability computation.If a candidate is accepted,the corresponding points P m are removed from P and the can-didates C m generated with points in P m are deleted from C. The algorithm terminates as soon as P(τ,|C|)for a user de-ﬁned minimal shape sizeτis large enough.In our implementation we use a standard score function that counts the number of compatible points for a shape candi-date[RL93][GBS03].The function has two free parame-ters:εspeciﬁes the maximum distance of a compatible point whileαrestricts the deviation of a points’normal from that of the shape.We also ensure that only points forming a con-nected component on the surface are considered(see sec.4.4).Algorithm1Extract shapes in the point cloud PΨ←/0{extracted shapes}C←/0{shape candidates}repeatC←C∪newCandidates(){see sec.4.1and4.3}m←bestCandidate(C){see sec.4.4}if P(|m|,|C|)>p t thenP←P\P m{remove points}Ψ←Ψ∪mC←C\C m{remove invalid candidates}end ifuntil P(τ,|C|)>p treturnΨ4.Our method4.1.Shape estimationAs mentioned above,the shapes we consider in this work are planes,spheres,cylinders,cones and tori which have be-tween three and seven parameters.Every3D-point p i sam-pleﬁxes only one parameter of the shape.In order to reduce the number of required points we compute an approximate surface normal n i for each point[HDD∗92],so that the ori-entation gives us two more parameters per sample.That way it is possible to estimate each of the considered basic shapes from only one or two point samples.However,always using one additional sample is advantageous,because the surplus parameters can be used to immediately verify a candidate and thus eliminate the need of evaluating many relatively low scored shapes[MC02].Plane For a plane,{p1,p2,p3}constitutes a minimal set when not taking into account the normals in the points.To conﬁrm the plausibility of the generated plane,the deviation of the plane’s normal from n1,n2,n3is determined and the candidate plane is accepted only if all deviations are less than the predeﬁned angleα.Sphere A sphere is fully deﬁned by two points with corre-sponding normal vectors.We use the midpoint of the short-est line segment between the two lines given by the points p1and p2and their normals n1and n2to deﬁne the center of the sphere c.We take r= p1−c + p2−c2as the sphere ra-dius.The sphere is accepted as a shape candidate only if all three points are within a distance ofεof the sphere and their normals do not deviate by more thanαdegrees.Cylinder To generate a cylinder from two points with nor-mals weﬁrst establish the direction of the axis with a= n1×n2.Then we project the two parametric lines p1+tn1 and p2+tn2along the axis onto the a·x=0plane and take their intersection as the center c.We set the radius to the dis-tance between c and p1in that plane.Again the cylinder isc The Eurographics Association and Blackwell Publishing2007.veriﬁed by applying the thresholds εand αto distance and normal deviation of the samples.Cone Although the cone,too,is fully deﬁned by two points with corresponding normals,for simplicity we use all three points and normals in its generation.To derive the po-sition of the apex c ,we intersect the three planes deﬁned by the point and normal pairs.Then the normal of the plane de-ﬁned by the three points {c +p 1−c p 1−c ,...,c +p 3−c p 3−c }givesthe direction of the axis a .Now the opening angle ωis givenas ω=∑i arccos ((p i −c )·a )3.Afterwards,similar to above,the cone is veriﬁed before becoming a candidate shape.Torus Just as in the case of the cone we use one more point than theoretically necessary to ease the computations required for estimation,i.e.four point and normal pairs.The rotational axis of the torus is found as one of the up to two lines intersecting the four point-normal lines p i +λn i [MLM01].To choose between the two possible axes,a full torus is estimated for both choices and the one which causes the smaller error in respect to the four points is selected.To ﬁnd the minor radius,the points are collected in a plane that is rotated around the axis.Then a circle is computed using three points in this plane.The major radius is given as the distance of the circle center to the plexityThe complexity of RANSAC is dominated by two major fac-tors:The number of minimal sets that are drawn and the cost of evaluating the score for every candidate shape.As we de-sire to extract the shape that achieves the highest possible score,the number of candidates that have to be considered is governed by the probability that the best possible shape is indeed detected,i.e.that a minimal set is drawn that deﬁnes this shape.4.2.1.ProbabilitiesConsider a point cloud P of size N and a shape ψtherein consisting of n points.Let k denote the size of a minimal set required to deﬁne a shape candidate.If we assume that any k points of the shape will lead to an appropriate candidate shape then the probability of detecting ψin a single pass is:P (n )= n k N k ≈ n N k(1)The probability of a successful detection P (n ,s )after s can-didates have been drawn equals the complementary of s con-secutive failures:P (n ,s )=1−(1−P (n ))s(2)Solving for s tells us the number of candidates T required to detect shapes of size n with a probability P (n ,T )≥p t :T ≥ln (1−p t )ln (1−P (n ))(3)Figure 2:A small cylinder that has been detected by ourmethod.The shape consists of 1066points and was detected among 341,587points.That corresponds to a relative size of 1/3000.For small P (n )the logarithm in the denominator can be approximated by its Taylor series ln (1−P (n ))=−P (n )+O (P (n )2)so that:T ≈−ln (1−p t )P (n )(4)Given the cost C of evaluating the cost function,the asymp-totic complexity of the RANSAC approach is O (TC )=O (1P (n )C ).4.3.Sampling strategyAs can be seen from the last formula,the runtime complexity is directly linked to the success rate of ﬁnding good sample sets.Therefore we will now discuss in detail how sampling is performed.4.3.1.Localized samplingSince shapes are local phenomena,the a priori probability that two points belong to the same shape is higher the smaller the distance between the points.In our sampling strategy we want to exploit this fact to increase the probability of draw-ing minimal sets that belong to the same shape.[MTN ∗02]have shown that non-uniform sampling based on locality leads to a signiﬁcantly increased probability of selecting a set of inliers.From a ball of given radius around an ini-tially unrestrainedly drawn sample the remaining samples are picked to obtain a complete minimal set.This requires to ﬁx a radius in advance,which they derive from a known (or assumed)outlier density and distribution.In our setup however,outlier density and distribution vary strongly for different models and even within in a single model,which renders a ﬁxed radius inadequate.Also,in our case,using minimal sets with small diameter introduces unnecessary stability issues in the shape estimation procedure for shapes that could have been estimated from samples spread farther apart.Therefore we propose a novel sampling strategy that is able to adapt the diameter of the minimal sets to both,outlier density and shape size.cThe Eurographics Association and Blackwell Publishing 2007.We use an octree to establish spatial proximity between sam-ples very efﬁciently.When choosing points for a new candi-date,we draw theﬁrst sample p1without restrictions among all points.Then a cell C is randomly chosen from any level of the octree such that p1is contained in C.The k−1other samples are then drawn only from within cell C.The effect of this sampling strategy can be expressed in a new probability P local(n)forﬁnding a shapeψof size n:P local(n)=P(p1∈ψ)P(p2...p k∈ψ|p2...p k∈C)(5) Theﬁrst factor evaluates to n/N.The second factor obvi-ously depends on the choice of C.C is well chosen if it con-tains mostly points belonging toψ.The existence of such a cell is backed by the observation that for most points on a shape,except on edges and corners,there exists a neighbor-hood such that all of the points therein belong to that shape. Although in general it is not guaranteed that this neighbor-hood is captured in the cells of the octree,in the case of real-life data,shapes have to be sampled with an adequate density for reliable representation and,as a consequence,for all but very few points such a neighborhood will be at least as large as the smallest cells of the octree.For the sake of analysis,we assume that there exists a C for every p i∈ψsuch thatψwill be supported by half of the points in C, which accounts for up to50%local noise and outliers.We conservatively estimate the probability ofﬁnding a good C by1d where d is the depth of the octree(in practice a path of cells starting at the highest good cell to a good leaf will be good as well).The conditional probability for p2,p3∈ψinthe case of a good cell is then described by (|C|/2k−1)(|C|k−1)≈(12)k−1.And substituting yields:P local(n)=nNd2k−1(6)As large shapes can be estimated from large cells(and with high probability this will happen),the stability of the shape estimation is not affected by the sampling strategy.The impact of this sampling strategy is best illustrated with an example.The cylinder depicted in Figure2consists of 1066points.At the time that it belongs to one of the largest shapes in the point-cloud,341,547points of the original2 million still remain.Thus,it then comprises only three thou-sandth of the point-cloud.If an ordinary uniform sampling strategy were to be applied,151,522,829candidates would have to be drawn to achieve a detection probability of99%. With our strategy only64,929candidates have to be gen-erated for the same probability.That is an improvement by three orders of magnitude,i.e.in this case that is the differ-ence between hours and seconds.4.3.1.1.Level weighting Choosing C from a proper level is an important aspect of our sampling scheme.Therefore we can further improve the sampling efﬁciency by choosing C from a level according to a non-uniform distribution that re-ﬂects the likelihood of the respective level to contain a good cell.To this end,the probability P l of choosing C from level l isﬁrst initialized with1d.Then for every level l,we keep track of the sumσl of the scores achieved by the candidates generated from a cell on level l.After a given number of candidates has been tested,a new distribution for the levels is computed.The new probabilityˆP l of the level l is given asˆPl=xσlwP l+(1−x)1d,(7)where w=∑d i=1σPi.We set x=.9to ensure that at all times at least10%of the samples are spread uniformly over the levels to be able to detect when new levels start to become of greater importance as more and more points are removed from P.4.3.2.Number of candidatesIn section4.2we gave a formula for the number of candi-dates necessary to detect a shape of size n with a given prob-ability.However,in our case,the size n of the largest shape is not known in advance.Moreover,if the largest candidate has been generated early in the process we should be able to de-tect this lucky case and extract the shape well before achiev-ing a precomputed number of candidates while on the other hand we should use additional candidates if it is still unsure that indeed the best candidate has been detected.Therefore, instead ofﬁxing the number of candidates,we repeatedly an-alyze small numbers t of additional candidates and consider the best oneψm generated so far each time.As we want to achieve a low probability that a shape is extracted which is not the real maximum,we observe the probability P(|ψm|,s) with which we would have found another shape of the same size asψm.Once this probability is higher than a threshold p t(we use99%)we conclude that there is a low chance that we have overlooked a better candidate and extractψm.The algorithm terminates as soon as P(τ,s)>p t.4.4.ScoreThe score functionσP is responsible for measuring the qual-ity of a given shape candidate.We use the following aspects in our scoring function:•To measure the support of a candidate,we use the number of points that fall within anε-band around the shape.•To ensure that the points inside the band roughly follow the curvature pattern of the given primitive,we only count those points inside the band whose normals do not deviate from the normal of the shape more than a given angleα.•Additionally we incorporate a connectivity measure: Among the points that fulﬁll the previous two conditions, only those are considered that constitute the largest con-nected component on the shape.c The Eurographics Association and Blackwell Publishing2007.。

spatio-temporall...

Spatio-Temporal LSTM with Trust Gates for3D Human Action Recognition817 respectively,and utilized a SVM classiﬁer to classify the actions.A skeleton-based dictionary learning utilizing group sparsity and geometry constraint was also proposed by[8].An angular skeletal representation over the tree-structured set of joints was introduced in[9],which calculated the similarity of these fea-tures over temporal dimension to build the global representation of the action samples and fed them to SVM forﬁnal classiﬁcation.Recurrent neural networks(RNNs)which are a variant of neural nets for handling sequential data with variable length,have been successfully applied to language modeling[10–12],image captioning[13,14],video analysis[15–24], human re-identiﬁcation[25,26],and RGB-based action recognition[27–29].They also have achieved promising performance in3D action recognition[30–32].Existing RNN-based3D action recognition methods mainly model the long-term contextual information in the temporal domain to represent motion-based dynamics.However,there is also strong dependency between joints in the spatial domain.And the spatial conﬁguration of joints in video frames can be highly discriminative for3D action recognition task.In this paper,we propose a spatio-temporal long short-term memory(ST-LSTM)network which extends the traditional LSTM-based learning to two con-current domains(temporal and spatial domains).Each joint receives contextual information from neighboring joints and also from previous frames to encode the spatio-temporal context.Human body joints are not naturally arranged in a chain,therefore feeding a simple chain of joints to a sequence learner can-not perform well.Instead,a tree-like graph can better represent the adjacency properties between the joints in the skeletal data.Hence,we also propose a tree structure based skeleton traversal method to explore the kinematic relationship between the joints for better spatial dependency modeling.In addition,since the acquisition of depth sensors is not always accurate,we further improve the design of the ST-LSTM by adding a new gating function, so called“trust gate”,to analyze the reliability of the input data at each spatio-temporal step and give better insight to the network about when to update, forget,or remember the contents of the internal memory cell as the representa-tion of long-term context information.The contributions of this paper are:(1)spatio-temporal design of LSTM networks for3D action recognition,(2)a skeleton-based tree traversal technique to feed the structure of the skeleton data into a sequential LSTM,(3)improving the design of the ST-LSTM by adding the trust gate,and(4)achieving state-of-the-art performance on all the evaluated datasets.2Related WorkHuman action recognition using3D skeleton information is explored in diﬀerent aspects during recent years[33–50].In this section,we limit our review to more recent RNN-based and LSTM-based approaches.HBRNN[30]applied bidirectional RNNs in a novel hierarchical fashion.They divided the entire skeleton toﬁve major groups of joints and each group was fedSpatio-Temporal LSTM with Trust Gates for3D Human Action RecognitionJun Liu1,Amir Shahroudy1,Dong Xu2,and Gang Wang1(B)1School of Electrical and Electronic Engineering,Nanyang Technological University,Singapore,Singapore{jliu029,amir3,wanggang}@.sg2School of Electrical and Information Engineering,University of Sydney,Sydney,Australia******************.auAbstract.3D action recognition–analysis of human actions based on3D skeleton data–becomes popular recently due to its succinctness,robustness,and view-invariant representation.Recent attempts on thisproblem suggested to develop RNN-based learning methods to model thecontextual dependency in the temporal domain.In this paper,we extendthis idea to spatio-temporal domains to analyze the hidden sources ofaction-related information within the input data over both domains con-currently.Inspired by the graphical structure of the human skeleton,wefurther propose a more powerful tree-structure based traversal method.To handle the noise and occlusion in3D skeleton data,we introduce newgating mechanism within LSTM to learn the reliability of the sequentialinput data and accordingly adjust its eﬀect on updating the long-termcontext information stored in the memory cell.Our method achievesstate-of-the-art performance on4challenging benchmark datasets for3D human action analysis.Keywords:3D action recognition·Recurrent neural networks·Longshort-term memory·Trust gate·Spatio-temporal analysis1IntroductionIn recent years,action recognition based on the locations of major joints of the body in3D space has attracted a lot of attention.Diﬀerent feature extraction and classiﬁer learning approaches are studied for3D action recognition[1–3].For example,Yang and Tian[4]represented the static postures and the dynamics of the motion patterns via eigenjoints and utilized a Na¨ıve-Bayes-Nearest-Neighbor classiﬁer learning.A HMM was applied by[5]for modeling the temporal dynam-ics of the actions over a histogram-based representation of3D joint locations. Evangelidis et al.[6]learned a GMM over the Fisher kernel representation of a succinct skeletal feature,called skeletal quads.Vemulapalli et al.[7]represented the skeleton conﬁgurations and actions as points and curves in a Lie group c Springer International Publishing AG2016B.Leibe et al.(Eds.):ECCV2016,Part III,LNCS9907,pp.816–833,2016.DOI:10.1007/978-3-319-46487-950。

Stata MI命令详解说明书

Multiple imputationEstimate model parameters from each imputation, and combine the results in one easy step using mi estimate .Choose from many supported estimation commands, and simply preﬁx them with mi estimate . Select how many imputations to use during estimation, request a detailed MI summary, and more.Missing data occur frequently in practice. MI is one of the most flexible ways of handling missing data. Its three stages are multiply imputing missing values, estimating model parameters from each imputed dataset, and combining multiple estimation results in one ﬁnal inference. In Stata, you can use the mi command to perform these three stages in two simple steps.Use predictive mean matching, linear, logistic, Poisson, and other regressions to impute variables of di erent types. Use multiple imputation using ch ained equations (MICE), multivariate normal imputation (MVN), and monotoneImpute missing values using mi impute .• Support for all three stages of MI: impute missing values,estimate model parameters, and combine estimation results • Imputation– Nine univariate methods– Multivariate methods: MICE (FCS) and MVN – Monotone and arbitrary missing-value pa erns – Add your own methods • Estimation : estimate and combine in one easy step • Inference : linear and nonlinear combinations, hypothesis testing, predictions• MI data : e cient storage, veriﬁcation, import, full data management• Control Panel to guide you through your MI analysis imputation to impute multiple variables. Add your own imputation methods. With MICE, build flexible imputation models—use any of the nine univariate methods, customize prediction equations, include functions of imputed variables, perform conditional imputation, and more.Already have imputed data? Simply import them to Stata for further MI analysis. For example, to import imputed datasets imp1, imp2, ..., imp5 from NHANES, use. mi import nhanes1 mymidata, using(imp{1-5}) id(obs)Impute missing dataEstimate and combine: One easy step© 2023 StataCorp LLC | Stata is a registered trademark of StataCorp LLC, 4905 Lakeway Drive, College Station, TX 77845, USA./multiple-imputationA er estimation, for example, perform hypothesis testing.At any stage of your analysis, perform data management as if you are working with one dataset, and mi will replicate the changes correctly across the imputed datasets. Stata o ers full data management of MI data: create or drop variables and observations, change values, merge or append ﬁles, add imputations, and more.Accidentally dropped an observation from one of the imputed datasets, or changed a value of a variable, or dropped a variable, or ...? Stata veriﬁes the integrity of your MI data each time the mi command is run. (You can also do this manually by using mi update .) For example, Stata checks that complete variables contain the same values in the imputed data as in the original data, that incomplete variables contain the same nonmissing values in the imputed data as in the original,and more.If an inconsistency is detected, Stata tries to ﬁx the problem and notiﬁes you about the result.Estimate transformations of coe cients, compute predictions, and more.Stata’s mi command uniquely provides full data management support, veriﬁcation of integrity of MI data at any step of the analysis, and multiple formats for storing MI data e ciently. And you can even add your own imputation methods!Use an intuitive MI Control Panel to guide you through all the stages of your MI analysis—from examining missing values and th eir pa erns to performing MI inference. Th e corresponding Stata commands are produced with every step for reproducibility and, if desired, later interactive use.Stata o ers several styles for storing your MI data: you can store imputations in one ﬁle or separate ﬁles or in one variable or multiple variables. Some styles are more memory e cient, and others are more computationally e cient. Also, some tasks are easier in speciﬁc styles.You can start with one style at the beginning of your MI analysis, for example, “full long”, in which imputations are saved as extra observations:. mi set flongIf needed, switch to another style during your mi session, for example, to the wide style, in which imputations are saved as extra variables:. mi convert wideCan’t ﬁnd an imputation method you need? With li le e ort, you can program your own. Write a program to impute your variables once, and then simply use it with mi impute to obtain multiple imputations.program mi_impute_cmd_mymethod... program imputing missing values once ...end. mi impute mymethod ..., add(5) ...Manage imputed dataMultiple storage formatsAdd your own imputation methodsVerify imputed dataInferenceIn addition...Control Panel。

《神经网络与深度学习综述DeepLearning15May2014

Draft:Deep Learning in Neural Networks:An OverviewTechnical Report IDSIA-03-14/arXiv:1404.7828(v1.5)[cs.NE]J¨u rgen SchmidhuberThe Swiss AI Lab IDSIAIstituto Dalle Molle di Studi sull’Intelligenza ArtiﬁcialeUniversity of Lugano&SUPSIGalleria2,6928Manno-LuganoSwitzerland15May2014AbstractIn recent years,deep artiﬁcial neural networks(including recurrent ones)have won numerous con-tests in pattern recognition and machine learning.This historical survey compactly summarises relevantwork,much of it from the previous millennium.Shallow and deep learners are distinguished by thedepth of their credit assignment paths,which are chains of possibly learnable,causal links between ac-tions and effects.I review deep supervised learning(also recapitulating the history of backpropagation),unsupervised learning,reinforcement learning&evolutionary computation,and indirect search for shortprograms encoding deep and large networks.PDF of earlier draft(v1):http://www.idsia.ch/∼juergen/DeepLearning30April2014.pdfLATEX source:http://www.idsia.ch/∼juergen/DeepLearning30April2014.texComplete BIBTEXﬁle:http://www.idsia.ch/∼juergen/bib.bibPrefaceThis is the draft of an invited Deep Learning(DL)overview.One of its goals is to assign credit to those who contributed to the present state of the art.I acknowledge the limitations of attempting to achieve this goal.The DL research community itself may be viewed as a continually evolving,deep network of scientists who have inﬂuenced each other in complex ways.Starting from recent DL results,I tried to trace back the origins of relevant ideas through the past half century and beyond,sometimes using“local search”to follow citations of citations backwards in time.Since not all DL publications properly acknowledge earlier relevant work,additional global search strategies were employed,aided by consulting numerous neural network experts.As a result,the present draft mostly consists of references(about800entries so far).Nevertheless,through an expert selection bias I may have missed important work.A related bias was surely introduced by my special familiarity with the work of my own DL research group in the past quarter-century.For these reasons,the present draft should be viewed as merely a snapshot of an ongoing credit assignment process.To help improve it,please do not hesitate to send corrections and suggestions to juergen@idsia.ch.Contents1Introduction to Deep Learning(DL)in Neural Networks(NNs)3 2Event-Oriented Notation for Activation Spreading in FNNs/RNNs3 3Depth of Credit Assignment Paths(CAPs)and of Problems4 4Recurring Themes of Deep Learning54.1Dynamic Programming(DP)for DL (5)4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL (6)4.3Occam’s Razor:Compression and Minimum Description Length(MDL) (6)4.4Learning Hierarchical Representations Through Deep SL,UL,RL (6)4.5Fast Graphics Processing Units(GPUs)for DL in NNs (6)5Supervised NNs,Some Helped by Unsupervised NNs75.11940s and Earlier (7)5.2Around1960:More Neurobiological Inspiration for DL (7)5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) (8)5.41979:Convolution+Weight Replication+Winner-Take-All(WTA) (8)5.51960-1981and Beyond:Development of Backpropagation(BP)for NNs (8)5.5.1BP for Weight-Sharing Feedforward NNs(FNNs)and Recurrent NNs(RNNs)..95.6Late1980s-2000:Numerous Improvements of NNs (9)5.6.1Ideas for Dealing with Long Time Lags and Deep CAPs (10)5.6.2Better BP Through Advanced Gradient Descent (10)5.6.3Discovering Low-Complexity,Problem-Solving NNs (11)5.6.4Potential Beneﬁts of UL for SL (11)5.71987:UL Through Autoencoder(AE)Hierarchies (12)5.81989:BP for Convolutional NNs(CNNs) (13)5.91991:Fundamental Deep Learning Problem of Gradient Descent (13)5.101991:UL-Based History Compression Through a Deep Hierarchy of RNNs (14)5.111992:Max-Pooling(MP):Towards MPCNNs (14)5.121994:Contest-Winning Not So Deep NNs (15)5.131995:Supervised Recurrent Very Deep Learner(LSTM RNN) (15)5.142003:More Contest-Winning/Record-Setting,Often Not So Deep NNs (16)5.152006/7:Deep Belief Networks(DBNs)&AE Stacks Fine-Tuned by BP (17)5.162006/7:Improved CNNs/GPU-CNNs/BP-Trained MPCNNs (17)5.172009:First Ofﬁcial Competitions Won by RNNs,and with MPCNNs (18)5.182010:Plain Backprop(+Distortions)on GPU Yields Excellent Results (18)5.192011:MPCNNs on GPU Achieve Superhuman Vision Performance (18)5.202011:Hessian-Free Optimization for RNNs (19)5.212012:First Contests Won on ImageNet&Object Detection&Segmentation (19)5.222013-:More Contests and Benchmark Records (20)5.22.1Currently Successful Supervised Techniques:LSTM RNNs/GPU-MPCNNs (21)5.23Recent Tricks for Improving SL Deep NNs(Compare Sec.5.6.2,5.6.3) (21)5.24Consequences for Neuroscience (22)5.25DL with Spiking Neurons? (22)6DL in FNNs and RNNs for Reinforcement Learning(RL)236.1RL Through NN World Models Yields RNNs With Deep CAPs (23)6.2Deep FNNs for Traditional RL and Markov Decision Processes(MDPs) (24)6.3Deep RL RNNs for Partially Observable MDPs(POMDPs) (24)6.4RL Facilitated by Deep UL in FNNs and RNNs (25)6.5Deep Hierarchical RL(HRL)and Subgoal Learning with FNNs and RNNs (25)6.6Deep RL by Direct NN Search/Policy Gradients/Evolution (25)6.7Deep RL by Indirect Policy Search/Compressed NN Search (26)6.8Universal RL (27)7Conclusion271Introduction to Deep Learning(DL)in Neural Networks(NNs) Which modiﬁable components of a learning system are responsible for its success or failure?What changes to them improve performance?This has been called the fundamental credit assignment problem(Minsky, 1963).There are general credit assignment methods for universal problem solvers that are time-optimal in various theoretical senses(Sec.6.8).The present survey,however,will focus on the narrower,but now commercially important,subﬁeld of Deep Learning(DL)in Artiﬁcial Neural Networks(NNs).We are interested in accurate credit assignment across possibly many,often nonlinear,computational stages of NNs.Shallow NN-like models have been around for many decades if not centuries(Sec.5.1).Models with several successive nonlinear layers of neurons date back at least to the1960s(Sec.5.3)and1970s(Sec.5.5). An efﬁcient gradient descent method for teacher-based Supervised Learning(SL)in discrete,differentiable networks of arbitrary depth called backpropagation(BP)was developed in the1960s and1970s,and ap-plied to NNs in1981(Sec.5.5).BP-based training of deep NNs with many layers,however,had been found to be difﬁcult in practice by the late1980s(Sec.5.6),and had become an explicit research subject by the early1990s(Sec.5.9).DL became practically feasible to some extent through the help of Unsupervised Learning(UL)(e.g.,Sec.5.10,5.15).The1990s and2000s also saw many improvements of purely super-vised DL(Sec.5).In the new millennium,deep NNs haveﬁnally attracted wide-spread attention,mainly by outperforming alternative machine learning methods such as kernel machines(Vapnik,1995;Sch¨o lkopf et al.,1998)in numerous important applications.In fact,supervised deep NNs have won numerous of-ﬁcial international pattern recognition competitions(e.g.,Sec.5.17,5.19,5.21,5.22),achieving theﬁrst superhuman visual pattern recognition results in limited domains(Sec.5.19).Deep NNs also have become relevant for the more generalﬁeld of Reinforcement Learning(RL)where there is no supervising teacher (Sec.6).Both feedforward(acyclic)NNs(FNNs)and recurrent(cyclic)NNs(RNNs)have won contests(Sec.5.12,5.14,5.17,5.19,5.21,5.22).In a sense,RNNs are the deepest of all NNs(Sec.3)—they are general computers more powerful than FNNs,and can in principle create and process memories of ar-bitrary sequences of input patterns(e.g.,Siegelmann and Sontag,1991;Schmidhuber,1990a).Unlike traditional methods for automatic sequential program synthesis(e.g.,Waldinger and Lee,1969;Balzer, 1985;Soloway,1986;Deville and Lau,1994),RNNs can learn programs that mix sequential and parallel information processing in a natural and efﬁcient way,exploiting the massive parallelism viewed as crucial for sustaining the rapid decline of computation cost observed over the past75years.The rest of this paper is structured as follows.Sec.2introduces a compact,event-oriented notation that is simple yet general enough to accommodate both FNNs and RNNs.Sec.3introduces the concept of Credit Assignment Paths(CAPs)to measure whether learning in a given NN application is of the deep or shallow type.Sec.4lists recurring themes of DL in SL,UL,and RL.Sec.5focuses on SL and UL,and on how UL can facilitate SL,although pure SL has become dominant in recent competitions(Sec.5.17-5.22). Sec.5is arranged in a historical timeline format with subsections on important inspirations and technical contributions.Sec.6on deep RL discusses traditional Dynamic Programming(DP)-based RL combined with gradient-based search techniques for SL or UL in deep NNs,as well as general methods for direct and indirect search in the weight space of deep FNNs and RNNs,including successful policy gradient and evolutionary methods.2Event-Oriented Notation for Activation Spreading in FNNs/RNNs Throughout this paper,let i,j,k,t,p,q,r denote positive integer variables assuming ranges implicit in the given contexts.Let n,m,T denote positive integer constants.An NN’s topology may change over time(e.g.,Fahlman,1991;Ring,1991;Weng et al.,1992;Fritzke, 1994).At any given moment,it can be described as aﬁnite subset of units(or nodes or neurons)N= {u1,u2,...,}and aﬁnite set H⊆N×N of directed edges or connections between nodes.FNNs are acyclic graphs,RNNs cyclic.Theﬁrst(input)layer is the set of input units,a subset of N.In FNNs,the k-th layer(k>1)is the set of all nodes u∈N such that there is an edge path of length k−1(but no longer path)between some input unit and u.There may be shortcut connections between distant layers.The NN’s behavior or program is determined by a set of real-valued,possibly modiﬁable,parameters or weights w i(i=1,...,n).We now focus on a singleﬁnite episode or epoch of information processing and activation spreading,without learning through weight changes.The following slightly unconventional notation is designed to compactly describe what is happening during the runtime of the system.During an episode,there is a partially causal sequence x t(t=1,...,T)of real values that I call events.Each x t is either an input set by the environment,or the activation of a unit that may directly depend on other x k(k<t)through a current NN topology-dependent set in t of indices k representing incoming causal connections or links.Let the function v encode topology information and map such event index pairs(k,t)to weight indices.For example,in the non-input case we may have x t=f t(net t)with real-valued net t= k∈in t x k w v(k,t)(additive case)or net t= k∈in t x k w v(k,t)(multiplicative case), where f t is a typically nonlinear real-valued activation function such as tanh.In many recent competition-winning NNs(Sec.5.19,5.21,5.22)there also are events of the type x t=max k∈int (x k);some networktypes may also use complex polynomial activation functions(Sec.5.3).x t may directly affect certain x k(k>t)through outgoing connections or links represented through a current set out t of indices k with t∈in k.Some non-input events are called output events.Note that many of the x t may refer to different,time-varying activations of the same unit in sequence-processing RNNs(e.g.,Williams,1989,“unfolding in time”),or also in FNNs sequentially exposed to time-varying input patterns of a large training set encoded as input events.During an episode,the same weight may get reused over and over again in topology-dependent ways,e.g.,in RNNs,or in convolutional NNs(Sec.5.4,5.8).I call this weight sharing across space and/or time.Weight sharing may greatly reduce the NN’s descriptive complexity,which is the number of bits of information required to describe the NN (Sec.4.3).In Supervised Learning(SL),certain NN output events x t may be associated with teacher-given,real-valued labels or targets d t yielding errors e t,e.g.,e t=1/2(x t−d t)2.A typical goal of supervised NN training is toﬁnd weights that yield episodes with small total error E,the sum of all such e t.The hope is that the NN will generalize well in later episodes,causing only small errors on previously unseen sequences of input events.Many alternative error functions for SL and UL are possible.SL assumes that input events are independent of earlier output events(which may affect the environ-ment through actions causing subsequent perceptions).This assumption does not hold in the broaderﬁelds of Sequential Decision Making and Reinforcement Learning(RL)(Kaelbling et al.,1996;Sutton and Barto, 1998;Hutter,2005)(Sec.6).In RL,some of the input events may encode real-valued reward signals given by the environment,and a typical goal is toﬁnd weights that yield episodes with a high sum of reward signals,through sequences of appropriate output actions.Sec.5.5will use the notation above to compactly describe a central algorithm of DL,namely,back-propagation(BP)for supervised weight-sharing FNNs and RNNs.(FNNs may be viewed as RNNs with certainﬁxed zero weights.)Sec.6will address the more general RL case.3Depth of Credit Assignment Paths(CAPs)and of ProblemsTo measure whether credit assignment in a given NN application is of the deep or shallow type,I introduce the concept of Credit Assignment Paths or CAPs,which are chains of possibly causal links between events.Let usﬁrst focus on SL.Consider two events x p and x q(1≤p<q≤T).Depending on the appli-cation,they may have a Potential Direct Causal Connection(PDCC)expressed by the Boolean predicate pdcc(p,q),which is true if and only if p∈in q.Then the2-element list(p,q)is deﬁned to be a CAP from p to q(a minimal one).A learning algorithm may be allowed to change w v(p,q)to improve performance in future episodes.More general,possibly indirect,Potential Causal Connections(PCC)are expressed by the recursively deﬁned Boolean predicate pcc(p,q),which in the SL case is true only if pdcc(p,q),or if pcc(p,k)for some k and pdcc(k,q).In the latter case,appending q to any CAP from p to k yields a CAP from p to q(this is a recursive deﬁnition,too).The set of such CAPs may be large but isﬁnite.Note that the same weight may affect many different PDCCs between successive events listed by a given CAP,e.g.,in the case of RNNs, or weight-sharing FNNs.Suppose a CAP has the form(...,k,t,...,q),where k and t(possibly t=q)are theﬁrst successive elements with modiﬁable w v(k,t).Then the length of the sufﬁx list(t,...,q)is called the CAP’s depth (which is0if there are no modiﬁable links at all).This depth limits how far backwards credit assignment can move down the causal chain toﬁnd a modiﬁable weight.1Suppose an episode and its event sequence x1,...,x T satisfy a computable criterion used to decide whether a given problem has been solved(e.g.,total error E below some threshold).Then the set of used weights is called a solution to the problem,and the depth of the deepest CAP within the sequence is called the solution’s depth.There may be other solutions(yielding different event sequences)with different depths.Given someﬁxed NN topology,the smallest depth of any solution is called the problem’s depth.Sometimes we also speak of the depth of an architecture:SL FNNs withﬁxed topology imply a problem-independent maximal problem depth bounded by the number of non-input layers.Certain SL RNNs withﬁxed weights for all connections except those to output units(Jaeger,2001;Maass et al.,2002; Jaeger,2004;Schrauwen et al.,2007)have a maximal problem depth of1,because only theﬁnal links in the corresponding CAPs are modiﬁable.In general,however,RNNs may learn to solve problems of potentially unlimited depth.Note that the deﬁnitions above are solely based on the depths of causal chains,and agnostic of the temporal distance between events.For example,shallow FNNs perceiving large“time windows”of in-put events may correctly classify long input sequences through appropriate output events,and thus solve shallow problems involving long time lags between relevant events.At which problem depth does Shallow Learning end,and Deep Learning begin?Discussions with DL experts have not yet yielded a conclusive response to this question.Instead of committing myself to a precise answer,let me just deﬁne for the purposes of this overview:problems of depth>10require Very Deep Learning.The difﬁculty of a problem may have little to do with its depth.Some NNs can quickly learn to solve certain deep problems,e.g.,through random weight guessing(Sec.5.9)or other types of direct search (Sec.6.6)or indirect search(Sec.6.7)in weight space,or through training an NNﬁrst on shallow problems whose solutions may then generalize to deep problems,or through collapsing sequences of(non)linear operations into a single(non)linear operation—but see an analysis of non-trivial aspects of deep linear networks(Baldi and Hornik,1994,Section B).In general,however,ﬁnding an NN that precisely models a given training set is an NP-complete problem(Judd,1990;Blum and Rivest,1992),also in the case of deep NNs(S´ıma,1994;de Souto et al.,1999;Windisch,2005);compare a survey of negative results(S´ıma, 2002,Section1).Above we have focused on SL.In the more general case of RL in unknown environments,pcc(p,q) is also true if x p is an output event and x q any later input event—any action may affect the environment and thus any later perception.(In the real world,the environment may even inﬂuence non-input events computed on a physical hardware entangled with the entire universe,but this is ignored here.)It is possible to model and replace such unmodiﬁable environmental PCCs through a part of the NN that has already learned to predict(through some of its units)input events(including reward signals)from former input events and actions(Sec.6.1).Its weights are frozen,but can help to assign credit to other,still modiﬁable weights used to compute actions(Sec.6.1).This approach may lead to very deep CAPs though.Some DL research is about automatically rephrasing problems such that their depth is reduced(Sec.4). In particular,sometimes UL is used to make SL problems less deep,e.g.,Sec.5.10.Often Dynamic Programming(Sec.4.1)is used to facilitate certain traditional RL problems,e.g.,Sec.6.2.Sec.5focuses on CAPs for SL,Sec.6on the more complex case of RL.4Recurring Themes of Deep Learning4.1Dynamic Programming(DP)for DLOne recurring theme of DL is Dynamic Programming(DP)(Bellman,1957),which can help to facili-tate credit assignment under certain assumptions.For example,in SL NNs,backpropagation itself can 1An alternative would be to count only modiﬁable links when measuring depth.In many typical NN applications this would not make a difference,but in some it would,e.g.,Sec.6.1.be viewed as a DP-derived method(Sec.5.5).In traditional RL based on strong Markovian assumptions, DP-derived methods can help to greatly reduce problem depth(Sec.6.2).DP algorithms are also essen-tial for systems that combine concepts of NNs and graphical models,such as Hidden Markov Models (HMMs)(Stratonovich,1960;Baum and Petrie,1966)and Expectation Maximization(EM)(Dempster et al.,1977),e.g.,(Bottou,1991;Bengio,1991;Bourlard and Morgan,1994;Baldi and Chauvin,1996; Jordan and Sejnowski,2001;Bishop,2006;Poon and Domingos,2011;Dahl et al.,2012;Hinton et al., 2012a).4.2Unsupervised Learning(UL)Facilitating Supervised Learning(SL)and RL Another recurring theme is how UL can facilitate both SL(Sec.5)and RL(Sec.6).UL(Sec.5.6.4) is normally used to encode raw incoming data such as video or speech streams in a form that is more convenient for subsequent goal-directed learning.In particular,codes that describe the original data in a less redundant or more compact way can be fed into SL(Sec.5.10,5.15)or RL machines(Sec.6.4),whose search spaces may thus become smaller(and whose CAPs shallower)than those necessary for dealing with the raw data.UL is closely connected to the topics of regularization and compression(Sec.4.3,5.6.3). 4.3Occam’s Razor:Compression and Minimum Description Length(MDL) Occam’s razor favors simple solutions over complex ones.Given some programming language,the prin-ciple of Minimum Description Length(MDL)can be used to measure the complexity of a solution candi-date by the length of the shortest program that computes it(e.g.,Solomonoff,1964;Kolmogorov,1965b; Chaitin,1966;Wallace and Boulton,1968;Levin,1973a;Rissanen,1986;Blumer et al.,1987;Li and Vit´a nyi,1997;Gr¨u nwald et al.,2005).Some methods explicitly take into account program runtime(Al-lender,1992;Watanabe,1992;Schmidhuber,2002,1995);many consider only programs with constant runtime,written in non-universal programming languages(e.g.,Rissanen,1986;Hinton and van Camp, 1993).In the NN case,the MDL principle suggests that low NN weight complexity corresponds to high NN probability in the Bayesian view(e.g.,MacKay,1992;Buntine and Weigend,1991;De Freitas,2003), and to high generalization performance(e.g.,Baum and Haussler,1989),without overﬁtting the training data.Many methods have been proposed for regularizing NNs,that is,searching for solution-computing, low-complexity SL NNs(Sec.5.6.3)and RL NNs(Sec.6.7).This is closely related to certain UL methods (Sec.4.2,5.6.4).4.4Learning Hierarchical Representations Through Deep SL,UL,RLMany methods of Good Old-Fashioned Artiﬁcial Intelligence(GOFAI)(Nilsson,1980)as well as more recent approaches to AI(Russell et al.,1995)and Machine Learning(Mitchell,1997)learn hierarchies of more and more abstract data representations.For example,certain methods of syntactic pattern recog-nition(Fu,1977)such as grammar induction discover hierarchies of formal rules to model observations. The partially(un)supervised Automated Mathematician/EURISKO(Lenat,1983;Lenat and Brown,1984) continually learns concepts by combining previously learnt concepts.Such hierarchical representation learning(Ring,1994;Bengio et al.,2013;Deng and Yu,2014)is also a recurring theme of DL NNs for SL (Sec.5),UL-aided SL(Sec.5.7,5.10,5.15),and hierarchical RL(Sec.6.5).Often,abstract hierarchical representations are natural by-products of data compression(Sec.4.3),e.g.,Sec.5.10.4.5Fast Graphics Processing Units(GPUs)for DL in NNsWhile the previous millennium saw several attempts at creating fast NN-speciﬁc hardware(e.g.,Jackel et al.,1990;Faggin,1992;Ramacher et al.,1993;Widrow et al.,1994;Heemskerk,1995;Korkin et al., 1997;Urlbe,1999),and at exploiting standard hardware(e.g.,Anguita et al.,1994;Muller et al.,1995; Anguita and Gomes,1996),the new millennium brought a DL breakthrough in form of cheap,multi-processor graphics cards or GPUs.GPUs are widely used for video games,a huge and competitive market that has driven down hardware prices.GPUs excel at fast matrix and vector multiplications required not only for convincing virtual realities but also for NN training,where they can speed up learning by a factorof50and more.Some of the GPU-based FNN implementations(Sec.5.16-5.19)have greatly contributed to recent successes in contests for pattern recognition(Sec.5.19-5.22),image segmentation(Sec.5.21), and object detection(Sec.5.21-5.22).5Supervised NNs,Some Helped by Unsupervised NNsThe main focus of current practical applications is on Supervised Learning(SL),which has dominated re-cent pattern recognition contests(Sec.5.17-5.22).Several methods,however,use additional Unsupervised Learning(UL)to facilitate SL(Sec.5.7,5.10,5.15).It does make sense to treat SL and UL in the same section:often gradient-based methods,such as BP(Sec.5.5.1),are used to optimize objective functions of both UL and SL,and the boundary between SL and UL may blur,for example,when it comes to time series prediction and sequence classiﬁcation,e.g.,Sec.5.10,5.12.A historical timeline format will help to arrange subsections on important inspirations and techni-cal contributions(although such a subsection may span a time interval of many years).Sec.5.1brieﬂy mentions early,shallow NN models since the1940s,Sec.5.2additional early neurobiological inspiration relevant for modern Deep Learning(DL).Sec.5.3is about GMDH networks(since1965),perhaps theﬁrst (feedforward)DL systems.Sec.5.4is about the relatively deep Neocognitron NN(1979)which is similar to certain modern deep FNN architectures,as it combines convolutional NNs(CNNs),weight pattern repli-cation,and winner-take-all(WTA)mechanisms.Sec.5.5uses the notation of Sec.2to compactly describe a central algorithm of DL,namely,backpropagation(BP)for supervised weight-sharing FNNs and RNNs. It also summarizes the history of BP1960-1981and beyond.Sec.5.6describes problems encountered in the late1980s with BP for deep NNs,and mentions several ideas from the previous millennium to overcome them.Sec.5.7discusses aﬁrst hierarchical stack of coupled UL-based Autoencoders(AEs)—this concept resurfaced in the new millennium(Sec.5.15).Sec.5.8is about applying BP to CNNs,which is important for today’s DL applications.Sec.5.9explains BP’s Fundamental DL Problem(of vanishing/exploding gradients)discovered in1991.Sec.5.10explains how a deep RNN stack of1991(the History Compressor) pre-trained by UL helped to solve previously unlearnable DL benchmarks requiring Credit Assignment Paths(CAPs,Sec.3)of depth1000and more.Sec.5.11discusses a particular WTA method called Max-Pooling(MP)important in today’s DL FNNs.Sec.5.12mentions aﬁrst important contest won by SL NNs in1994.Sec.5.13describes a purely supervised DL RNN(Long Short-Term Memory,LSTM)for problems of depth1000and more.Sec.5.14mentions an early contest of2003won by an ensemble of shallow NNs, as well as good pattern recognition results with CNNs and LSTM RNNs(2003).Sec.5.15is mostly about Deep Belief Networks(DBNs,2006)and related stacks of Autoencoders(AEs,Sec.5.7)pre-trained by UL to facilitate BP-based SL.Sec.5.16mentions theﬁrst BP-trained MPCNNs(2007)and GPU-CNNs(2006). Sec.5.17-5.22focus on ofﬁcial competitions with secret test sets won by(mostly purely supervised)DL NNs since2009,in sequence recognition,image classiﬁcation,image segmentation,and object detection. Many RNN results depended on LSTM(Sec.5.13);many FNN results depended on GPU-based FNN code developed since2004(Sec.5.16,5.17,5.18,5.19),in particular,GPU-MPCNNs(Sec.5.19).5.11940s and EarlierNN research started in the1940s(e.g.,McCulloch and Pitts,1943;Hebb,1949);compare also later work on learning NNs(Rosenblatt,1958,1962;Widrow and Hoff,1962;Grossberg,1969;Kohonen,1972; von der Malsburg,1973;Narendra and Thathatchar,1974;Willshaw and von der Malsburg,1976;Palm, 1980;Hopﬁeld,1982).In a sense NNs have been around even longer,since early supervised NNs were essentially variants of linear regression methods going back at least to the early1800s(e.g.,Legendre, 1805;Gauss,1809,1821).Early NNs had a maximal CAP depth of1(Sec.3).5.2Around1960:More Neurobiological Inspiration for DLSimple cells and complex cells were found in the cat’s visual cortex(e.g.,Hubel and Wiesel,1962;Wiesel and Hubel,1959).These cellsﬁre in response to certain properties of visual sensory inputs,such as theorientation of plex cells exhibit more spatial invariance than simple cells.This inspired later deep NN architectures(Sec.5.4)used in certain modern award-winning Deep Learners(Sec.5.19-5.22).5.31965:Deep Networks Based on the Group Method of Data Handling(GMDH) Networks trained by the Group Method of Data Handling(GMDH)(Ivakhnenko and Lapa,1965; Ivakhnenko et al.,1967;Ivakhnenko,1968,1971)were perhaps theﬁrst DL systems of the Feedforward Multilayer Perceptron type.The units of GMDH nets may have polynomial activation functions imple-menting Kolmogorov-Gabor polynomials(more general than traditional NN activation functions).Given a training set,layers are incrementally grown and trained by regression analysis,then pruned with the help of a separate validation set(using today’s terminology),where Decision Regularisation is used to weed out superﬂuous units.The numbers of layers and units per layer can be learned in problem-dependent fashion. This is a good example of hierarchical representation learning(Sec.4.4).There have been numerous ap-plications of GMDH-style networks,e.g.(Ikeda et al.,1976;Farlow,1984;Madala and Ivakhnenko,1994; Ivakhnenko,1995;Kondo,1998;Kord´ık et al.,2003;Witczak et al.,2006;Kondo and Ueno,2008).5.41979:Convolution+Weight Replication+Winner-Take-All(WTA)Apart from deep GMDH networks(Sec.5.3),the Neocognitron(Fukushima,1979,1980,2013a)was per-haps theﬁrst artiﬁcial NN that deserved the attribute deep,and theﬁrst to incorporate the neurophysiolog-ical insights of Sec.5.2.It introduced convolutional NNs(today often called CNNs or convnets),where the(typically rectangular)receptiveﬁeld of a convolutional unit with given weight vector is shifted step by step across a2-dimensional array of input values,such as the pixels of an image.The resulting2D array of subsequent activation events of this unit can then provide inputs to higher-level units,and so on.Due to massive weight replication(Sec.2),relatively few parameters may be necessary to describe the behavior of such a convolutional layer.Competition layers have WTA subsets whose maximally active units are the only ones to adopt non-zero activation values.They essentially“down-sample”the competition layer’s input.This helps to create units whose responses are insensitive to small image shifts(compare Sec.5.2).The Neocognitron is very similar to the architecture of modern,contest-winning,purely super-vised,feedforward,gradient-based Deep Learners with alternating convolutional and competition lay-ers(e.g.,Sec.5.19-5.22).Fukushima,however,did not set the weights by supervised backpropagation (Sec.5.5,5.8),but by local un supervised learning rules(e.g.,Fukushima,2013b),or by pre-wiring.In that sense he did not care for the DL problem(Sec.5.9),although his architecture was comparatively deep indeed.He also used Spatial Averaging(Fukushima,1980,2011)instead of Max-Pooling(MP,Sec.5.11), currently a particularly convenient and popular WTA mechanism.Today’s CNN-based DL machines proﬁta lot from later CNN work(e.g.,LeCun et al.,1989;Ranzato et al.,2007)(Sec.5.8,5.16,5.19).5.51960-1981and Beyond:Development of Backpropagation(BP)for NNsThe minimisation of errors through gradient descent(Hadamard,1908)in the parameter space of com-plex,nonlinear,differentiable,multi-stage,NN-related systems has been discussed at least since the early 1960s(e.g.,Kelley,1960;Bryson,1961;Bryson and Denham,1961;Pontryagin et al.,1961;Dreyfus,1962; Wilkinson,1965;Amari,1967;Bryson and Ho,1969;Director and Rohrer,1969;Griewank,2012),ini-tially within the framework of Euler-LaGrange equations in the Calculus of Variations(e.g.,Euler,1744). Steepest descent in such systems can be performed(Bryson,1961;Kelley,1960;Bryson and Ho,1969)by iterating the ancient chain rule(Leibniz,1676;L’Hˆo pital,1696)in Dynamic Programming(DP)style(Bell-man,1957).A simpliﬁed derivation of the method uses the chain rule only(Dreyfus,1962).The methods of the1960s were already efﬁcient in the DP sense.However,they backpropagated derivative information through standard Jacobian matrix calculations from one“layer”to the previous one, explicitly addressing neither direct links across several layers nor potential additional efﬁciency gains due to network sparsity(but perhaps such enhancements seemed obvious to the authors).。

mobilenetv3 激活函数

MobileNetV3激活函数详解MobileNetV3是一种轻量级的卷积神经网络模型，广泛应用于移动设备和嵌入式系统中。

激活函数在神经网络中起到了至关重要的作用，能够引入非线性因素，增加模型的表达能力。

MobileNetV3中使用了一些特定的激活函数，本文将详细解释这些函数的定义、用途和工作方式。

1. Hard SwishHard Swish是MobileNetV3中最常用的激活函数之一。

它是由Google提出的一种非线性激活函数，结合了ReLU和Swish两种激活函数的优点。

Hard Swish函数定义如下：f(x) = x * relu6(x + 3) / 6其中relu6(x)表示输入x与0之间取较大值，并将结果限制在[0, 6]范围内。

Hard Swish可以看作是ReLU和Swish的结合体，具有ReLU快速收敛和Swish平滑非线性特点。

Hard Swish在MobileNetV3中被广泛应用于卷积层之后，能够有效地提高模型的准确率，并且具有计算效率高、参数量少等优势。

2. Hard SigmoidHard Sigmoid是另一种常见的激活函数，也是MobileNetV3中使用的激活函数之一。

它是对Sigmoid函数的一种近似，能够在保持计算效率的同时引入非线性因素。

Hard Sigmoid函数定义如下：f(x) = relu6(x + 3) / 6其中relu6(x)同样表示输入x与0之间取较大值，并将结果限制在[0, 6]范围内。

Hard Sigmoid可以看作是对Sigmoid函数进行了截断，去掉了曲线两端的平缓区域，使其更加接近于阶跃函数。

Hard Sigmoid在MobileNetV3中主要被用于深度可分离卷积层(Depthwise Separable Convolution)，能够有效地提高模型的准确率，并且具有计算效率高、参数量少等优势。

3. MishMish是一种新颖的激活函数，由Misra提出。

提示工程(prompt)

客户问题：你好，我有一个关于我的汽车保险的问题。分类主题：1
客户问题：{在此插入新问题} 分类主题：
之前
单个家庭的年用水需求是多少？包含四个在家的人的家庭每年平均200天，使用量平均为100 每天多少升水？
单户家庭的年用水需求包含四个人，他们平均在家的时间为每年200天，平均使用100升的每天的用水量为每年800万升。
2nd prompt—答案提取
一个普通人在一分钟内能打出25拳。一场比赛持续5个回合，每回合3分钟。有多少他投掷了多少拳？
让我们一步一步地思考并解释这个计算过程。一步一步地。
在一分钟内，乔打了25拳。
在三分钟内，乔打了3 * 25 = 75拳。
在五轮中，乔投掷了5*75 = 375次拳。
在一分钟内，乔打了25拳。在三分钟内，乔打了3 * 25 = 75拳。在五轮中，乔投掷了5*75 = 375次拳。
提示链
摘要
请总结信息关于产品 [{'text': 'iPhone', 'type': '产品'}], {'text': '苹果', 'type': 'ORGANIZATION’}]
iPhone是一系列智能手机。由苹果设计并销售 Inc. 这些设备结合了计算机、相机和互联网单一设备中的连接性，以及以其简洁的设计而闻名用户友好的界面。
人工智能机器学习深度学习
生成式人工智能
1956
人工智能
计算机科学领域中致力于创造的分支能够复制或超越人类智能的智能机器人类智能
1997
机器学习
人工智能的子集，使机器能够从现有数据，并在该数据的基础上进行改进，以使其决策或预测

1、下载文档前请自行甄别文档内容的完整性，平台不提供额外的编辑、内容补充、找答案等附加服务。
2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
3、如文档侵犯您的权益，请联系客服反馈,我们会尽快为您处理(人工客服工作时间：9:00-18:30)。