The Minimum-Norm-Point Algorithm Applied to Submodular Function Minimization and
状态压缩

状态压缩Abstract信息学发展势头迅猛,信息学奥赛的题目来源遍及各行各业,经常有一些在实际应用中很有价值的问题被引入信息学并得到有效解决。
然而有一些问题却被认为很可能不存在有效的(多项式级的)算法,本文以对几个例题的剖析,简述状态压缩思想及其应用。
Keywords状态压缩、集合、Hash、NPCContentIntroducti o n作为OIers,我们不同程度地知道各式各样的算法。
这些算法有的以O(logn)的复杂度运行,如二分查找、欧几里德GCD算法(连续两次迭代后的余数至多为原数的一半)、平衡树,有的以O(n)运行,例如二级索引、块状链表,再往上有O(n)、O(n p log q n)……大部分问题的算法都有一个多项式级别的时间复杂度上界1,我们一般称这类问题2为P类(deterministic Polynomial-time)问题,例如在有向图中求最短路径。
然而存在几类问题,至今仍未被很好地解决,人们怀疑他们根本没有多项式时间复杂度的算法,它们是NPC(NP-Complete)和NPH(NP-Hard)类,例如问一个图是否存在哈密顿圈(NPC)、问一个图是否不存在哈密顿圈(NPH)、求一个完全图中最短的哈密顿圈(即经典的Traveling Salesman Problem货郎担问题,NPH)、在有向图中求最长(简单)路径(NPH),对这些问题尚不知有多项式时间的算法存在。
P和NPC都是NP(Non-deterministic Polynomial-time)的子集,NPC则代表了NP类中最难的一类问题,所有的NP类问题都可以在多项式时间内归约到NPC问题中去。
NPH包含了NPC和其他一些不属于NP(也更难)的问题(即NPC是NP与NPH的交集),NPC问题的最优化版本一般是NPH的,例如问一个图是否存在哈密顿圈是NPC的,但求最短的哈密顿圈则是NPH的,原因在于我们可以在多项式时间内验证一个回路是否真的是哈密顿回路,却无法在多项式时间内验证其是否是最短的,NP类要求能在多项式时间内验证问题的一个解是否真的是一个解,所以最优化TSP问题不是NP的,而是NPH的。
1978-More-The Levenberg-Marquardt algorithm-implementation and theory

but if D is diagonal, then E has axes along the coordinate directions and the length of the ith semi-axis is A/d..
i
We now consider the solution of (2.1) in some generality, and thus the problem (2.3) min{IIf+Jpll : IIDpll~ A} The basis for the Levenberg-Marquardt = p(l) for some
THE LEVE_~BERG-MARQUARDT IMPLEMENTATION
ALGORITHM:
AND THEORY
Jorge J. Mor~
i.
Introduction Let F: R n ÷ R m be continuously differentiable, of and consider the nonlinear least
4
ilI] I
where D~ = x½~TD~ is still a diagonal matrix and R is a (possibly singular) upper triangular matrix of order n.
108
In the second stage, compute the QR decomposition (3.4).
squares problem of finding a local minimizer
(1.1)
Levenberg numerical
I ~(x) = 7
交替极小化算法

交替极小化算法交替极小化算法(Alternating Direction Method of Multipliers,ADMM)是一种用于解决带有线性或者凸二次约束的优化问题的迭代算法。
它最初由Gabriel Peyré和Stéphane Mallat在2010年提出,并在之后的几年里得到了广泛的应用和研究。
ADMM算法的基本思想是将原始问题转化为一系列等价的子问题,然后通过交替优化这些子问题来逐步逼近原始问题的最优解。
具体来说,ADMM算法将原始问题表示为如下形式:$$\min_{x,z} f(x) + g(z)$$$$\text{s.t. } Ax+Bz=c$$其中$f(x)$和$g(z)$分别是$x$和$z$的凸函数,$A,B$是线性变换矩阵,$c$是常数向量。
ADMM算法通过增加一个拉格朗日乘子项$\frac{\rho}{2}\|Ax+Bz-c+\lambda\|^2$将上述问题转化为如下形式:$$\min_{x,z} f(x) + g(z) + \frac{\rho}{2}\|Ax+Bz-c+\lambda\|^2$$其中$\rho>0$是一个常数,$\lambda$是拉格朗日乘子向量。
然后,ADMM算法通过交替优化以下三个子问题来求解上述问题:1. $x$子问题:$$x^{k+1} = \arg\min_x f(x) +\frac{\rho}{2}\|Ax^{k}+Bz^k-c+\lambda^k\|^2$$2. $z$子问题:$$z^{k+1} = \arg\min_z g(z) +\frac{\rho}{2}\|Ax^{k+1}+Bz-c+\lambda^k\|^2$$3. $\lambda$子问题:$$\lambda^{k+1} = \lambda^k +Ax^{k+1}+Bz^{k+1}-c$$其中$k$是迭代次数。
ADMM算法的优点在于它可以处理大规模的优化问题,并且可以应用于分布式计算环境中。
整数规划 孙小玲 课件2

Example. Consider the following 0-1 knapsack problem: max 8x1 + 11x2 + 6x3 + 4x4 s.t. 5x1 + 7x2 + 4x3 + 3x4 ≤ 14 x ∈ {0, 1}4 .
5/1
6/1
7/1
8/1
Solving Integer Knapsack by B&B
◮ ◮ ◮ ◮
◮
◮
◮
◮
Best First Depth-First Hybrid Strategies Best Estimate
◮
◮
One way to minimize overall solution time is to try to minimize the size of the search tree. We can achieve this by choosing the subproblem with the best bound (lowest lower bound if we are minimizing). A candidate node is said to be critical if its lower bound is also a lower bound of the original IP. Every critical node will be processed no matter what the search order Best first is guaranteed to examine only critical nodes, thereby minimizing the size of the search tree. Drawbacks of Best First
图形科学英文版PPT课件image (9)

assoc(A,V) = sum of weights of all edges in V that touch A
Perceptual and Sensory Augmented Computing Computer Vision WS 08/09
Slide credit: Svetlana Lazebnik
B. Leibe
5
Recap: NCuts: Overall Procedure
1. Construct a weighted graph G=(V,E) from an image. 2. Connect each pair of pixels, and assign graph edge
weights,
Perceptual and Sensory Augmented Computing Computer Vision WS 08/09
W (i, j ) Prob. that i and j belong to the same region.
3. Solve ( D W ) y Dy for the smallest few
Part based Approaches
• 3D Reconstruction • Motion and Tracking
2
Recap: Image Segmentation
• Goal: identify groups of pixels that go together
Perceptual and Sensory Augmented Computing Computer Vision WS 08/09
新型滑动最小二乘技术的实现及其在无网格法中应用

2 滑动最小二乘(MLS)近似
B. Nayroles 等[2]提出的 DEM 和 T. Belytschko 等[3]的 EFGM 都以 MLS 近似为基础。在边界为Γ 的 域Ω 中,函数 u(x)可拟出插值 u ( x) :
h
n I m j
将基函数正交化,得
u h = ∑∑ p j ( x)( A −1 ( x) B( x)) jI u I
第 25 卷
增 1
2006 年 2 月
岩石力学与工程学报 Chinese Journal of Rock Mechanics and Engineering
Vol.25 Supp.1 Feb., 2006
新型滑动最小二乘技术的实现及其 在无网格法中应用
张延军 1,王恩志 2
(1. 吉林大学 建设工程学院,吉林 长春 130026;2. 清华大学 水利水电工程系,北京 100084)
2D
• 3005 •
d=
i =1,n − 2 j =i +1,n−1 k ≠ j,n
∑ ∑ ∑ w w w (Θ
( xi,x j,xk )) 2
(8)
权函数,在节点处的计算点取一微位移 ε = 10−3 ,其 余按常规 MLS 近似求解,可计算得到基函数 α 1 ,
Θ 2 D ( xi,x j,xk ) = − x j yi + xk yi + xi y j − xk y j − xi yk − x j yk
Sp (i ) ( x) , α i ( x) 为正规方程的系数。上述公式中,
当包含奇异权函数 w( x,xk ) 的函数奇异性发生时,
可通过取极限方法避免奇异性的发生[6
3.2 Breitkopf 给出的公式求解法
基于Kriging模型的自适应多阶段并行代理优化算法
第27卷第11期2021年11月计算机集成制造系统Vol.27No.11 Computer Integrated Manufacturing Systems Nov.2021DOI:10.13196/j.cims.2021.11.016基于Kriging模型的自适应多阶段并行代理优化算法乐春宇,马义中+(南京理工大学经济管理学院,江苏南京210094)摘要:为了充分利用计算资源,减少迭代次数,提出一种可以批量加点的代理优化算法。
该算法分别采用期望改进准则和WB2(Watson and Barnes)准则探索存在的最优解并开发已存在最优解的区域,利用可行性概率和多目标优化框架刻画约束边界。
在探索和开发阶段,设计了两种对应的多点填充算法,并根据新样本点和已知样本点的距离关系,设计了两个阶段的自适应切换策略。
通过3个不同类型算例和一个工程实例验证算法性能,结果表明,该算法收敛更快,其结果具有较好的精确性和稳健性。
关键词:Kriging模型;代理优化;加点准则;可行性概率;多点填充中图分类号:O212.6文献标识码:AParallel surrogate-based optimization algorithm based on Kriging model usingadaptive multi-phases strategyYUE Chunyu,MA Yizhong+(School o£Economics and Management,Nanjing University of Science and Technology,Nanjing210094,China) Abstract:To make full use of computing resources and reduce the number of iterations,a surrogate-based optimization algorithm which could add batch points was proposed.To explore the optimum solution and to exploit its area, the expected improvement and the WB2criterion were used correspondingly.The constraint boundary was characterized by using the probability of feasibility and the multi-objective optimization framework.Two corresponding multi-points infilling algorithms were designed in the exploration and exploitation phases and an adaptive switching strategy for this two phases was designed according to the distance between new sample points and known sample points.The performance of the algorithm was verified by three different types of numerical and one engineering benchmarks.The results showed that the proposed algorithm was more efficient in convergence and the solution was more precise and robust.Keywords:Kriging model;surrogate-based optimization;infill sampling criteria;probabil让y of feasibility;multipoints infill0引言现代工程优化设计中,常采用高精度仿真模型获取数据,如有限元分析和流体动力学等E,如何在优化过程中尽可能少地调用高精度仿真模型,以提高优化效率,显得尤为重要。
fmincon interior point理解
fmincon interior point理解`fmincon`的内点算法(interior-point algorithm)是一种用于求解约束优化问题的数学规划方法。
它通过在可行域内部寻找一个点,使得目标函数在该点处取得最小值,同时满足所有给定的约束条件。
以下是对`fmincon`内点算法的一些理解:1. 重置违反边界的变量:在使用默认的内点算法时,如果某个变量的初始值或迭代过程中的值超出了设定的边界(lb ≤x ≤ub),`fmincon`会将其重置到边界区域内部,而不是简单地将其设置为边界值。
2. 处理非线性约束:`fmincon`不仅可以处理线性约束,还能处理非线性约束。
在进行优化时,它会尝试满足由`nonlcon`定义的非线性不等式c(x) ≤0 和等式ceq(x) = 0。
3. 利用导数信息:当提供目标函数的梯度(即导数)信息时,内点算法可以更快地找到更为准确的解。
如果还提供了黑塞矩阵(Hessian matrix),则可以进一步提高求解过程的效率和稳定性。
4. 算法比较:与`fmincon`中的其他算法相比,内点算法在处理边界条件时有其独特的方式。
例如,对于'trust-region-reflective'算法,违反边界的分量也会被重置到边界区域内部,而其他算法可能会将违反边界的分量设置为最近的边界值。
5. 适用范围:内点算法适用于解决凸优化问题,特别是线性规划和二次规划问题。
它在运筹学和工程优化领域中有着广泛的应用。
6. MATLAB实现:在MATLAB中,可以通过调用`fmincon`函数并设置相应的选项来使用内点算法。
这通常涉及到定义目标函数、初始值、线性和非线性约束条件,以及变量的边界。
总的来说,内点算法是一种强大且常用的优化算法,尤其适合于处理带有复杂约束条件的优化问题。
通过合理地设置算法参数和使用导数信息,可以提高求解过程的效率和准确性。
带有混合约束的二次半定规划的内点算法
称 矩 阵 , 阶半 正定 矩 阵与 ”阶正 定矩 阵 , ( 表 示 X 的特 征 值 。定 义 内积 为 : C, X) < x> 一t( C) F 范 r x , -
数 为 :l 一 、< c, > 。对 于矩阵 u一( 。 和 一( ) U。 l l I c / / c “) , V表示 Ha a r , u。 一“ u 。 d mad积 即( V) ・ 。
( 5)
因为 lgd t 1g 是 严 格 凹 的 , 以 对 每 一 > 0存 在 唯一 的解 ( , , , ) 足 以上 o e z,o 所 f 满 KKT 条 件 , {X , , ): ≤ < 称 为 中心路 径 。当 一 0时得 到原 问题 的最 优解 。 ( , t 0 ≤O } 3 为 简单起 见 , ( ) 写成 以下 形式 : 将 5式
第 4期
黎
博 , :带有 混合 约束 的二 次半 定规 划 的内点算 法 等
(B ) Q P D
m 6 号 x c> 。e+ 。 a n+ 一 < ,x ~c t g x Q d gz
S t A + B t Z~ C— Q( ) 0 .. — X 一
y∈ R t , ∈尺 , ZE S
基 金 项 目 :山 东省 高 等学 校 科 技 计 划 项 目(II 5 JO ) A0
作者简介 : 黎
博 ( 5) 女 , 北 人 , 士 研 究 生 , 究 方 向 为最 优 化 理 论 与方 法 。 1 8 一, 河 9 硕 研
通 讯 作 者 :田 志远 , 岛大 学 数 学 科 学 学 院 教 授 。E maltts @1 6 cm 青 — i tsx .o : 2
n + 6 一 < x , x Qc > ( QDS DP) A + B t z- C- Q ( ) o — X 一
一种低复杂度的LDPC码的分层最小和译码算法
一种低复杂度的LDPC码的分层最小和译码算法摘要本文中,我们提出了一种低复杂度的LDPC码分层译码算法,利用惰性调度简化了变量节点的运算量,降低了译码器的功率消耗,并对LDPC码译码器的部分硬件结构的改进也做了说明。
仿真结果表明,与和积算法、最小和算法相比,改进的算法使得LDPC译码器的运行量减少接近60%,并保持了译码的良好性能。
关键词低密度奇偶校验码;分层修正最小和译码算法;惰性调度低密度奇偶校验(Low-density Parity-check)码[1] 是可以实际应用,性能接近Shannon极限的纠错编码,因而得到广泛关注,对于现在的无线通信有着至关重要的作用。
在LDPC码译码时,由于校验节点与变量节点之间有大量的信息传送,因此降低LDPC码的译码复杂度与译码器的功率消耗,关键在于减少信息传送的运算量。
Gallager在提出LDPC码的同时给出了一种基于概率域的迭代译码算法,这就是目前被普遍采用的置信传播算法(Belief Propagation),又称为和积算法(Sum-Product Algorithm简称SPA)。
借助于和积算法,LDPC码可获得最优的译码性能,但校验节点计算中的双曲余切函数为算法的硬件带来了很大的困难。
分层最小和译码算法用min(x)最小值函数代替了复杂的函数,大大降低了译码算法的复杂度,而译码性能与和积算法非常接近。
一、分层最小和译码算法分层译码的思想是将LDPC码的校验矩阵按水平方向分层,每一层可看作独立的码,各层的交集构成完整的码,假设校验矩阵可分为M层,其中每一层的最大列重为1,译码时顺序处理每一层,所有M层参加完一次译码后完成一次迭代。
记变量节点i的初始对数似然率(LLR)信息为;第n次迭代中,变量节点i传向第l层中检验节点j的LLR信息为;第l层的校验节点j传向变量节点i的LLR信息为;变量节点i 的后验概率信息为;与校验节点j相连的变量节点集合记为N(j)。
- 1、下载文档前请自行甄别文档内容的完整性,平台不提供额外的编辑、内容补充、找答案等附加服务。
- 2、"仅部分预览"的文档,不可在线预览部分如存在完整性等问题,可反馈申请退款(可完整预览的文档不适用该条件!)。
- 3、如文档侵犯您的权益,请联系客服反馈,我们会尽快为您处理(人工客服工作时间:9:00-18:30)。
The Minimum-Norm-Point Algorithm Applied to Submodular Function Minimization andLinear ProgrammingSatoru FUJISHIGE∗,Takumi HAY ASHI†,and Shigueo ISOTANI‡September,2006AbstractIn the present paper we consider applications of the minimum-norm-point algo-rithm to submodular function minimization and linear programming.Although com-binatorial polynomial algorithms for submodular function minimization(SFM)haverecently been obtained and linear programming problems can be solved in polyno-mial time by interior-point algorithms,there still remain(open)problems of reducingthe complexity of the SFM algorithms and of devising a strongly polynomial algo-rithm for linear programming.The present paper has been motivated by an attemptto solve these problems.We show some possible approaches to them by means ofthe minimum-norm-point putational results on submodular functionminimization reveals that our algorithm outperforms the existing polynomial algo-rithms for SFM.We also carry out preliminary experiments on our algorithm for LPby using MATLAB,which shows some interesting behavior of our algorithm. Keywords:The minimum-norm-point algorithm,submodular function minimization, linear programming,zonotopes1.IntroductionPhilip Wolfe[15]presented an algorithm forfinding the minimum-norm point in the con-vex hull of a givenfinite set of points in the n-dimensional Euclidean space R n(von ∗Corresponding author.Research Institute for Mathematical Sciences,Kyoto University,Kyoto606-8502,Japan.E-mail:fujishig@kurims.kyoto-u.ac.jp†Research Institute for Mathematical Sciences,Kyoto University,Kyoto606-8502,Japan.E-mail:thayashi@kurims.kyoto-u.ac.jp‡Faculdade de Ciˆe ncias Econˆo micas e Administrativas de Osasco–FAC-FITO,Osasco,SP,Brazil.E-mail:shigueo isotani@1Hohenbalken[7]also gave essentially the same algorithm for more general objective functions).In the present paper we consider applications of the minimum-norm-point algorithm to submodular function minimization and linear binatorial polynomial algorithms for submodular function minimization(SFM)have been obtained by[10,9,13],and linear programming problems can be solved in polynomial time by interior-point algorithms(see,e.g.,[1]).However,there still remain(open)problems of reducing the complexity of the SFM algorithms and of devising a strongly polynomial algorithm for linear programming.The present paper has been motivated by an attempt to solve these problems.We will show some possible approaches to them by means of the minimum-norm-point algorithm.The minimum-norm-point algorithm keeps a simplex(a set of affinely independent set of points)chosen from the given point set P.Updating such a simplex requires a solution of a linear optimization problem over the convex hullˆP of P,and the algorithm works if the linear optimization can(efficiently)be done over the polytopeˆP.In the original problem setting by Wolfe[15]the polytopeˆP is expressed as the convex hull of the set P of given points.Hence,the linear optimization overˆP can be done trivially by the evaluation of a linear function on given points in P.For a general polytope Q,however, the number of extreme points of Q can be exponentially large with respect to dimension n,so that the minimum-norm-point algorithm cannot be used in the original,trivial way. However,there are interesting classes of polytopes on which linear optimization can ef-ficiently be done,even if the number of extreme points of Q is exponentially large with respect to dimension n.The following is one of such classes of polytopes.It is well known that the greedy algorithm[2]works for base polyhedra associated with submodular functions(see[6]for details about submodular functions and related polyhedra).Hence we can easily make linear optimization over base polyhedra(although the number of extreme points can be n!).This fact leads us to an algorithm for submodular function minimization by means of the minimum-norm-point algorithm,which will be discussed in Section3.Moreover,we can formulate linear programming problems in terms of zonotope(the Minkowski sum of line segments;see,e.g.,[16]),which will be discussed in Section4. Zonotopes are also typical polytopes on which linear optimization can be done in a greedy way.We will show how to adapt the minimum-norm-point algorithm,based on this fact, to solve linear programming problems.The algorithm consists of solving a sequence of minimum-norm-point problems with a multi-dimensional Newton-like algorithm to be given in this paper.We examine the proposed algorithms for submodular function minimization and lin-ear programming by computational experiments in putational results for submodular function minimization will show that the minimum-norm-point algorithm outperforms the existing polynomial-time algorithms in[9,10,13].Compu-tational experiments for linear programming,though very preliminary,are also made by2using MATLAB,which will show some interesting behavior of the algorithm.2.The Minimum-Norm-Point AlgorithmIn this section we describe Wolfe’s algorithm[15]forfinding the minimum-norm point in a polytope for completeness.2.1.Description of the minimum-norm-point algorithmConsider the n-dimensional Euclidean space R n.Suppose that we are given afinite set P of points p i(i∈I)in R n.The problem is tofind the minimum-norm point x∗in the convex hullˆP of points p i(i∈I).Wolfe’s algorithm[15]is given as follows.The Minimum-Norm-Point AlgorithmInput:Afinite set P of points p i(i∈I)in R n.Output:The minimum-norm point x∗in the convex hullˆP of the points p i(i∈I). Step1:Choose any point p in P and put S:={p}andˆx:=p.Step2:Find a pointˆp in P that minimizes the linear function ˆx,p =nk=1ˆx(k)p(k)inp∈P.Put S:=S∪{ˆp}.If ˆx,ˆp = ˆx,ˆx ,then return x∗=ˆx and halt;else go to Step3.Step3:Find the minimum-norm point y in the affine hull of points in S.If y lies in the relative interior of the convex hull of S,then putˆx:=y and go to Step2. Step4:Let z be the point that is the nearest to y among the intersection of the convex hull of S and the line segment[y,ˆx]between y andˆx.Also let S ⊂S be the unique subset of S such that z lies in the relative interior of the convex hull of S .Put S:=S andˆx:=z. Go to Step3.(End)The cycle formed by Step2and Step3is called a major cycle,and the one by Step3 and Step4a minor cycle.Every major cycle increases the size of the simplex S by one, while every minor cycle decreases the size of the simplex S by at least one.A simplex S is called a corral if the minimum-norm point in the affine hull of S lies in the relative interior of the convex hull of S.When we go from Step3to Step2in a major cycle,the current simplex S is a corral.Note that every corral S uniquely determines the minimum-norm pointˆx and that every time we get a new corral,the norm of the newˆx strictly decreases.Also note that at most n−1repetitions of Step3and Step4in a minor cycle3give a corral,so that the Wolfe algorithm described above terminates in afinite number of steps.(It is open to determine whether the Wolfe algorithm runs in polynomial time.) In Step3,for S={p i|i∈I}we have y= i∈Iµi p i with i∈Iµi=1.Note that y lies in the relative interior of the convex hull of S if and only ifµi>0for all i∈I,where recall that S is affinely independent.In Step4,bothˆx and y are expressed asˆx= i∈Iλi p i and y= i∈Iµi p i.Then,the point z is determined in such a way that z=(1−β)ˆx+βy,(1−β)λi+βµi≥0for all i∈I,andβis as large as possible. Remark:When implementing the Wolfe algorithm,we should take care of numerical errors by introducing small tolerance intervals for decisions such as‘α=β?’.Besides these,the algorithm is self-correcting,so that it is stable against numerical errors.2 2.2.Applicability of the algorithmThe Wolfe algorithm requires linear optimization in Step2,which can be done by com-puting ˆx,p for all points p in P.If the number of points in P is exponential in the dimension of the space R n,then it becomes hard to perform the linear optimization in Step2.Now,suppose that the set P is implicitly given as the set of extreme points of a polytope Q in R n.Then the Wolfe algorithm works if linear optimization over Q can efficiently be made.There are classes of polytopes on which linear optimization can efficiently be done.For example,we have(1)base polyhedra,associated with submodular functions,on which the so-called greedyalgorithmfinds optimal(extreme)points,and(2)zonotopes on which every linear optimization can be done in a greedy way,where a zonotope is the Minkowski sum of line segments(or an affine transformation of a unit hypercube).Remark:A pointed polyhedron is called edge-polynomial[5]if the number of edge vec-tors of the polyhedron is polynomial in the dimension of the input data space,where edge vectors are identified up to nonzero multiples.Base polyhedra and zonotopes are typical edge-polynomial polyhedra.The number of edge vectors of base polyhedra is O(n2)with n being the dimension of the space,and that of zonotopes is at most the number of the generators.It should be noted that linear optimization over any edge-polynomial polyhe-dron is easy(solvable in strongly polynomial time)under certain conditions,so that the minimum-norm-point algorithm works for edge-polynomial polyhedra.2 We shall show how the Wolfe algorithm works for base polyhedra(in Section3),and how linear programming problems can be formulated in terms of zonotope and can be solved by the Wolfe algorithm(in Section4).43.Base Polyhedra and Submodular Function Minimiza-tionIn this section we show how the Wolfe algorithm can be used to minimize submodular functions.3.1.Submodular functions and base polyhedraLet E be afinite nonempty set and f be a submodular function on2E,i.e.,f:2E→R satisfiesf(X)+f(Y)≥f(X∪Y)+f(X∩Y)(3.1) for any X,Y⊆E.We suppose that f(∅)=0without loss of generality.We then define polyhedraP(f)={x|x∈R E,∀X∈2E:x(X)≤f(X)},(3.2)B(f)={x|x∈P(f),x(E)=f(E)}.(3.3)Here,P(f)is called the submodular polyhedron and B(f)the base polyhedron,associated with submodular function f on2E.Remark:Since B(f)defined as above is bounded,it is also called a base polytope.Note that P(f)is always unbounded.In the general theory of submodular functions(see[6]) we consider a distributive lattice D⊆2E(a set of subsets of E that is closed with respect to set union∪and intersection∩)and a submodular function f on D.We assume that ∅,E∈D and f(∅)=0.Then B(f)is defined similarly as in(3.2),and is bounded only if D=2V.2 The linear optimization over base polyhedron B(f)can easily be made by the greedy algorithm of Edmonds[2].Here we assume that we are given an oracle for evaluation of the function value f(X)for any X⊆E.The Greedy AlgorithmInput A weight vector w∈R E.Output:An optimal x∗∈B(f)that minimizes the linear objective functione∈E w(e)x(e)in x∈B(f).Step1:Find a linear ordering e1,e2,···,e n of elements of E such thatw(e1)≤w(e2)≤···≤w(e n).(3.4) Step2:Computex∗(e i)=f({e1,e2,···,e i})−f({e1,e2,···,e i−1})(i=1,2,···,n).(3.5)5Return x∗.(End)We also have the following theorem that characterizes the minimizers of a submodular function f:2E→R with f(∅)=0.Theorem3.1([2]):We havemin{f(X)|X⊆E}=max{x−(E)|x∈B(f)},(3.6) where x−(e)=min{x(e),0}for e∈E.Moreover,if f is integer-valued,then the maximum in the right-hand side is attained by an integral base x∈B(f).2 Note that for any X⊆E and x∈B(f)we have f(X)≥x−(E).The gap f(X)−x−(E)evaluates an upper bound for to what extent X is close to a minimizer of f.In particular,if f is integer-valued,the gap f(X)−x−(E)being less than one implies that X is a minimizer of f.3.2.The minimum-norm point in a base polyhedron and submodularfunction minimizationWe have the following theorem.Theorem3.2([4],[6,Sec.7.1.(a)]):Let x∗be the minimum-norm point in the base poly-hedron B(f)given by(3.3).DefineA−={e|e∈E,x∗(e)<0},(3.7)A+={e|e∈E,x∗(e)≤0}.(3.8) Then,A−is the unique minimal minimizer of f,and A+the unique maximal minimizer of f.2Because of this theorem we can solve the submodular function minimization problem byfinding the minimum-norm point in the base polyhedron B(f).The minimum-norm-point algorithm described in Section2can directly be employed to solve the submodular function minimization problem by means of the greedy algorithm of pu-tational results are given in Section3.3.6putational resultsCombinatorial polynomial algorithms for submodular function minimization(SFM)were devised independently by Iwata,Fleischer,and Fujishige[10],and Schrijver[13].Also Fleischer and Iwata[3]proposed a polynomial preflow-push algorithm,which has the same complexity as Schrijver’s([14]).Currently the fastest SFM algorithm has been obtained by Iwata[9].See a nice survey[11]for more details about the developments in SFM algorithms(also see[6,Chapter VI]).The following computational results on SFM algorithms are based on a report of[8].putational SetupWe used a Dynabook G6/X18PDE with an Intel Pentium4,CPU1.80GHz,768MB of memory and running Linux RedHat version2.4.18.All programs are written in C lan-guage and compiled with gcc using the-O4optimization option.We denote by FW the proposed SFM algorithm by means of the minimum-norm-point algorithm[4].The Iwata-Fleischer-Fujishige algorithm[10]is denoted by SFM3and Schrijver’s algorithm[13]by LEX2.We also have Fleischer and Iwata’s algorithm[3], denoted by PR.Moreover,HYBRID is an algorithm,proposed by Iwata[9],that com-bines techniques involved in SFM3and PR.The FW program was,first,written in FORTRAN language by Masahiro Nakayama (in his graduation thesis at the University of Tsukuba in February,1985).We rewrote the program in C language and improved some part of it.The other programs were written by Satoru Iwata.We employed Quick Sort for the sorting algorithm required in the greedy algorithm.We tested the algorithms using two kinds of submodular functions.One is proposed by Satoru Iwata and the other is a class of cut functions.3.3.2.Iwata’s Test FunctionThe submodular function suggested by Satoru Iwata is(5j−2n)(X⊆V)f(X)=|X||V\X|−j∈Xwhere V={1,2,···,n}.The results on this function are shown in Table1and Table2.This class of test problems is very special for FW.Except for FW,HYBRID outper-formed the others.7Table1:Running times for Iwata’s functionRunning time(sec)n FW HYBRID SFM3LEX2PR1000.000.41 1.002644.52277.362000.00 4.9218.693000.0021.77115.444000.0067.12369.135000.00166.73894.336000.01325.262820.837000.01568.54Table2:Number of generated extreme bases for Iwata’s functionNumber of basesn FW HYBRID SFM3LEX2PR10021163766337348373324200237324618300267107309400210803991450021670118835600222011338497002286993.3.3.Cut FunctionsIn the case of cut functions,we need to generate networks.We used the generator GEN-RMF available from DIMACS Challenge[17].Each generated network has b grid-likeframes of size(a×a).The number of vertices is a2b and that of arcs5a2b−4ab−a2.All vertices in each frame are connected to its grid neighbors and each vertex is connected by an arc to a vertex randomly chosen from the next frame.All the running times reported here are in seconds,and we only report the user CPU time.We generatedfive instances for each problem family of specified size,using dif-ferent random seeds.Each number shown in the tables is the averaged time overfive runs.We used GENRMF to produce two kinds of networks as follows:•Genrmf-long.The number of vertices of a generated graph is n=2x.The parame-ters are a=2x/4and b=2x/2.•Genrmf-wide.The number of vertices of a generated network is n=2x.The parameters are a=22x/5and b=2x/5.8We used the submodular function minimization algorithms to compute minimum cuts. The running times for the computation are shown in Table3and Table4,and numbers of generated extreme bases in Table5and Table6.Figure1and Figure2,respectively, represent Table3and Table4.For the genrmf-long networks LEX2and PR were faster than HYBRID.However,for the genrmf-wide networks LEX2was slower than HYBRID.In both cases FW outper-formed the others.Figures3,4,5,and6show sample behaviors of iteration vs.duality gap for Genrmf-Long with n=63and m=222.Here,one iteration means a generation of a new extreme base.Table3:Results on Genrmf-LongRunning time(sec)n m FW HYBRID SFM3LEX2PR632220.040 4.02410.952 1.428 1.2421264530.36870.826280.52753.36023.2862561008 3.7927376.4753209.7003507.494525218046.05210084332366.21Table4:Results on Genrmf-WideRunning time(sec)n m FW HYBRID SFM3LEX2PR752900.056 3.34020.588 4.195 3.4861476020.41055.996749.135141.497572.3363241395 4.5964265.1489607.3602433.578576254427.17010244608172.529Table 5:Results on Genrmf-Long Number of bases n mFW HYBRID SFM3LEX2PR 63222982302928288526191812645322111232814067822805732256100851569095087571460552521801353100843322980Table 6:Results on Genrmf-Wide Number of bases n mFW HYBRID SFM3LEX2PR 75290100135641850775635191476021968024066346387876943241395429661802140662055357625447661024460814860.010.11101001000R u n n i n g t i m e (s e c )Number of vertices (power of 2)Figure 1:Number of vertices vs.time on Genrmf-Long100.010.11101001000R u n n i n g t i m e (s e c )Number of vertices (power of 2)Figure 2:Number of vertices vs.time on Genrmf-Wide 05000001e+061.5e+062e+06102030405060708090Iteration FWFigure 3:Iteration vs.duality gap by FW1105000001e+061.5e+062e+06020004000600080001000012000Iteration HYBRIDFigure 4:Iteration vs.duality gap by HYBRID 05000001e+061.5e+062e+0605000100001500020000Iteration SFM3Figure 5:Iteration vs.duality gap by SFM31205000001e+061.5e+062e+0650100150200250300350Iteration LEX2Figure 6:Iteration vs.duality gap by LEX24.Zonotopes and Linear ProgrammingIn this section,we consider the linear programming (LP)problem and show how to adapt the Wolfe algorithm to solve LP.4.1.Reformulation of linear programming problemsWe consider the linear programming problem given in the following form:(LP)Maximize cx =nj =1c (j )x (j )subject to Ax =b,(4.1)l ≤x ≤u,where A is an m ×n matrix,b an m -dimensional column vector,l and u n -dimensional col-umn vectors,x an n -dimensional column variable vector (in R n ),and c an n -dimensional row vector (in (R n )∗,the dual space of R n ).Remark :The ordinary standard form of LP is given by (4.1)with l =0and u =(+∞,+∞,···,+∞)T ,where T denotes the matrix transpose.It should be noted that in our LP form (4.1)vectors l and u are finite-valued,so that our model is slightly restric-tive.However,it suffices to consider such a bounded feasible region practically as well as theoretically (see [12,Theorem 2.2]).213Define an (m +1)×n matrix¯A=A c .(4.2)Also define the polytope Z ={z |z =¯Ax,l ≤x ≤u }.(4.3)Note that polytope Z is,what is called,a zonotope (with possible translation).The LP problem (4.1)can be reformulated as follows.(LP) Maximize γsubject tob γ ∈Z ,(4.4)where γis a scalar variable in R .Remark :DefineL = b γ γ∈R .(4.5)Note that L is a line parallel to the last coordinate space and that Problem (LP) is to find the point of maximum last coordinate in the intersection of line L and zonotope Z .2For any ¯c ∈(R n +1)∗an optimal solution ˆz of the problem of minimizing¯c z =n +1i =1¯c (i )z (i )(4.6)over zonotope Z in (4.3)can easily be computed as ˆz =¯Ax with d =¯c ¯Aand x (j )=u (j )if d (j )<0l (j )otherwise (j =1,2,···,n ).(4.7)Therefore,the minimum-norm-point algorithm works for Z .4.2.The LP-Newton algorithmIn order to solve Problem (LP) (or Problem (LP))we introduce a new optimization method which we call the LP-Newton algorithm .It consists of repeated minimum-norm-point algorithms as follows.The LP-Newton Algorithm LPNInput :Data A,b,c,l,u for Problem (LP).14Output:An optimal solution x∗of Problem(LP)or decision that Problem(LP)is infea-sible.Step1:Computex(j)=u(j)if c(j)>0l(j)otherwise(4.8)for each j=1,2,···,n.Putγ:=cx.Step2:Put¯b:=(b T,γ)T(where T denotes the matrix transpose).By using the minimum-norm-point algorithmfind the point z in Z that is the nearest to¯b,where z is expressed as a convex combination of affinely independent extreme points y k(k∈K)of Z,i.e., z= k∈Kλk y k(with k∈Kλk=1andλk>0(k∈K))and each y k is given by y k=¯Ax k with l≤x k≤u(k∈K).Let z=(˜z T,ζ)T.If z=¯b,then put x∗= k∈Kλk x k,return x∗,and halt;else ifζ≥γ,then return‘Problem(LP)is infeasible’and halt.Step3:Computeγ:={(˜z−b)T b−(z−¯b)T z)}/(γ−ζ).Go to Step2.(End)Note that the initial x computed in Step1corresponds to¯c=(0,···,0,1)in(4.6).If the algorithm does not halt at the end of an execution of Step2,we have a hyper-plane H z expressed by(z−¯b)T(y−z)=0(4.9) in a variable vector y in R n+1.The newγcomputed in Step3gives the point¯b=(b T,γ)T that is the intersection point of L and H z(see Figure7).In Figure7,z0,z1,z2,z3represent the sequence of z s computed in Step2,and¯b0,¯b1,¯b2=z3that of¯b s.Computational results about the behavior of the LP-Newton algorithm for linear pro-gramming are given in Section4.3.Remark:Linear programming problems can be reduced tofinding feasible solutions in systems of linear inequalities,and the latter problems can further be reduced to minimum-norm-point problems for zonotopes.Here we are,however,interested in solving optimiza-tion problems directly.2 Remark:We can consider a metric other than the Euclidean one,such as(¯A¯A T)−1when ¯A is of row-full rank.It is left for future work to choose an appropriate metric in the space of the zonotope.215Figure7:An illustration of the LP-Newton algorithm(z3=¯b2).putational resultsConsider Problem(LP)Maximize cx=nj=1c(j)x(j)subject to Ax=b,l≤x≤u,in(4.1).We generate at random each component a(i,j)of A uniformly from[0,1],b(i)ofb from[10,11],and c(j)ofc from[−12,12].We set l=0and u(j)=10(j=1,2,···,n).We programmed our LP-Newton algorithm(LPN)by MATLAB.We carried out com-putational experiments of LPN for LP on Sun Fire V440with SPARC/Solaris10(3/05), CPU1.6GHz×4,8GB of memory,using MATLAB version7.1.0.183(R14)Service Pack-age3.Table7shows10-run averages of the running time of LPN,the number of the Newton steps(Step2)of LPN,and the number of generated extreme points of zonotope Z.Note that the major part of Step2of LPN is to carry out the minimum-norm-point procedure of Wolfe and that the number of generated extreme points of Z is equal to the total number16of executed major cycles of the minimum-norm-point procedure.(The last row of Table7 gives the10-run averages of the running time of the LP solver linprog available within the MATLAB package,for reference.)We observe that the running time of LPN depends on m but seems indifferent to values of n.The number of the Newton steps(Step2)of LPN is relatively small and increases very slowly with respect to m.Table8shows sample behaviors of objective function values computed by LPN.The objective function values converge to optimal values very quickly,as expected.Table7:Results for Algorithm LPNAveraged running time,number of steps,and number of generated extreme points m101010505050100100100n200350500200350500200350500 time(sec)0.0470.0470.051 1.52 1.36 2.2326.1017.8218.97 #Newton steps 3.80 3.90 3.90 5.70 5.20 5.009.00 6.90 6.10 #extreme points8.18.88.765.252.448.6418.7274.4305.7 linprog(sec)0.2250.1370.1860.300.480.740.79 1.38 2.03Table8:Sample behaviors of LPNIteration vs.(objective function value−optimal value)iteration1234567 (m,n)=(10,500)618.89 6.4580.00240.0———(m,n)=(50,500)585.67 1.9480.02060.000040.0——(m,n)=(100,500)595.95 3.4540.17060.000550.000020.0—5.Concluding RemarksThe computational results on submodular function minimization(SFM)have shown that the minimum-norm-point SFM algorithm FW runs very fast,and suggest that FW is strongly polynomial.It is,however,open to determine the complexity of FW for SFM.We have also proposed a new algorithm LPN for LP by means of the minimum-norm-point algorithm and the LP-Newton algorithm.The present results indicate that the number of the Newton steps is small and that the computation time is seemingly indiffer-ent to the dimension n of the original variable vector space.The running time of LPN implemented by MATLAB is,however,far from competitive with existing commercial codes such as CPLEX.Our computational experiments are only preliminary and require much more further to examine its behavior when implemented,say,by C or C++,which will be left for future work.17AcknowledgementsWe are grateful to Satoru Iwata for providing us with his programs of the SFM algorithms, and to Hiroshi Hirai for his help in carrying out computational experiments for LP.The present research was supported partly by a Grant-in-Aid from the Ministry of Education, Culture,Sports,Science and Technology of Japan and by Japan International Cooperation Agency.References[1]A.Ben-Tal and A.Nemirovski:Lectures on Modern Convex Optimization—Analysis,Algorithms,and Engineering Applications(MPS/SIAM Series on Opti-mization)(SIAM,2001).[2]J.Edmonds:Submodular functions,matroids,and certain polyhedra.Proceedings ofthe Calgary International Conference on Combinatorial Structures and Their Appli-cations(R.Guy,H.Hanani,N.Sauer and J.Sch¨o nheim,eds.,Gordon and Breach, New York,1970),pp.69–87;also in:Combinatorial Optimization—Eureka,You Shrink!(M.J¨u nger,G.Reinelt,and G.Rinaldi,eds.,Lecture Notes in Computer Science2570,Springer,Berlin,2003),pp.11–26.[3]L.Fleischer and S.Iwata:A push-relabel framework for submodular function mini-mization and applications to parametric optimization.Discrete Applied Mathematics 131(2003)311–322.[4]S.Fujishige:Submodular systems and related topics.Mathematical ProgrammingStudy22(1984)113–131.[5]S.Fujishige:Submodularity and polyhedra.4th Japanese-Hungarian Symposium onDiscrete Mathematics and Its Applications(Budapest,June3–6,2005).[6]S.Fujishige:Submodular Functions and Optimization,(Second Edition)(Annals ofDiscrete Mathematics58)(Elsevier,Amsterdam,2005).[7]B.von Hohenbalken:Afinite algorithm to maximize certain pseudoconcave func-tions on polytopes.Mathematical Programming8(1975)189–206.[8]S.Isotani and S.Fujishige:Submodular function minimization:Computational ex-periments.Unpublished manuscript,2003.[9]S.Iwata:A faster scaling algorithm for minimizing submodular functions.SIAMJournal on Computing32(2003)833–840.18[10]S.Iwata,L.Fleischer,and S.Fujishige:A combinatorial strongly polynomial algo-rithm for minimizing submodular functions.Journal of ACM48(2001)761–777.[11]S.T.McCormick:Submodular function minimization.In:Discrete Optimization(Handbooks in Operations Research and Management Science12)(K.Aardal,G.L.Nemhauser,and R.Weismantel,eds.,Elsevier,Amsterdam,2005),Chapter7,pp.321–391.[12]C.H.Papadimitriou and K.Steiglitz:Combinatorial Optimization—Algorithms andComplexity(Prentice-Hall,New Jersey,1982).[13]A.Schrijver:A combinatorial algorithm minimizing submodular functions instrongly polynomial time.Journal of Combinatorial Theory,Ser.B80(2000)346–355.[14]J.Vygen:A note on Schrijver’s submodular function minimization algorithm.Jour-nal of Combinatorial Theory B88(2003)399–402.[15]P.Wolfe:Finding the nearest point in a polytope.Mathematical Programming11(1976)128–149.[16]G.M.Ziegler:Lectures on Polytopes(Graduate Texts in Mathematics152)(Springer,Berlin,1995).[17]The First DIMACS international algorithm implementation challenge:The coreexperiments,1990.Available at ftp:///pub/netflow/general-info/core.tex.19。