第5章 离散时间系统最优控制
最优控制
最优控制综述 摘要：最优化方法是一种求极值的方法，即在一组约束为等式或不等式的条件下，使系统的目标函数达到极值，即最大值或最小值。从经济意义上说，是在一定的人力、物力和财力资源条件下，使经济效果达到最大（如产值、利润），或者在完成规定的生产或经济任务下，使投入的人力、物力和财力等资源为最少。而最优控制通常针对控制系统而言，目的在于使一个机组、一台设备或一个生产过程实现局部最优。本文重点阐述了最优系统常用的变分法、极小值原理和动态规划三种方法的基本理论及其在典型系统设计中的应用。 关键词：变分法、极小值原理、动态规划 1 引言 最优控制是分析控制系统常用的方法，是现代控制理论的核心之一。它尤其与航空航天的制导、导航和控制技术密不可分。最优控制理论所研究的问题可以概括为：对一个受控的动力学系统或运动过程，从一类允许的控制方案中找出一个最优的控制方案，使系统的运动在由某个初始状态转移到指定的目标状态的同时，其性能指标最优。 这类问题广泛存在于技术领域或社会问题中。例如，确定一个最优控制方式使空间飞行器由一个轨道转换到另一轨道过程中燃料消耗最少，选择一个温度的调节规律和相应的原料配比使化工反应过程的产量最多，制定一项最合理的人口政策使人口发展过程中的老化指数、抚养指数和劳动力指数为最优等，都是一些经典的最优控制问题。 最优控制问题是要在满足约束条件下寻求最优控制函数，使目标泛函取极值。求解动态最优化问题的方法主要有古典变分法，极小值原理及动态规划法等。 2 研究最优控制的前提条件 2.1状态方程 对连续时间系统: x t=f x t,u t,t 对离散时间系统：x(k+1)=f x k,u k,k k=0,1,……，（N-1）
离散时间系统最优控制离散时间系统最优控制
第五章离散时间系统最优控制
?前面所讨论的都是关于连续时间系统的最优控制问题。?现实世界中，很多实际系统本质上是时间离散的。?即使是系统是时间连续的，因为计算机是基于时间和数值上都离散的数字技术的，实行计算机控制时必须将时间离散化后作为离散系统处理。 引言 ?因此，有必要讨论离散时间系统的最优控制问题。 ?离散时间系统仍然属于连续变量动态系统(CVDS)范畴。注意与离散事件动态系统(DEDS)的区别。 ? CVDS 与DEDS 是自动化领域的两大研究范畴，考虑不同的自动化问题。
5.1 离散时间系统最优控制问题的提法 (1) 离散系统最优控制举例——多级萃取过程最优控制 ?萃取是指可被溶解的物质在两种互不相溶的溶剂之间的转移，一般用于将要提取的物质从不易分离的溶剂中转移到容易分离的溶剂中。 ?多级萃取是化工生产中提取某种价值高、含量低的物质的常用生产工艺。 萃取器萃取器萃取器萃取器V u (0)u (1)u (k -1)u (N －1) V V V V V V 含物质A 的混合物以流量V 进入萃取器1，混合物中A 浓度x (0); 萃取剂以流量u (0)通过萃取器1，单位体积萃取剂带走A 的量为z (0); 一般萃取过程的萃取物含量均较低，可认为通过萃取器1后混合物流量仍为V ; 流出萃取器1的混合物中A 物质的浓度为x (1)。以此类推至萃取器N 。 1 2 k N x (0) z (0)z (1) z (k-1) z (N -1) x (1) x (2) x (k -1) x (k ) x (N ) x (N -1) 多级萃取过程
(2) 离散系统最优控制问题的提法 给定离散系统状态方程(5-1-6)和初始状态 (5-1-7) 其中分别为状态向量和控制向量，f 为连续可微的n 维 函数向量。考虑性能指标 1 ,,1,0],),(),([)1( N k k k u k x f k x 0 )0(x x m n R k u R k x )(,)( 1 N 其中Φ、L 连续可微。 ?离散系统的最优控制问题就是确定最优控制序列u *(0)，u *(1)，…，u *(N -1)，使性能指标J 达到极小(或极大)值。 ? 将最优控制序列u *(0)，u *(1)，…，u *(N -1)依次代入状态方程，并利用初始条件，可以解出最优状态序列x *(1)，x *(2)，…，x *(N )，也称为最优轨线。 (5-1-8) ] ),(),([]),([k k k u k x L N N x J
§7.4动态规划与离散系统最优控制
§ 7.4 动态规划与离散系统最优控制 1. 动态规划基本原理 最优性原则应有如此性质: 即无论(整个过程的)初始状态和初始决策如何，其余(后段)各决策对于由第一个决策(后)所形成的状态作为(后段)初始状态来说，必须也是一个最优策略。 A B C D E 最优性原则 图7.5
用式表示 1() ()min{(,())(())},1,2,,n n n n n u x J x R x u x J u x n N -=+= 阶段变量n (分析次序) 状态变量x 决策变量()n u x 决策组11{,, ,}n n u u u - 损失(效益)函数:(,)n R x u 对x 用决策n u 所付代价(效益) 后部最优策略函数()n J x 由x 至终最小损失(最大效益)
A 到D 的最短路线 解 3阶段的决策过程， 在CD 段(首), (分析)阶段变量1n =; 7.6 图A 2C 1 B D 2 B 3 B 1 C 3 C 4 5 55 6 3 3) b (A 2 C 1B D 2 B 3 B 1 C 3 C 4 4 5 55 55 66677 7 3 3 (a) 3 =n 1 =n 2 =n
111111*********()(,)3,();()(,)5,();()(,)3,(). J C R C D u C D J C R C D u C D J C R C D u C D ========= 在BC 段(首), (分析)阶段变量2n =； 21111,2,3 ()min{(,)()} min{73,65,53}8i i i J B R B C J C ==+=+++=，213()u B C =； 22211,2,3 ()min{(,)()} min{63,55,73}9i i i J B R B C J C ==+=+++=，221()u B C =； 23311,2,3 ()min{(,)()} min{53,65,73}8 i i i J B R B C J C ==+=+++=，231()u B C =；
最优控制习题答案
最优控制习题答案 1.设系统方程及初始条件为? ??=+-=)()() (2)()(1211t x t x t u t x t x &&，???==0)0(1)(21x t x 。约束 5.1)(≤t u 。若系统终态)(f t x 自由，利用连续系统极大值原理求)(*t u 性能指标，)3(2x J =取最小值。 解： 2.设一阶离散时间系统为)()()1(k u k x k x +=+，初值2)0(=x ，性 能指标为∑=+=20 2 2 )(21)2(k k u x J ，试用离散系统最小值原理求解最优控 制序列：)2(),1(),0(u u u ，使J 取极小值。 解： 3.软着落、空对空导弹的拦截问题、防空拦截问题。 解答： 4.设离散系统状态方程为)(2.00)(101.01)1(k u k x k x ?? ? ???+??????=+，已知边界条件?? ? ???=01)0(x ，??????=00)1(x 。试用离散系统最小值原理求最优控制序 列，使性能指标∑==1 02 )(03.0k k u J 取极小值，并求出最优的曲线序列。 解：属于控制无约束，N 不变，终端固定的离散最优控制问题，构造离 散 哈 密 尔 顿 函 数 )](2.0)()[1()](1.0)()[1()(03.0)(222112k u k x k k x k x k k u k H ++++++=λλ 其中)1(),1(21++k k λλ为给定拉个朗日乘子序列，由伴随方程：
)1()()(111+=??= k k x H k λλ，)1()1(1.0) ()(2122+++=??=k k k x H k λλλ得出 ?? ?+==+==) 2()2(1.0)1(),2()1() 1()1(1.0)0(),1()0(2121121211λλλλλλλλλλ， 由 极 值 条 件 ??? ????>=??=++=??0 06.0)(0)1(2.0)(06.0) (22 2k u H k k u k u H λ极小)1(310)(2+-=k k u λ可使min )(=k H ，令k=0和k=1的?? ??? -=-=) 2(310 )1(*)1(310)0(*22λλu u ，)(k u 带入状态方程并令k=0和1得到： 5.求 泛 函 dt x x x x J ?++=1 02 221211],[&&满足边界条件 π===-=)3(,0)0(,0)3(,3)0(2211x x x x 和约束条件36221=+t x 的 极值曲线。 解：应用拉格朗日乘子法，新目标函数为： dt t x t x x J )36)((1[2 21 1 022211-++++=?λ&&，令哈密尔顿函数为： )36(12 212221-++++=t x x x H λ&&，可以得到无约束条件新的泛函1 J 的欧拉方程为0)1(2)(22 211 1111=++-=??-??x x x dt d x x H dt d x H &&&&λ (1)
基于自适应动态规划的一类带有时滞的离散时间非线性系统的最优控制
Vol.36,No.1ACTA AUTOMATICA SINICA January,2010 An Optimal Control Scheme for a Class of Discrete-time Nonlinear Systems with Time Delays Using Adaptive Dynamic Programming WEI Qing-Lai1ZHANG Hua-Guang2LIU De-Rong1ZHAO Yan3 Abstract In this paper,an optimal control scheme for a class of nonlinear systems with time delays in both state and control variables with respect to a quadratic performance index function is proposed using a new iterative adaptive dynamic programming (ADP)algorithm.By introducing a delay matrix function,the explicit expression of the optimal control is obtained using the dynamic programming theory and the optimal control can iteratively be obtained using the adaptive critic technique.Convergence analysis is presented to prove that the performance index function can reach the optimum by the proposed method.Neural networks are used to approximate the performance index function,compute the optimal control policy,solve delay matrix function,and model the nonlinear system,respectively,for facilitating the implementation of the iterative ADP algorithm.Two examples are given to demonstrate the validity of the proposed optimal control scheme. Key words Adaptive dynamic programming(ADP),approximate dynamic programming,time delay,optimal control,nonlinear system,neural networks DOI10.3724/SP.J.1004.2010.00121 The optimal control problem of nonlinear systems has always been a key focus in the control?eld in the last several decades.Coupled with this is the fact that noth-ing can happen instantaneously,as is so often presumed in many mathematical models.So strictly speaking,time delays exist in the most practical control systems.Time delays may result in degradation in the control e?ciency even instability of the control systems.So there have been many studies on the control systems with time delay in various research?elds,such as electrical,chemical engineer-ing,and networked control[1?2].The optimal control prob-lem for the time-delay systems always attracts considerable attention of the researchers and many results have been obtained[3?5].In general,the optimal control for the time-delay systems is an in?nite-dimensional control problem[3], which is very di?cult to solve.So many analysis and appli-cations are limited to a very simple case:the linear systems with only state delays[6].For nonlinear case with state de-lays,the traditional method is to adopt fuzzy method and robust method,which transforms the nonlinear time-delay systems to linear systems[7].For systems with time delays both in states and controls,it is still an open problem[4?5]. The main di?culty lies in the formulation of the optimal controller which must use the information of the delayed control term so as to obtain an e?cient control.This makes the analysis of the system much more di?cult,and there is no method strictly facing this problem even in the linear cases.This motivates our research. Adaptive dynamic programming(ADP)is a powerful tool in solving optimal control problems[8?9]and has at-tached considerable attention from many researchers in re-cent years,such as[10?16].However,most of the results focus on the optimal control problems without delays.To Manuscript received September5,2008;accepted March3,2009 Supported by National High Technology Research and Development Program of China(863Program)(2006AA04Z183),National Nat-ural Science Foundation of China(60621001,60534010,60572070, 60774048,60728307),and the Program for Changjiang Scholars and Innovative Research Groups of China(60728307,4031002) 1.Key Laboratory of Complex Systems and Intelligence Sci-ence,Institute of Automation,Chinese Academy of Sciences,Beijing 100190,P.R.China 2.School of Information Science and Engi-neering,Northeastern University,Shenyang110004,P.R.China 3. Department of Automatic Control Engineering,Shenyang Institute of Engineering,Shenyang110136,P.R.China the best of our knowledge,there are no results discussing how to use ADP to solve the time-delay optimal control problems.In this paper,the time-delay optimal control problem is solved by the iterative ADP algorithm for the ?rst time.By introducing a delay matrix function,the explicit expression of the optimal control function is ob-tained.The optimal control can iteratively be obtained us-ing the proposed iterative ADP algorithm which avoids the in?nite-dimensional computation.Also,it is proved that the performance index function converges to the optimum using the proposed iterative ADP algorithm. This paper is organized as follows.Section1presents the preliminaries.In Section2,the time-delay optimal control scheme is proposed based on iterative ADP algorithm.In Section3,the neural network implementation for the con-trol scheme is discussed.In Section4,two examples are given to demonstrate the e?ectiveness of the proposed con-trol scheme.The conclusion is drawn in Section5. 1Preliminaries Basically,we consider the following discrete-time a?ne nonlinear system with time delays in state and control vari-ables as follows: x(k+1)=f(x(k),x(k?σ))+g0(x(k),x(k?σ))u(k)+ g1(x(k),x(k?σ))u(k?τ)(1) with the initial condition given by x(s)=φ(s),s=?σ,?σ+1,···,0,where x(k)∈R n is the state vector, f:R n×R n→R n and g0,g1:R n×R n→R n×m are dif-ferentiable functions and the control u(k)∈R m.The state and control delaysσandτare both nonnegative integral numbers.Assume that f(x(k),x(k?σ))+g0(x(k),x(k?σ))u(k)+g1(x(k),x(k?σ))u(k?τ)is Lipschitz continuous on a set?in R n containing the origin,and that system(1) is controllable in the sense that there exists a bounded con-trol on?that asymptotically stabilizes the system.In this paper,how to design an optimal state feedback controller for this class of delayed discrete-time systems is mainly dis-cussed.Therefore,it is desired to?nd the optimal control u(x)satisfying u(x(k))=u(k)to minimize the generalized performance functional as follows: