Convergence analysis of the exterior-point method I. Griva, R. Polyak AMS Meeting 2007, Oxford, March 16 * x Outline 1. Overview. Equality and inequality constraints. Conceptual difficulties. 2. Exterior point method (EPM). 3. Local and global convergence analysis of EPM. Numerical results. 4. Conclusion. Constrained optimization problem EIXxxf f n ∩=∈ ℜ→ℜ ),(min : 1 X Equality constraints { } pixgxE i n , ,1,0)(: ==ℜ∈= Inequality constraints { } mixcxI i n , ,1,0)(: =≥ℜ∈= abledifferentily continuous twiceare ,, ii gcf Equality constraints ( ) )(), ,()(,0)( s.t.),(min 1 xgxgxgxgxf p == Lagrangian ∑ = −= p i ii xgyxfyxL 1 )()(),( Lagrange multipliers ), ,( 1 p yyy = Optimality conditions         = =∇−∇ ⇔ =∇ =∇ 0)( 0)()( 0),( 0),( xg yxgxf yxL yxL T y x )( ofJacobian theis)( xgxg∇ Optimality conditions         = =∇−∇ ⇔ =∇ =∇ 0)( 0)()( 0),( 0),( xg yxgxf yxL yxL T x λ )( ofJacobian theis)( xgxg∇ Newton’s method – quadratic convergence rate       − ∇+∇− =       ∆ ∆       ∇ ∇−∇ )( )()( 0)( )(),( 2 xg yxgxf y x xg xgyxL TT x xxx ∆+=: ˆ yyy ∆+=: ˆ ),(solution enough to close is ),(ion approximat If ** yxyx 2 ** 2 ** ˆ , ˆ yyyyxxxx −≤−−≤− αα Inequality constraints ( ) )(), ,()(,0)( s.t.),(min 1 xcxcxcxcxf m =≥ )( 1 xc )( 3 xc )( 2 xc )( 4 xc 3 )( β =xf * 2 )( ββ ==xf 1 )( β =xf 321 βββ >> { } 0)(: :set Active ** == xciI i { } 3,2 * =I { } 0)(: :set Inactive *0 >= xciI i { } 4,1 0 =I * x Inequality constraints * ,0)( s.t.),(min Iixcxf i ∈= ( ) )(), ,()(,0)( s.t.),(min 1 xcxcxcxcxf m =≥ ⇔ Choosing the active set is a combinatorial problem!!! Optimality conditions    = =∇−∇ 0)( 0)()( xY c yxcxf T )( ofJacobian theis)( xcxc∇ Newton’s method       − ∇+∇− =       ∆ ∆       ∇ ∇−∇ )( )()( )()( )(),( 2 xYc yxcxf y x xCxcY xcyxL TT x )( i yd iagY = ( ) )()( xcdiagxC i = 0,0)( ≥≥ yxc ?0,0)( :itynonnegativ eaccount th into take toHow ≥≥ yxc xxcYxCyy ∆∇−−=∆ − )()( solution near theaccuracy of loss Possible 1 Challenges Complementarity!!! Interior-point method    = =∇−∇ exYc yxcxf T µ )( 0)()( )( ofJacobian theis)( xcxc∇ Newton’s method       − ∇+∇− =       ∆ ∆       ∇ ∇−∇ )( )()( )()( )(),( 2 xYce yxcxf y x xCxcY xcyxL TT x µ )( i yd iagY = ( ) )()( xcdiagxC i = Complementarity relaxation xxcYxCexCyy ∆∇−+−=∆ −− )()()( solution near theaccuracy of loss Possible 11 µ Questions 1. How to come up with the primal-dual system that does not have any complementarity related to inequality constraints? 2. How to relax the requirements of keeping the trajectory in the interior? 3. What would be theoretical convergence properties of such a method? 4. What would be its practical efficiency?