Đang tải... (xem toàn văn)
Tài liệu về optimization
Convex OptimizationNguyen Thi Thu VanUniversity of Science2008-2009tvnguyen (University of Science) Convex Optimization 1 / 108 References1. Hiriart–Urruty, J.B. and Lemar´echal, C. : Convex Analysis andMinimization Algorithms, Volumes I and II, Springer, Berlin (1993)2. Rockafellar, R. T. : Convex Analysis, Princeton University Press,Princeton, New Jersey (1970)3. Strodiot, J.J. : An Introduction to Nonsmooth Optimization, Universityof Namur, Belgium (2005)tvnguyen (University of Science) Convex Optimization 2 / 108 OutlineChapter 1. Convex sets and convex functions taking the infinity valueChapter 2. Topological properties for sets and functionsChapter 3. Duality for sets and functionsChapter 4. Subdifferential calculus for convex functionsChapter 5. Duality in convex optimizationtvnguyen (University of Science) Convex Optimization 3 / 108 Chapter 1 Convex sets and convex functions taking the infinity valueChapter 1.Convex sets and convex functions taking the infinity valuetvnguyen (University of Science) Convex Optimization 4 / 108 Chapter 1 Convex sets and convex functions taking the infinity valueConvex setDefinition. A subset C of IRnis convex if∀x, y ∈ C ∀t ∈ [0, 1] tx + (1 − t)y ∈ CProposition. If C is convex, then its interior int C and its closureCare convexConvexity is preserved by the following operations :Let I be an arbitrary set. If Ci⊆ IRn, i ∈ I, are convex, thenC = ∩i∈ICiis convexLet C and D be two convex sets in IRnand let a and b be two realnumbers. Then the following set is convex :aC + bD := {ac + bd | c ∈ C , d ∈ D}tvnguyen (University of Science) Convex Optimization 5 / 108 Chapter 1 Convex sets and convex functions taking the infinity valueIllustrationY X X Y convex non convextvnguyen (University of Science) Convex Optimization 6 / 108 Chapter 1 Convex sets and convex functions taking the infinity valueExamples of convex setsThe following are some examples of convex sets :(1) Hyperplane : S = {x|pTx = α}, where p is a nonzero vector in IRn,called the normal to the hyperplane, and α is a scalar.(2) Half-space : S = {x|pTx ≤ α}, where p is a nonzero vector in IRn,and α is a scalar.(3) Open half-space : S = {x|pTx < α}, where p is a nonzero vector inIRnand α is a scalar.(4) Polyhedral set : S = {x|Ax ≤ b}, where A is an m × n matrix, and bis an m vector. (Here the inequality should be interpretedelementwise.)tvnguyen (University of Science) Convex Optimization 7 / 108 Chapter 1 Convex sets and convex functions taking the infinity valueExamples of convex sets(5) Polyhedral cone : S = {x|Ax ≤ 0}, where A is an m × n matrix.(6) Cone spanned by a finite number of vectors :S = {x|x =mj=1λjaj|λj≥ 0, j = 1, . . . , m}, where a1, . . . , amaregiven vectors in IRn.(7) Neighborhood : Nε(¯x) = {x ∈ IRn|x − ¯x < ε}, where ¯x is a fixedvector in IRnand ε > 0.tvnguyen (University of Science) Convex Optimization 8 / 108 Chapter 1 Convex sets and convex functions taking the infinity valueConvex coneSome of the geometric optimality conditions that we will study use convexcones.Definition. A nonempty set C in IRnis called a cone with vertex zeroif x ∈ C implies that αx ∈ C for all α ≥ 0. If, in addition, C is convex,then C is called a convex cone.00xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxConvex coneNonconvex conetvnguyen (University of Science) Convex Optimization 9 / 108 Chapter 1 Convex sets and convex functions taking the infinity valueConvex combination and convex hull of a setDefinition. x is said to be a convex combination of x1, . . . , xmif thereexist α1≥ 0, . . . , αm≥ 0 such thatx = α1x1+ · · · + αmxm, and α1+ · · · + αm= 1.The convex hull of C (denoted conv C ) is the intersection of all convexsubsets containing C.Proposition (Carath´eodory’s lemma). Let C ⊆ IRn. Then eachelement of conv C is a convex combination of at most n + 1 points of Ctvnguyen (University of Science) Convex Optimization 10 / 108 [...]... set tvnguyen (University of Science) Convex Optimization 16 / 108 Outline Chapter 1. Convex sets and convex functions taking the infinity value Chapter 2. Topological properties for sets and functions Chapter 3. Duality for sets and functions Chapter 4. Subdifferential calculus for convex functions Chapter 5. Duality in convex optimization tvnguyen (University of Science) Convex Optimization 3 / 108 Chapter 1... closed function. tvnguyen (University of Science) Convex Optimization 38 / 108 Chapter 2 Topological properties for sets and functions Illustration tvnguyen (University of Science) Convex Optimization 43 / 108 Chapter 2 Topological properties for sets and functions Chapter 2. Topological properties for sets and functions tvnguyen (University of Science) Convex Optimization 26 / 108 Chapter 1 Convex sets and... value Illustration tvnguyen (University of Science) Convex Optimization 11 / 108 References 1. Hiriart–Urruty, J.B. and Lemar´echal, C. : Convex Analysis and Minimization Algorithms, Volumes I and II, Springer, Berlin (1993) 2. Rockafellar, R. T. : Convex Analysis, Princeton University Press, Princeton, New Jersey (1970) 3. Strodiot, J.J. : An Introduction to Nonsmooth Optimization, University of Namur, Belgium (2005) tvnguyen... Then (i) cl f : IR n → IR ∪ {+∞} is proper convex, (ii) cl f and f coincide on ri dom f . tvnguyen (University of Science) Convex Optimization 40 / 108 Chapter 3. Duality for sets and functions Chapter 3. Duality for sets and functions tvnguyen (University of Science) Convex Optimization 46 / 108 Chapter 2 Topological properties for sets and functions Lower semi continuity Definition. Let f : IR n →... IR n tvnguyen (University of Science) Convex Optimization 37 / 108 Chapter 1 Convex sets and convex functions taking the infinity value Quasi-convex functions However if the sublevel sets are convex, the function f is not necessarily convex. See the figure below. A function satisfying that property is called quasi-convex. tvnguyen (University of Science) Convex Optimization 25 / 108 Chapter 1 Convex sets... us introduce another operation called the infimal convolution. tvnguyen (University of Science) Convex Optimization 19 / 108 Chapter 1 Convex sets and convex functions taking the infinity value Chapter 1. Convex sets and convex functions taking the infinity value tvnguyen (University of Science) Convex Optimization 4 / 108 Chapter 2 Topological properties for sets and functions Examples 1. Let C = {x}.... ( k i=1 α i x i ) ≤ k i=1 α i f (x i ). Let us recall that α = (α 1 , . . . , α k ) ∈ ∆ k means all the α i , i = 1, . . . , k are nonnegative and k i=1 α i = 1. tvnguyen (University of Science) Convex Optimization 17 / 108 Chapter 1 Convex sets and convex functions taking the infinity value Convex hull of a function Let f : IR n → IR ∪ {+∞} be proper and minorized by an affine function, i.e., there exists... inf{ k j=1 α j f (x j ) : k = 1, 2, . . . ; α ∈ ∆ k , x j ∈ dom f , k j=1 α j x j = x} where ∆ k = {(α 1 , . . . , α k ) : k j=1 α j = 1, α j ≥ 0 for j = 1, . . . , k} tvnguyen (University of Science) Convex Optimization 22 / 108 Chapter 2 Topological properties for sets and functions Linear Subspaces Let us recall that a subspace L is a subset of IR n which satisfies the property : ∀x, y ∈ L, ∀α ∈ IR, x +... an affine set. Then there exists a unique subspace L parallel to A. Moreover one has A = L + a for every a ∈ A. (iii) The translate of a subspace is an affine set. tvnguyen (University of Science) Convex Optimization 30 / 108 Chapter 2 Topological properties for sets and functions Closure or l.s.c hull of a convex function Definition. The closure (or l.s.c hull) of a convex function f is the function cl... . . . , x k }. Then aff C = k i=1 α i x i k i=1 α i = 1 Examples : aff {x} = {x}, aff [x, y] is the line generated by x and y. aff B(0, 1) = IR n . tvnguyen (University of Science) Convex Optimization 32 / 108 Chapter 2 Topological properties for sets and functions Proposition. Let f : IR n → IR ∪ {+∞} be proper convex. Then f is locally Lipschitz continuous on ri dom f In particular f . Convex OptimizationNguyen Thi Thu VanUniversity of Science2008-2009tvnguyen (University of Science) Convex Optimization 1 / 108 References1.. : An Introduction to Nonsmooth Optimization, Universityof Namur, Belgium (2005)tvnguyen (University of Science) Convex Optimization 2 / 108 OutlineChapter