Astral Space

From three of today’s top researchers in machine learning, a groundbreaking new theory for understanding convex minimization at infinityNumerous fields of study rely on methods for minimizing convex functions. Not all convex functions, however, have finite minimizers; some can only be minimized by a sequence as it heads to infinity, making it considerably more challenging to prove correct convergence to a minimizer. This book develops an expansive new theory for understanding such minimizers at infinity, introducing astral space, a compact extension of Euclidean space to which such points at infinity have been added. Astral space is constructed to be as small as possible while still ensuring that all linear functions can be continuously extended to the new space. These favorable properties make it especially compatible with standard convex analysis, whose key notions are systematically extended to the new space, providing the foundation for a more complete theory. Astral space includes Euclidean space but is neither a vector space nor a metric space. Nevertheless, it is sufficiently well-structured to admit useful and meaningful extensions of the most important concepts from convex analysis, including convexity of sets and functions, conjugacy, separation theorems, subdifferentials, as well as central topics from optimization and applications, such as Fenchel duality, KKT conditions, and exponential-family distributions. Applied to widely used algorithms, these tools afford simplified proofs of convergence, even when the only minimizers are at infinity. All these and more are fully explored and elucidated with care and rigor, beginning with a review of general topology and convex analysis, with numerous figures and examples throughout.