Glossary¶

This is an incomplete glossary of terms that I use throughout this course. The goal is to have this be a one-stop-shop for defining words and jargon, as well as any phrases that I might use that are unclear to non-native English speakers.

Some of these words have a particular meaning within this course but might mean something else outside the course, even in a technical setting. One example is in this course an “optimizer” is a distinct idea from a “solver,” but other resources might use those words interchangeably. I hope to clear up some potential confusion by explaining words here.

Also, if you’re looking for a more theoretical and complete explanation of some terms, I suggest checking the index of the Engineering Design Optimization textbook and reading more about the specific terms you’re interested in.

Technical words¶

adjoint¶

The adjoint method is an analytic approach to obtaining the derivative information of a system. It is especially efficient for systems where there are more design variables than function of interest. The adjoint equation is \(\frac{\partial r}{\partial u}^{\top} \psi=\frac{\partial f}{\partial u}^{\top}\) where the columns of \(\psi\) are called the adjoint vectors. I won’t go into details here, but Sec. 6.7 in this book introduces and explains the adjoint method.

C0 discontinuity¶

A discontinuity in the design space; a jump; non-contiguous

C1 discontinuity¶

A discontinuity in the derivative space; non-smooth functions; sharp or with kinks

coloring¶

The idea of coloring crops up in a few different fields, including graph theory, CFD, and derivative computation. In this course we’re generally talking about derivative computation. Derivative coloring allows you to compute the derivatives of a system by “painting” certain parts of the Jacobian different colors if they do not influence each other. This OpenMDAO doc page presents the idea of separable problems and coloring quite well.

The benefit of coloring is that if you have some derivative entries in your Jacobian that do not depend on each other, you can reduce the computational cost of solving for derivatives by using coloring to reduce the number of linear solves you need.

A simple example is in the case of multipoint aircraft performance. Imagine we’re optimizing an aircraft and considering its performance across a few different flight conditions. The operating design variables at each one of those flight conditions does not influence the performance at other points, so you can separate the derivative computation using coloring. OpenMDAO can handle this behind the scenes.

constraint¶

These are limitations or requirements that the system must satisfy in order to be considered a valid solution. Constraints can be imposed by the design requirements, physical laws, or other factors.

design variables¶

These are the parameters or variables that can be adjusted during the optimization process. The values of the design variables are typically chosen to minimize the objective function while satisfying the constraints.

discretization¶

The discretization of a model is how it is divided up so that computational methods can be used. Examples include the mesh for CFD, the number of points in a time integration, or how an object is divided into nodes and members for finite element analysis. The discretization is different than the paramterization in that the discretization affects how the analysis of the system is performed, not the optimization.

explicit¶

An explicit relationship is one where the outputs depend only on the inputs. This is a common and simple type of relationship. \(y=x^2+2\) is an example of an explicit function. There are OpenMDAO ExplicitComponents and explicit functions. Even if you only have explicit components in your model, you might have implicit relationships due to feedback coupling in your system.

feasibility¶

A measure of how feasible a design point is within an optimization problem. Specifically, it’s a measure of how “feasible” the solution is, which means how satisfied the constraints are. A design is considered feasible if all of the constraints are satisfied. The feasibility is a numerical measure of the constraint violation; smaller values are better and mean that your constraints are more satisfied.

function of interest¶

This is a common term for all objectives and constraints. You can think of design variables as the inputs to an optimization problem and the functions of interest as the outputs.

gradient-based¶

These optimizers use gradient information to search the design space. Specifically, derivatives of the functions of interest (objective and constraints) with respect to the design variables are used. This is usually first-order information, though some methods also use second derivatives (though this is relatively rare). In general, gradient-based methods are more efficient than gradient-free methods if your problem is smooth and you compute derivative efficiently. This is especially true for large design spaces.

gradient-free¶

This is a type of optimizer that uses only the objective and constraint value information to search the design space. It does not use any gradient (derivative) information. These are also called zeroth-order algorithms. Genetic algorithms and particle swarm methods are examples of gradient-free methods.

implicit¶

An implicit relationship is one where the outputs depend on both the inputs and outputs of a system. The word implicit appears in multiple contexts, including OpenMDAO ImplicitComponents, implicit functions, and the idea of implicit coupling between disciplines. In general, any time you have an implicit relationship you need a solver on the system to resolve that implicitness. \(y=y^4+1\) is an example of an implicit relationship. See also explicit.

Jacobian¶

The Jacobian is the matrix of all first-order partial derivatives. You can think of the Jacobian of a component as the derivatives of the outputs with respect to the inputs. A Jacobian for the full optimization problem is the collection of derivatives for the functions of interest (objective and constraints) wrt the design variables.

model¶

This is the system that you are modeling to represent a physical machine or design. The “multidisciplinary model” consists of multiple disciplines coupled together. The model is also known as the system.

multimodal¶

A multimodal system is one where there are multiple locally optimal designs. You can think of this as a “bumpy” design space, or one with multiple hills and valleys. Multimodal systems exist in real physical situations, such as the optimal placement of a wind turbine within a wind farm or the best route in a grid-like city. Sometimes multimodal systems exist because of your modeling assumptions or computational methods, not necessarily due to physical phenomena. See also unimodal.

objective¶

This is a mathematical function that is used to evaluate the performance of a system. The objective function is typically defined as a measure of the system’s goodness and it is used to guide the optimization process. Conventionally, the objective is minimized within an optimization problem.

optimality¶

A measure of how optimal a given design point is; how well it minimizes a given objective function. Generally, the smaller the optimality value, the better. Depending on the optimization method used, this might not include a notion of if the constraints are satisfied or violated. Optimality is related and distinct from feasibility.

optimizer¶

The optimizer is the numerical algorithm used to find the optimal solution to a design problem. Optimizers can be gradient-free or gradient-based, can be global or local, and come from a variety of mathematical backgrounds. The optimizer is what converges the optimization problem. In this course, an optimizer is distinct from a solver, though other resources may use the terms interchangeably.

parameterization¶

In this course, the parameterization of a system is how it is controlled by the optimizer or designer. You can think of the parameterization as how the design variables are constructed from the available design space.

An example in how you parameterize a system is that of a wind turbine tower’s diameter. You might say that the tower has to be a single diameter all the way from bottom to top. Or you might say you set the diameter at the bottom and the top to be two different values and the tower linearly tapers between them. Or you might have multiple control points along the tower so that you could have an arbitrary shape. These are all examples of different parameterizations for the same type of design problem. Each one grants you different amounts of flexibility and complexity.

This is a separate but related idea to discretization. Your parameterization and discretization could be one in the same, but this is usually not a good idea, e.g. you should not let the optimizer move CFD mesh nodes around individually. By having the parameterization separate from the discretization, you can study how the discretization affects your analysis results and how the parameterization affects the optimization results. This is a subtle yet important distinction.

pseudospectral¶

This is a pretty specialized term that refers to a type of solution method used in mission design and time integration. See this Dymos doc page for more context.

residual¶

The residual is the difference between the current state values in a system and the converged values. In general, when solving an implicit system, you want to converge your residuals to 0 or some acceptable tolerance close to 0. Residual values are printed by OpenMDAO when a solver converges and they look like this:

    NL: Newton 1 ; 0.00130322117 0.921608691
    NL: Newton 2 ; 0.000828003297 0.585545301
    NL: Newton 3 ; 7.02986117e-06 0.00497135964
    NL: Newton 4 ; 2.64550002e-08 1.87083809e-05
    NL: Newton 5 ; 3.78431444e-13 2.67618202e-10
    NL: Newton Converged

This printout shows the solver iteration, the absolute residual value, and the relative residual value. The first relative residual value will always be 1 by definition. You can specify solver tolerances using either or both of the absolute and relative tolerances.

rubber duck¶

A method of debugging code by saying a problem out loud; talking to a rubber duck often helps you overcome programming issues. See this wiki page for more details.

solver¶

Solvers are used to resolve feedback coupling and compute derivatives across a system. Solvers come in many different types, but the main delineation is between nonlinear and linear solvers. Solvers converge the nonlinear or linear systems, not the optimization problem.

unimodal¶

A unimodal system has one global optimum. This means that there are no local extrema besides the singular optimal point. Imagine a paraboloid; there’s one and only one optimum. See also multimodal.

Glossary

Contents

Glossary¶

Technical words¶