Debugging your optimizations part 2¶

Hello everybody and welcome to part two of debugging your optimizations. This is a continuation of a previous video so if you have not seen part one yet I highly recommend checking it out.

You remember this checklist. Today I will be going through numbers two, three, and four. These tips will be mostly focused on your optimization problem formulation, understanding that, and working within it. And then we’ll also look at starting from a feasible point in your design space. Most of today’s lesson will be focused on using parts of the notebook and me kind of walking you through them.

So first we need to check our optimization problem formulation. It’s way too easy to write a poorly posed optimization problem and I’ll get into what that means. But the whole idea here is that you want to set your optimizer up for success. You want to give it a problem that it can solve well. You want to make sure that you understand what kind of solutions you’ll be getting out of it. What I mean by a poorly posed problem is that it’s unnecessarily challenging for the optimizer to solve. This might be due to mathematical reasons or computational reasons.

There are a few common things to look for if you’re concerned about poorly posed problems. One is competing constraints. So if you have a constraint on a set of design variables and it constrain an output and there’s no way to satisfy them at the same time, you’re going to run into issues. Your optimizer cannot solve this problem; there’s no way to satisfy all these constraints. So if you’re seeing that the optimizer is kind of stuck in an infeasible point to the design space you could try turning off constraints one at a time and see if it helps the optimizer get out of those humps.

A related idea is if your lower or upper bounds on design variables or constraints are unnecessarily restrictive, you want to make sure that they’re based on something that’s realistic. Maybe it’s based on the manufacturing tolerances, maybe it’s based on something else. You need to make sure that both the units and the magnitudes for your bounds make sense for your problem. For example, if you’re trying to control the span of an aircraft wing and that aircraft is hauling, you know 50 passengers, and the span is an upper bound of two meters, something’s not going to work here. Hopefully if you’re the one setting up your optimization problem you have some notion of what kind of physical bounds make sense.

This last point here; copy and paste oversights. It’s all too common. You start with a working script for an optimization, you copy it over for another case, and you forget to change something that you should change. I’ve done this way too many times. I can’t count how many times I’ve done this. One time I was looking at the wing design of a Boeing 777, so a huge commercial airliner, and later copied parts of the script over for a hand launched UAV, so just a few pounds UAV. It’s easy to take some values, not think about it too much, not even know the hundreds of lines in your run script, and use them so again. Look for any kind of typos, especially copy and paste oversights.

But these are just examples of how you can sometimes get poorly posed problems. Let’s talk about some well-posed problems. The idea is that you have no design variables that are controlling the same thing. Let’s say that you’re looking at an aircraft wing, and you have angle of attack, but you also have twist. If you’re controlling the angle of attack of the aircraft and the twist along all parts of the span the design space is not entirely independent. You want to avoid this because it allows the optimizer to have multiple design variables that control the same thing and that just really confuses the optimizer.

In this example I’m showing the root twist is fixed, we have eight twist control areas all outboard. However, the part that touches the fuselage is fixed. Because we also have angle of attack as a design variable here this avoids having any sort of competing design variables. Additionally you want to make sure that there’s a combination of constraints and design variables that allow for a feasible point in the design space. If there is no feasible point in the design space, of course the optimizer cannot succeed.

I’ll just briefly touch on these other ones but the whole idea here is that you want the optimizer to make steady progress throughout the optimization, to change all the design variables that it needs to, kind of be sensitive to them by the same order of magnitude, and then lastly once you get a converged answer you need to make sure you can interpret the results. So a well-posed optimization problem is usually one where the results make sense. If you get some garbage coming out of your optimizer maybe it’s not well-posed, maybe it’s not fully converged. There’s a few different things for you to check out.

So scrolling down here I have an example of a poorly posed optimization problem. I suggest you take a look at this, try running it, and try changing some of the bounds or some of the constraints to see if you can get it to converge well.

My next suggestion, number three here, is to try a simpler optimization problem. What I mean by this is that you can use fewer design variables or fewer constraints. It sounds great to really try to run the most complicated problem that you actually want to run but you may need to start with a simpler one and kind of build your way up. I highly suggest doing this; it’ll save you a lot of time in the end. Additionally, and this is also related, you can use a cheaper analysis. If you have like a very detailed mesh that you want to use for your optimization, maybe start with a courser mesh, something that’s cheaper to run so that you can debug your optimization.

But so I now just want to highlight one example of an actual case, an actual project where I built up the optimization problem piece by piece. This is going to look pretty busy but I just want to introduce the idea of doing this. You don’t need to understand what all these words mean. This entire project was focused on both designing the airfoils for a blade as well as the span-wise quantities, like twist and chord. We are looking at aerostructural performance and so here I’m trying to build up the complexity of the optimization problem piece by piece. I start with 0_0, make initial case, which is just analysis. Then 0_1 is twist only, so I’m only controlling the twist of the blade. Then I’m also looking at the chord and twist at the same time. Another time I’m adding structural thickness of the spar caps as another design variable. So the idea here is to very purposely build up the complexity of the design problems and have folders with each of the the actual run scripts they use so you can add on to them. You can imagine if I want to run something with many types of design variables here I’m not going to jump straight to this one right here, I’m going to slowly build it up make sure that everything works, make sure that it’s behaving as expected, it follows my intuition.

This is a really helpful way to help debug your optimizations and just make sure everything’s working as intended. Then once you’ve run these cases you can also kind of post-process them. So this is part of what I call a kind of monster plot and you can think of it as going from left to right as increasing the optimization complexity. And this allows me to very quickly look at a few different cases here. Again we have this make initial case, the analysis, then we have twist, chord, spar caps, and airfoil design, and then even more design variables here, so many that is being cut off by the PDF. But the whole idea here is, okay, if I have these different cases and I’m slowly building up complexity, I want to see how they perform, I want to see the different levels of complexity and what they mean for both the design variables and the performance of the actual optimization. When you look at it graphically maybe something stands out in the more complex case that you don’t see in the simpler case and you can say “oh, maybe I need to look at that in more detail.”

I’m going to flash up a table here detailing one of the ultimate optimization problems for my dissertation. There’s a lot going on here. There are many design variables; there are arrays, they have different values, different limits, and there are many, many constraints as well. I wish I could have, but I did not just start with this optimization problem because that would not have worked. You must start with a very simple one and work your way up by verifying the results as you go and solving each subsequent optimization problem and slowly increasing the complexity; I was able to run this very complex problem. Again if you’re trying to run a problem, maybe one not as complex as this, or one even more complex, and you run into some issues, a good way to debug is to use a simpler problem formulation to start.

Now here’s tip number four; it’s to start from a feasible point in the design space. Some optimizers mathematically require that you start from a feasible point. Now they say “hey, get me close and I’ll try to figure it out myself,” but the whole idea here is that you need to make sure your initial condition makes sense. If you’re trying to design an aircraft wing and it breaks under the initial load, that means that one of the constraints is not satisfied, maybe it makes sense to kind of beef up that initial condition, make sure that you can start from a feasible point in the design space, and that would kind of set your optimizer up for success. Additionally you might save time, it may take fewer iterations to converge. Now there may be the case where you don’t know how to make a feasible point in the design space. I highly recommend that you try to figure one out and you try to start your optimizer there. This again kind of prevents your optimizer from getting trapped in infeasible points in the design space. I would also like to acknowledge a lot of optimizers can traverse both feasible and infeasible points in the design space and move through them. But at the same time you usually want to start with a feasible point in the space.

So as you know this was just part two in a multi-part series on debugging your optimization. We will have more later on and make sure to check out part one if you haven’t already. Please let me know in the comments if you have any questions and I can address them in the follow-on to this lecture. Guys, gals, and non-binary pals; thank you for watching!