Finding the minimum of a multimodal function
Contents
Finding the minimum of a multimodal function¶
Level: Intermediate
Topic: Optimization
Can you find the global minimum of this multimodal function? Where in the design space does this occur and what is the corresponding function value? How would you explore this design space?
Please don’t look at the function definition and use your math smarts to know where the minimum is. Heck, if you can do that, that’s impressive, but that’s not really the point of this exercise.
import openmdao.api as om
import numpy as np
prob = om.Problem()
comp = om.ExecComp('obj = 0.1*(x+y) -abs(sin(x) * cos(y) * exp(abs(1 - (x**2 + y**2)**0.5/pi)))')
prob.model.add_subsystem('blackbox', comp, promotes=['*'])
prob.driver = om.ScipyOptimizeDriver()
prob.driver.options['optimizer'] = 'SLSQP'
prob.model.add_design_var('x', lower=-10, upper=10)
prob.model.add_design_var('y', lower=-10, upper=10)
prob.model.add_objective('obj')
prob.setup()
prob.run_driver();
Optimization terminated successfully (Exit mode 0)
Current function value: -1.6115904097239777
Iterations: 7
Function evaluations: 7
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Solution approaches¶
You have a few different options here. You could:
Simply plot the design space since this is a 2D function that’s relatively cheap.
Run a global gradient-free optimizer on the function.
Run a gradient-based optimizer from multiple starting points, keeping track of where the lowest value occurs.
Let’s walk through these together and I’ll discuss their pros and cons.
Plot the design space¶
This is the easiest and most intuitive way to see a design space. But this wouldn’t be reasonable in the general case for higher dimensional or expensive models. It also wouldn’t necessarily give you the optimum, but you could at least take a look and see how the function looks. Let’s do that now using the DOEDriver from OpenMDAO.
import matplotlib as mpl
import matplotlib.pyplot as plt
%matplotlib inline
prob = om.Problem()
prob.model.add_subsystem('blackbox', comp, promotes=['*'])
prob.driver = om.DOEDriver(om.FullFactorialGenerator(levels=41))
prob.model.add_design_var('x', lower=-10, upper=10)
prob.model.add_design_var('y', lower=-10, upper=10)
prob.model.add_objective('obj')
prob.driver.add_recorder(om.SqliteRecorder("cases.sql"))
prob.setup()
prob.run_driver();
/home/john/anaconda3/envs/course/lib/python3.8/site-packages/openmdao/recorders/sqlite_recorder.py:227: UserWarning:The existing case recorder file, cases.sql, is being overwritten.
/home/john/anaconda3/envs/course/lib/python3.8/site-packages/openmdao/visualization/opt_report/opt_report.py:123: DriverWarning:The optimizer report is not applicable for the DOEDriver Driver which does not support optimization
# check recorded cases from each case file
filename = "cases.sql"
cr = om.CaseReader(filename)
cases = cr.list_cases('driver', out_stream=None)
values = []
for case in cases:
outputs = cr.get_case(case).outputs
values.append((outputs['x'], outputs['y'], outputs['obj']))
values = np.squeeze(np.array(values))
figure = plt.figure()
contour = plt.tricontourf(values[:, 0], values[:, 1], values[:, 2], levels=21)
plt.colorbar()
plt.show()
Oh wow, that’s a pretty rough design space. There are definitely a lot of local minima. But, based on this visualization we can sort of see four main low points near the corners. But which one of those four points is lowest? We probably need to dig deeper into the data, beyond just visualizing it.
sorted_values = values[values[:, 2].argsort(), :]
print(sorted_values[:3, :])
[[ -8. -9.5 -20.66199056]
[ -8. -10. -19.79686168]
[ 8. -9.5 -19.06199056]]
All right, it looks like based on the plot data, the lowest point in the function space is -20.66199 at (-8, -9.5). But is that actually the lowest point in this space? Probably not because we had to discretize the space to plot it; we only have access to points on this grid. Let’s try some other methods.
Run a global gradient-free optimizer¶
We’ll now use a simple genetic algorithm from OpenMDAO to optimize this model.
prob = om.Problem()
prob.model.add_subsystem('blackbox', comp, promotes=['*'])
prob.driver = om.SimpleGADriver()
prob.driver.options['bits'] = {'x': 8, 'y': 8}
prob.driver.options['max_gen'] = 5
prob.model.add_design_var('x', lower=-10, upper=10)
prob.model.add_design_var('y', lower=-10, upper=10)
prob.model.add_objective('obj')
prob.driver.add_recorder(om.SqliteRecorder("GA_cases.sql"))
prob.setup()
prob.run_driver();
/home/john/anaconda3/envs/course/lib/python3.8/site-packages/openmdao/recorders/sqlite_recorder.py:227: UserWarning:The existing case recorder file, GA_cases.sql, is being overwritten.
# check recorded cases from each case file
filename = "GA_cases.sql"
cr = om.CaseReader(filename)
cases = cr.list_cases('driver', out_stream=None)
values = []
for case in cases:
outputs = cr.get_case(case).outputs
values.append((outputs['x'], outputs['y'], outputs['obj']))
values = np.squeeze(np.array(values))
sorted_values = values[values[:, 2].argsort(), :]
print(sorted_values[0, :])
[ -8.03921569 -9.76470588 -20.88419693]
Run multiple gradient-based optimizations¶
prob = om.Problem()
prob.model.add_subsystem('blackbox', comp, promotes=['*'])
prob.driver = om.ScipyOptimizeDriver()
prob.driver.options['optimizer'] = 'SLSQP'
prob.model.add_design_var('x', lower=-10, upper=10)
prob.model.add_design_var('y', lower=-10, upper=10)
prob.model.add_objective('obj')
np.random.seed(1234)
n_samples = 100
values = []
random_starts = np.random.rand(n_samples, 2) * 20 - 10
for (x, y) in random_starts:
prob.setup()
prob.set_val('x', x)
prob.set_val('y', y)
prob.run_driver();
values.append((prob['x'], prob['y'], prob['obj']))
values = np.squeeze(np.array(values))
Optimization terminated successfully (Exit mode 0)
Current function value: -2.5163226799777205
Iterations: 7
Function evaluations: 9
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.548639620822383
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.185370448272815
Iterations: 6
Function evaluations: 7
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.551420299976121
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -8.047972489021983
Iterations: 6
Function evaluations: 10
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.864212749912046
Iterations: 7
Function evaluations: 8
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.5219023495075503
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -11.555259952709907
Iterations: 8
Function evaluations: 11
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -19.36996687742009
Iterations: 6
Function evaluations: 10
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.571121924446147
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.21898685990108
Iterations: 7
Function evaluations: 9
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.853988271458861
Iterations: 4
Function evaluations: 7
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.573863167636631
Iterations: 5
Function evaluations: 8
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.548639487893976
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -10.36101597957613
Iterations: 5
Function evaluations: 6
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.896564505995247
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.877690424901129
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.5219020415486604
Iterations: 7
Function evaluations: 7
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -7.285741216253686
Iterations: 6
Function evaluations: 10
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -7.285741190072775
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -10.963385516840605
Iterations: 5
Function evaluations: 9
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.202161732007603
Iterations: 6
Function evaluations: 10
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.611590273359305
Iterations: 6
Function evaluations: 7
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -5.5289861241353915
Iterations: 8
Function evaluations: 9
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.240703403002165
Iterations: 5
Function evaluations: 6
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.6675165522796
Iterations: 6
Function evaluations: 10
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -8.906913536945543
Iterations: 3
Function evaluations: 7
Gradient evaluations: 3
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.961103710460614
Iterations: 7
Function evaluations: 9
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.614818715952591
Iterations: 6
Function evaluations: 10
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -17.437037218096012
Iterations: 8
Function evaluations: 13
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.5163226777953946
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.865086698310149
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -8.90691353707806
Iterations: 5
Function evaluations: 9
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -7.285741171579576
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -10.963385496358304
Iterations: 6
Function evaluations: 11
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.5219022520603236
Iterations: 6
Function evaluations: 7
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.231526297469605
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -10.585608935895351
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.8642130058468651
Iterations: 6
Function evaluations: 7
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.9789753719340295
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.0211842009289727
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.573863229459882
Iterations: 5
Function evaluations: 8
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.877690424932257
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.571121824904056
Iterations: 4
Function evaluations: 6
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.6115899917649823
Iterations: 5
Function evaluations: 9
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -5.528986147389784
Iterations: 7
Function evaluations: 8
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.614818686597976
Iterations: 6
Function evaluations: 10
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.8642129238343554
Iterations: 6
Function evaluations: 7
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.21898696512568
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.202161699701514
Iterations: 5
Function evaluations: 9
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.5219023260237914
Iterations: 6
Function evaluations: 7
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.2366544010053995
Iterations: 5
Function evaluations: 6
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.343900008204736
Iterations: 6
Function evaluations: 11
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.202161663214796
Iterations: 8
Function evaluations: 11
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -19.04805362701734
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -10.361016315225042
Iterations: 6
Function evaluations: 6
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -7.285741216401198
Iterations: 8
Function evaluations: 10
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -19.36996675397779
Iterations: 5
Function evaluations: 9
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.865086767434049
Iterations: 5
Function evaluations: 8
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.573863231639694
Iterations: 5
Function evaluations: 9
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.6148187162357
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.5219021032829243
Iterations: 4
Function evaluations: 6
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -5.85783926573642
Iterations: 5
Function evaluations: 8
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.516322678763987
Iterations: 4
Function evaluations: 7
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -20.98095921221227
Iterations: 7
Function evaluations: 10
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.667516811437427
Iterations: 8
Function evaluations: 11
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -5.225772319010425
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -10.36101630727762
Iterations: 5
Function evaluations: 6
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -8.361016315148248
Iterations: 5
Function evaluations: 5
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -10.361016233779605
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -16.30390683583646
Iterations: 5
Function evaluations: 6
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.8285837766790425
Iterations: 3
Function evaluations: 4
Gradient evaluations: 3
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.853988328662048
Iterations: 4
Function evaluations: 7
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.8776903702930925
Iterations: 4
Function evaluations: 7
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.3624195900087614
Iterations: 6
Function evaluations: 6
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.3311030745719545
Iterations: 3
Function evaluations: 4
Gradient evaluations: 3
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -5.857839314119491
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.5711218816188146
Iterations: 5
Function evaluations: 8
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -9.614818694416781
Iterations: 8
Function evaluations: 10
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -16.232351770570354
Iterations: 11
Function evaluations: 11
Gradient evaluations: 11
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.5163226798988685
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -8.047972457265182
Iterations: 7
Function evaluations: 12
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -7.070195324579004
Iterations: 5
Function evaluations: 5
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.6115904352943982
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -17.437037247074016
Iterations: 5
Function evaluations: 8
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.3624195818978928
Iterations: 4
Function evaluations: 5
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.551420299306278
Iterations: 8
Function evaluations: 9
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.516322604197141
Iterations: 7
Function evaluations: 8
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -2.5486395852310393
Iterations: 5
Function evaluations: 7
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.5219022241621536
Iterations: 6
Function evaluations: 7
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.6115904383777737
Iterations: 4
Function evaluations: 6
Gradient evaluations: 4
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -4.57386292929036
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -12.52601054221016
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.8642129645798633
Iterations: 8
Function evaluations: 9
Gradient evaluations: 8
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -1.704275824225047
Iterations: 5
Function evaluations: 6
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -3.535572671379102
Iterations: 7
Function evaluations: 8
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -19.04805354510481
Iterations: 7
Function evaluations: 11
Gradient evaluations: 7
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.853988334456597
Iterations: 5
Function evaluations: 8
Gradient evaluations: 5
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -8.906913536963868
Iterations: 6
Function evaluations: 8
Gradient evaluations: 6
Optimization Complete
-----------------------------------
Optimization terminated successfully (Exit mode 0)
Current function value: -6.96110371762455
Iterations: 6
Function evaluations: 9
Gradient evaluations: 6
Optimization Complete
-----------------------------------
sorted_values = values[values[:, 2].argsort(), :]
print(sorted_values[:3, :])
[[ -8.06001968 -9.66948136 -20.98095921]
[ 8.0499079 -9.6696239 -19.36996688]
[ 8.04990598 -9.66949297 -19.36996675]]
Cool! We’ve seen the lowest value yet, occurring at (-8.06002, -9.66948) and yielding -20.98096. Additionally, at least on my computer, this collection of gradient-based optimizations runs the quickest of any of the methods we’ve tried here. Obviously that’s problem-dependent, but it’s usually a good idea to start with multiple gradient-based optimizations instead of using a global gradient-free optimizer, just from a computational expense standpoint.
Recap¶
I hope this exercise proved to be useful for you to see how to investigate a pretty multimodal and sharp design space. Something like this could crop up in wind turbine layout design, aircraft engine performance optimization, or a slew of other cases. It really pays to systematically explore the design space and better understand it so that you can achieve optimal designs more quickly.