tag:blogger.com,1999:blog-6061887630060661987.post4076341170268576483..comments2024-03-13T13:54:35.259-04:00Comments on dlib C++ Library: A Global Optimization Algorithm Worth UsingDavis Kinghttp://www.blogger.com/profile/16577392965630448489noreply@blogger.comBlogger102125tag:blogger.com,1999:blog-6061887630060661987.post-50417522565601906412023-11-11T16:41:56.144-05:002023-11-11T16:41:56.144-05:00No problem, glad you find it useful :)No problem, glad you find it useful :)Davis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-51915462330582129372023-11-07T22:22:04.282-05:002023-11-07T22:22:04.282-05:00This comment has been removed by the author.Troyhttps://www.blogger.com/profile/13579153026566238581noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-62406092480942553232023-11-05T00:24:52.233-04:002023-11-05T00:24:52.233-04:00Davis, thank you! I found another place where you...Davis, thank you! I found another place where you used std::shared_ptr in another part of the code for this purpose and copied your approach there. That seems to work. Thanks for your reply and also for all of the work that you did in putting this together. I'm far from a C++ guru so I appreciate your help.Troyhttps://www.blogger.com/profile/13579153026566238581noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-12127953321822899462023-11-04T22:52:47.723-04:002023-11-04T22:52:47.723-04:00I would use std::optional or a similar type so tha...I would use std::optional or a similar type so that you can create a std::optional that is nullopt initially. Then when you have the information on hand to create the dlib::function_evaluation_request you assign the dlib::function_evaluation_request to the optional variable. If your compiler doesn't yet support std::optional std::shared_ptr or dlib::optional work fine too (dlib::optional is a backport of std::optional for people with older compilers).Davis Kinghttps://www.blogger.com/profile/09267306184784595640noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-66011612085678619712023-11-03T22:21:29.861-04:002023-11-03T22:21:29.861-04:00I love your work here and I'm eager to use it....I love your work here and I'm eager to use it. I'm using a cluster where I want to queue jobs to be done and then, as the results come in, queue new jobs (without waiting for all of the queued jobs to complete.) I'm having some trouble with function_evaluation_request. I have a vector of "Computation" defined below that keeps track of stuff like the UNIX process number, input parameters, and results for each job that I submit but since function_evaluation_request has the default constructor of "delete()", I can't declare it and then assign it in my "Computation" constructor. I'm sure there is a clever way around this but it currently has me stumped. What is the correct way to work around this situation? Thanks in advance.<br /><br />class Computation<br />{<br /> public:<br /> bool active = false;<br /> Slurm_process process;<br /> dlib::function_evaluation_request request;<br /> arma::colvec points;<br /> double result;<br /><br /> Computation(dlib::global_function_search & opt, void (*setup_function)(const arma::colvec &))<br /> {<br /> // All we do here is ask the global_function_search object what to evaluate next, then do what it asked<br /> active = true;<br /> dlib::function_evalueation_request test = opt.get_next_x();<br /> request = test;<br /> points = {request.x()(0), request.x()(1), request.x()(2)}; // Altitude, inclination, raan<br /> std::string this_dirname = "sim_alt" + std::to_string(points(0)) + "_i" + std::to_string(points(1)) + "_raan" + std::to_string(points(2)) + "_date22001";<br /> process = Slurm_process(make_config_files, this_dirname, points);<br /> }<br /><br /> bool get_results()<br /> {<br /> if (active && process.get_results(result))<br /> {<br /> // report the results back by calling function_evaluation_request's set() method.<br /> request.set(computation.result);<br /> active = false;<br /> }<br /> }<br />};<br />Troyhttps://www.blogger.com/profile/13579153026566238581noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-41936350622156745742022-12-10T19:48:17.896-05:002022-12-10T19:48:17.896-05:00Yeah, I should make this use a low discrepancy seq...Yeah, I should make this use a low discrepancy sequence. That would totally be a bit better :DDavis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-5855384299046758132022-11-18T08:41:29.569-05:002022-11-18T08:41:29.569-05:00Davis, thank you for making such high-quality work...Davis, thank you for making such high-quality work freely available for us all.<br /><br />As a response to your post a while back, <i>"I just evaluate 5000 random points and take the best. I thought about how you might try to optimize U(x) exactly, but I'm not sure there is an efficient way. And random search of U(x) is plenty good enough."</i><br /><br />Here is a neat post about low discrepancy quasi-random sequences which may be worth investigating.<br /><br />http://extremelearning.com.au/unreasonable-effectiveness-of-quasirandom-sequences/<br /><br />I particularly like the elegance of their generation and they have really nice scaling properties in higher dimensions.<br /><br />Quote from his site, <i> "The new R_d sequence is the only d-dimensional low discrepancy quasirandom sequence that does not require any selection of basis parameters."</i><br /><br />Thanks again for the great work<br />JacquesAnonymoushttps://www.blogger.com/profile/05716497770029645103noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-10780500395051268662022-10-13T05:20:40.261-04:002022-10-13T05:20:40.261-04:00This comment has been removed by the author.Annhttps://www.blogger.com/profile/03153473345437499388noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-8657370411812156862022-10-13T04:44:50.751-04:002022-10-13T04:44:50.751-04:00This comment has been removed by the author.Annhttps://www.blogger.com/profile/03153473345437499388noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-81199488462109477792022-10-13T04:20:22.958-04:002022-10-13T04:20:22.958-04:00For future readers who have difficulties in instal...For future readers who have difficulties in install dlib:<br /><br /> -- Using CMake version: 3.24.1<br /> -- Compiling dlib version: 19.24.0<br /> ...<br />Successfully installed dlib-19.24.0<br /><br />YEAH!!<br /><br />I fixed the installation on Windows 10. Only install Visual Studio is not sufficient, you have to install C++ explicitly by "Modify" then in the "Individual Components" tab of VS installer, under the name of "MSVC v140 - VS2015 C++ Build Tools (v14.00)" and needs to download 800 MB. <br /><br />I also read from a thread that <br />"Use the link to Visual C++ 2015 Build Tools. That will install Visual C++ 14.0 without installing Visual Studio." >> see https://stackoverflow.com/questions/44951456/pip-error-microsoft-visual-c-14-0-is-required<br /><br />========<br /><br />Above is for Windows, still not sure the installation on Mac system. Annhttps://www.blogger.com/profile/03153473345437499388noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-86687745141652254422022-10-13T03:53:44.981-04:002022-10-13T03:53:44.981-04:00This comment has been removed by the author.Annhttps://www.blogger.com/profile/03153473345437499388noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-53136572721872737872022-10-13T03:09:35.462-04:002022-10-13T03:09:35.462-04:00This comment has been removed by the author.Annhttps://www.blogger.com/profile/03153473345437499388noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-61366057784881236072022-10-12T21:54:26.476-04:002022-10-12T21:54:26.476-04:00Yeah no deriviatives or or anything like that are ...Yeah no deriviatives or or anything like that are required. You just need to provide the result of the function. E.g. dlib will say "what's the objective function output for inputs (1,2,3,4)?" and your code has to say "it's 5.678".Davis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-58264824529575768392022-10-12T11:20:33.306-04:002022-10-12T11:20:33.306-04:00Hi Davis,
I've been using BO to minimize a re...Hi Davis,<br /><br />I've been using BO to minimize a response surface which is from a surrogate model approximated by Gaussian Process Regression. The Acquisition function I adopted is Expected Improvement. The score I compare is the RMSE. The RMSE keeps decreasing with each step until a point then the RMSE is actually getting bigger. It might be that the hyperparams are not "good" enough. <br /><br />I find this article after some Q&A on stackexchange. I wonder if dlib can be used with the surrogate model (from GPR) which does not have an exact expression? like the one in the python example: "-abs(sin(x0)*cos(x1)*exp(abs(1-sqrt(x0*x0+x1*x1)/pi)))"?<br /><br />Or is BO the only option if I'm using a surrogate model? Thank you.Annhttps://www.blogger.com/profile/03153473345437499388noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-78691605037233052542022-03-28T08:05:54.745-04:002022-03-28T08:05:54.745-04:00No, you can't do it with python because python...No, you can't do it with python because python doesn't support threading.Davis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-20987835753809915502022-03-24T08:10:33.309-04:002022-03-24T08:10:33.309-04:00Hello Davis,
thank you for your great software! I ...Hello Davis,<br />thank you for your great software! I am seeing from the documentation that find_min_global allows concurrent function calls on multiple threads: can this be done also with its Python version? Thank you.Alessiohttps://www.blogger.com/profile/13685456794996157405noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-4418164277352708272022-03-06T15:05:16.237-05:002022-03-06T15:05:16.237-05:00That lipo package is news to me. I think you shou...That lipo package is news to me. I think you should use dlib directly if you want to use the optimizer in dlib (which is what this post is about).Davis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-61395616213254415592022-03-06T04:46:09.267-05:002022-03-06T04:46:09.267-05:00Thank you for lipo. I am using `from lipo import ...Thank you for lipo. I am using `from lipo import GlobalOptimizer` in Python and was wondering if there is any documentation for `categories` and `evaluations`. My guess is that evaluations sets some fixed points to evaluate at first but what is categories for?Unknownhttps://www.blogger.com/profile/03477220667684902066noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-49893957294445179322021-02-16T22:37:44.258-05:002021-02-16T22:37:44.258-05:00Sweet, glad to hear it :)Sweet, glad to hear it :)Davis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-1431567444101264882021-02-16T22:11:50.288-05:002021-02-16T22:11:50.288-05:00I wanted to let you know that find_min_global is c...I wanted to let you know that find_min_global is currently outperforming a dozen popular global optimizers. <br /><br />https://github.com/microprediction/optimizer-elo-ratings/tree/main/results/leaderboards/overall<br /><br />I'm really impressed and grateful for this library. Peter Cottonhttps://www.blogger.com/profile/05565219939665267344noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-17796766720422027162021-02-04T21:35:11.406-05:002021-02-04T21:35:11.406-05:00Yeah, it should work alright for that.Yeah, it should work alright for that.Davis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-841777204888758452021-02-04T04:50:21.798-05:002021-02-04T04:50:21.798-05:00Hello Davis,
Thank you so much for this post. Cou...Hello Davis,<br /><br />Thank you so much for this post. Could you please help me understand whether I can use this method with nested hyperparameters? i.e., constraining a child hyperparameter to be active and evaluated only when the parent hyperparameter is active. Thank you very much.Azuriehttps://www.blogger.com/profile/12217309453825985892noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-68752529221836002162020-09-28T08:00:02.485-04:002020-09-28T08:00:02.485-04:00It will be on pypi eventually when the next dlib v...It will be on pypi eventually when the next dlib version is released. For now you can just run python setup.py install and it will do the same thing.Davis Kinghttps://www.blogger.com/profile/16577392965630448489noreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-8486539160417811652020-09-25T10:28:44.672-04:002020-09-25T10:28:44.672-04:00Hi Davis,
Would the latest version with 35 paramet...Hi Davis,<br />Would the latest version with 35 parameters be available on pypi ? pip install makes life much easier when integrating with other tools.<br />Totally understand if you prefer not to push it to pypi for now.<br />Thank youAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-6061887630060661987.post-69589449670009686952020-09-25T09:59:30.561-04:002020-09-25T09:59:30.561-04:00Thank you ! appreciate the quick responseThank you ! appreciate the quick responseAnonymousnoreply@blogger.com