Using the Newton's method to find the global minimum of a 2D problem with a constraint











up vote
1
down vote

favorite












I am solving an optimization problem
$$min f(x_1,x_2)\
text{s.t.}~~~~ c(x_1,x_2)leq 0$$

In my problem, there are two min and two max and I am looking for the global min.



I know that with the constraint $c(x_1,x_2)leq 0$, there is only one point $(hat{x}_1,hat{x_2})$ in which $nabla f(x_1,x_2)=0 $, and that point in my global min.



To solve the problem efficiently, I start with an initial point (using an algorithm to be sure that the initial point satisfies $c<0$) which is close to the solution and I use Newton's method to solve $nabla f(x_1,x_2)=0 $. For the initial point, first I check the following constraints to be sure that the Newton's method converges to the min not max and escapes from a saddle point:



$$begin{aligned}
&1)~~~~~~~c(x_1,x_2)leq 0\
&2)~~~u^T nabla^2f(x_1,x_2) u>0\
&3)~~lambda_{min}( nabla^2f(x_1,x_2))>0
end{aligned}$$

If the above constraints are not satisfied, I use the steepest descend algorithm to push the point in the area in which all the constraints are satisfied.



Here are my questions: what is the criteria for the initial point in which Newton's method converges to the min and not max (I mean not go far away because the initial point is close to min)? Is Newton's method is the most efficient method? Are the constraints correct?



I would appreciate your answers.










share|cite|improve this question




























    up vote
    1
    down vote

    favorite












    I am solving an optimization problem
    $$min f(x_1,x_2)\
    text{s.t.}~~~~ c(x_1,x_2)leq 0$$

    In my problem, there are two min and two max and I am looking for the global min.



    I know that with the constraint $c(x_1,x_2)leq 0$, there is only one point $(hat{x}_1,hat{x_2})$ in which $nabla f(x_1,x_2)=0 $, and that point in my global min.



    To solve the problem efficiently, I start with an initial point (using an algorithm to be sure that the initial point satisfies $c<0$) which is close to the solution and I use Newton's method to solve $nabla f(x_1,x_2)=0 $. For the initial point, first I check the following constraints to be sure that the Newton's method converges to the min not max and escapes from a saddle point:



    $$begin{aligned}
    &1)~~~~~~~c(x_1,x_2)leq 0\
    &2)~~~u^T nabla^2f(x_1,x_2) u>0\
    &3)~~lambda_{min}( nabla^2f(x_1,x_2))>0
    end{aligned}$$

    If the above constraints are not satisfied, I use the steepest descend algorithm to push the point in the area in which all the constraints are satisfied.



    Here are my questions: what is the criteria for the initial point in which Newton's method converges to the min and not max (I mean not go far away because the initial point is close to min)? Is Newton's method is the most efficient method? Are the constraints correct?



    I would appreciate your answers.










    share|cite|improve this question


























      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      I am solving an optimization problem
      $$min f(x_1,x_2)\
      text{s.t.}~~~~ c(x_1,x_2)leq 0$$

      In my problem, there are two min and two max and I am looking for the global min.



      I know that with the constraint $c(x_1,x_2)leq 0$, there is only one point $(hat{x}_1,hat{x_2})$ in which $nabla f(x_1,x_2)=0 $, and that point in my global min.



      To solve the problem efficiently, I start with an initial point (using an algorithm to be sure that the initial point satisfies $c<0$) which is close to the solution and I use Newton's method to solve $nabla f(x_1,x_2)=0 $. For the initial point, first I check the following constraints to be sure that the Newton's method converges to the min not max and escapes from a saddle point:



      $$begin{aligned}
      &1)~~~~~~~c(x_1,x_2)leq 0\
      &2)~~~u^T nabla^2f(x_1,x_2) u>0\
      &3)~~lambda_{min}( nabla^2f(x_1,x_2))>0
      end{aligned}$$

      If the above constraints are not satisfied, I use the steepest descend algorithm to push the point in the area in which all the constraints are satisfied.



      Here are my questions: what is the criteria for the initial point in which Newton's method converges to the min and not max (I mean not go far away because the initial point is close to min)? Is Newton's method is the most efficient method? Are the constraints correct?



      I would appreciate your answers.










      share|cite|improve this question















      I am solving an optimization problem
      $$min f(x_1,x_2)\
      text{s.t.}~~~~ c(x_1,x_2)leq 0$$

      In my problem, there are two min and two max and I am looking for the global min.



      I know that with the constraint $c(x_1,x_2)leq 0$, there is only one point $(hat{x}_1,hat{x_2})$ in which $nabla f(x_1,x_2)=0 $, and that point in my global min.



      To solve the problem efficiently, I start with an initial point (using an algorithm to be sure that the initial point satisfies $c<0$) which is close to the solution and I use Newton's method to solve $nabla f(x_1,x_2)=0 $. For the initial point, first I check the following constraints to be sure that the Newton's method converges to the min not max and escapes from a saddle point:



      $$begin{aligned}
      &1)~~~~~~~c(x_1,x_2)leq 0\
      &2)~~~u^T nabla^2f(x_1,x_2) u>0\
      &3)~~lambda_{min}( nabla^2f(x_1,x_2))>0
      end{aligned}$$

      If the above constraints are not satisfied, I use the steepest descend algorithm to push the point in the area in which all the constraints are satisfied.



      Here are my questions: what is the criteria for the initial point in which Newton's method converges to the min and not max (I mean not go far away because the initial point is close to min)? Is Newton's method is the most efficient method? Are the constraints correct?



      I would appreciate your answers.







      numerical-methods numerical-optimization numerical-calculus






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Nov 21 at 23:55

























      asked Nov 21 at 23:03









      Joe Hofstrand

      305




      305



























          active

          oldest

          votes











          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "69"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          noCode: true, onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














           

          draft saved


          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3008506%2fusing-the-newtons-method-to-find-the-global-minimum-of-a-2d-problem-with-a-cons%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown






























          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















           

          draft saved


          draft discarded



















































           


          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3008506%2fusing-the-newtons-method-to-find-the-global-minimum-of-a-2d-problem-with-a-cons%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Berounka

          Sphinx de Gizeh

          Different font size/position of beamer's navigation symbols template's content depending on regular/plain...