Taylor expansion of likelihood function
$require{cancel}$
...For large samples, as a consequence of the central limit theorem, the
likelihood function approaches a gaussian, whose expected value is
equal to the maximum likelihood estimate. Indeed, expanding the
logarithm of the likelihood function around its maximum,
$$ln{mathscr{L}(theta)}=ln{mathscr{L}}(hattheta)-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2 + dots$$
for a "sufficient large" number of samples, the so called parabolic
approximation can be considered, such that the likelihood function can
be approximated by a gaussian-like expression, i.e.,
$$mathscr{L}(theta)propto e^{-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial {theta}^2}right|_hattheta(theta-hattheta)^2}...$$
I tried the Taylor expansion at $ hattheta$ as:
$$ln{mathscr{L}(theta)}=
ln{mathscr{L}}(hattheta)
+ cancel{left. dfrac{partial lnmathscr{L}}{partial theta}right|_{hattheta}left(theta-hatthetaright)}
color{red}+dfrac{1}{2}left. dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2
+ dots$$
Why the minus sign?
statistics taylor-expansion maximum-likelihood log-likelihood
asked Dec 5 '18 at 16:12
FredFred
1031
add a comment |
$require{cancel}$
...For large samples, as a consequence of the central limit theorem, the
likelihood function approaches a gaussian, whose expected value is
equal to the maximum likelihood estimate. Indeed, expanding the
logarithm of the likelihood function around its maximum,
$$ln{mathscr{L}(theta)}=ln{mathscr{L}}(hattheta)-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2 + dots$$
for a "sufficient large" number of samples, the so called parabolic
approximation can be considered, such that the likelihood function can
be approximated by a gaussian-like expression, i.e.,
$$mathscr{L}(theta)propto e^{-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial {theta}^2}right|_hattheta(theta-hattheta)^2}...$$
I tried the Taylor expansion at $ hattheta$ as:
$$ln{mathscr{L}(theta)}=
ln{mathscr{L}}(hattheta)
+ cancel{left. dfrac{partial lnmathscr{L}}{partial theta}right|_{hattheta}left(theta-hatthetaright)}
color{red}+dfrac{1}{2}left. dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2
+ dots$$
Why the minus sign?
statistics taylor-expansion maximum-likelihood log-likelihood
asked Dec 5 '18 at 16:12
FredFred
1031
Where does the quote come from by the way?
– epimorphic
Dec 5 '18 at 16:29
1
“Métodos Estatísticos em Física Experimental, Oguri V, 2014” (Portuguese)
– Fred
Dec 5 '18 at 16:36
add a comment |
$require{cancel}$
...For large samples, as a consequence of the central limit theorem, the
likelihood function approaches a gaussian, whose expected value is
equal to the maximum likelihood estimate. Indeed, expanding the
logarithm of the likelihood function around its maximum,
$$ln{mathscr{L}(theta)}=ln{mathscr{L}}(hattheta)-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2 + dots$$
for a "sufficient large" number of samples, the so called parabolic
approximation can be considered, such that the likelihood function can
be approximated by a gaussian-like expression, i.e.,
$$mathscr{L}(theta)propto e^{-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial {theta}^2}right|_hattheta(theta-hattheta)^2}...$$
I tried the Taylor expansion at $ hattheta$ as:
$$ln{mathscr{L}(theta)}=
ln{mathscr{L}}(hattheta)
+ cancel{left. dfrac{partial lnmathscr{L}}{partial theta}right|_{hattheta}left(theta-hatthetaright)}
color{red}+dfrac{1}{2}left. dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2
+ dots$$
Why the minus sign?
statistics taylor-expansion maximum-likelihood log-likelihood
asked Dec 5 '18 at 16:12
FredFred
1031
$require{cancel}$
...For large samples, as a consequence of the central limit theorem, the
likelihood function approaches a gaussian, whose expected value is
equal to the maximum likelihood estimate. Indeed, expanding the
logarithm of the likelihood function around its maximum,
$$ln{mathscr{L}(theta)}=ln{mathscr{L}}(hattheta)-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2 + dots$$
for a "sufficient large" number of samples, the so called parabolic
approximation can be considered, such that the likelihood function can
be approximated by a gaussian-like expression, i.e.,
$$mathscr{L}(theta)propto e^{-dfrac{1}{2}left|dfrac{partial^2lnmathscr{L}}{partial {theta}^2}right|_hattheta(theta-hattheta)^2}...$$
I tried the Taylor expansion at $ hattheta$ as:
$$ln{mathscr{L}(theta)}=
ln{mathscr{L}}(hattheta)
+ cancel{left. dfrac{partial lnmathscr{L}}{partial theta}right|_{hattheta}left(theta-hatthetaright)}
color{red}+dfrac{1}{2}left. dfrac{partial^2lnmathscr{L}}{partial
{theta}^2}right|_hatthetaleft(theta-hatthetaright)^2
+ dots$$
Why the minus sign?
statistics taylor-expansion maximum-likelihood log-likelihood
statistics taylor-expansion maximum-likelihood log-likelihood
asked Dec 5 '18 at 16:12
FredFred
1031
asked Dec 5 '18 at 16:12
FredFred
1031
asked Dec 5 '18 at 16:12
FredFred
1031
asked Dec 5 '18 at 16:12
FredFred
1031
asked Dec 5 '18 at 16:12
FredFred
1031
1031
Where does the quote come from by the way?
– epimorphic
Dec 5 '18 at 16:29
1
“Métodos Estatísticos em Física Experimental, Oguri V, 2014” (Portuguese)
– Fred
Dec 5 '18 at 16:36
add a comment |
Where does the quote come from by the way?
– epimorphic
Dec 5 '18 at 16:29
1
“Métodos Estatísticos em Física Experimental, Oguri V, 2014” (Portuguese)
– Fred
Dec 5 '18 at 16:36
Where does the quote come from by the way?
– epimorphic
Dec 5 '18 at 16:29
Where does the quote come from by the way?
– epimorphic
Dec 5 '18 at 16:29
1
1
“Métodos Estatísticos em Física Experimental, Oguri V, 2014” (Portuguese)
– Fred
Dec 5 '18 at 16:36
“Métodos Estatísticos em Física Experimental, Oguri V, 2014” (Portuguese)
– Fred
Dec 5 '18 at 16:36
add a comment |
1 Answer
1
active
oldest
votes
The minus sign comes from the fact that at the maximum of ln(L), the second order condition requires that necessarily ln(L)''<=0. And the opposite of the absolute value of ln(L)'' is equal in this case to ln(L)''.
answered Dec 5 '18 at 17:07
BertrandBertrand
813
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3027264%2ftaylor-expansion-of-likelihood-function%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The minus sign comes from the fact that at the maximum of ln(L), the second order condition requires that necessarily ln(L)''<=0. And the opposite of the absolute value of ln(L)'' is equal in this case to ln(L)''.
answered Dec 5 '18 at 17:07
BertrandBertrand
813
add a comment |
The minus sign comes from the fact that at the maximum of ln(L), the second order condition requires that necessarily ln(L)''<=0. And the opposite of the absolute value of ln(L)'' is equal in this case to ln(L)''.
answered Dec 5 '18 at 17:07
BertrandBertrand
813
add a comment |
The minus sign comes from the fact that at the maximum of ln(L), the second order condition requires that necessarily ln(L)''<=0. And the opposite of the absolute value of ln(L)'' is equal in this case to ln(L)''.
answered Dec 5 '18 at 17:07
BertrandBertrand
813
The minus sign comes from the fact that at the maximum of ln(L), the second order condition requires that necessarily ln(L)''<=0. And the opposite of the absolute value of ln(L)'' is equal in this case to ln(L)''.
answered Dec 5 '18 at 17:07
BertrandBertrand
813
edited Dec 5 '18 at 17:13
answered Dec 5 '18 at 17:07
BertrandBertrand
813
answered Dec 5 '18 at 17:07
BertrandBertrand
813
answered Dec 5 '18 at 17:07
BertrandBertrand
813
813
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3027264%2ftaylor-expansion-of-likelihood-function%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Where does the quote come from by the way?
– epimorphic
Dec 5 '18 at 16:29
1
“Métodos Estatísticos em Física Experimental, Oguri V, 2014” (Portuguese)
– Fred
Dec 5 '18 at 16:36