Perf stack trace is missing __libc_start_main for some processes
up vote
1
down vote
favorite
I have a C++ program compiled with -fno-omit-frame-pointer
. I run the same program in two processes and use perf record -p 24509,24470 -F 977 --call-graph fp -- sleep 120
then perf script
to get stack traces.
What I found is that all stack traces from 24509 have main | __libc_start_main
at the bottom but stack traces from 24470 only have main
.
My question is why perf stack trace doesn't have __libc_start_main for one process but have it for the other process even though they are running the same program.
My perf version is 3.10.0-514.36.5.el7.x86_64.debug
.
linux x86-64 glibc perf backtrace
|
show 2 more comments
up vote
1
down vote
favorite
I have a C++ program compiled with -fno-omit-frame-pointer
. I run the same program in two processes and use perf record -p 24509,24470 -F 977 --call-graph fp -- sleep 120
then perf script
to get stack traces.
What I found is that all stack traces from 24509 have main | __libc_start_main
at the bottom but stack traces from 24470 only have main
.
My question is why perf stack trace doesn't have __libc_start_main for one process but have it for the other process even though they are running the same program.
My perf version is 3.10.0-514.36.5.el7.x86_64.debug
.
linux x86-64 glibc perf backtrace
Are the two processes both started from the exact same executable, or different builds of it? Are they started with the same options and environment variables? Is this repeatable if you restart the processes and runperf record
again? Or even if you runperf record
again without restarting them?
– Peter Cordes
yesterday
1
Hi @PeterCordes, those two processes are started from the exact same executable. The only difference is that they log to different log files. Also I restarted the process that's missing__libc_start_main
and__libc_start_main
appears after restart. If I don't restart, the process is consistently missing__libc_start_main
.
– jjyao
10 hours ago
If you try repeated restarts, is it sometimes missing__libc_start_main
? Or was that a one-off that you can't repeat at all? Or maybe after running for a long time, something gets overwritten on the stack and the info is lost? And BTW, command line args and the environment are on the stack on process entry, but they're above the stack frames that lead to__libc_start_main
so should have no impact.
– Peter Cordes
9 hours ago
Have you upgraded your systemlibc.so.6
after starting the original process? Maybeperf
is using stack-unwind metadata from the file of that name in the filesystem, but the libc the process still has mapped has different symbol addresses so its.eh_frame
addresses don't match.
– Peter Cordes
9 hours ago
Hi @PeterCordes, you are right. I realized that after running the process for a long time,__libc_start_main
may disappear. That's why I can always see__libc_start_main
if I immediately runperf record
after restarting the process. So my question is why the stack gets overwritten if the process runs for a long time.
– jjyao
6 hours ago
|
show 2 more comments
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have a C++ program compiled with -fno-omit-frame-pointer
. I run the same program in two processes and use perf record -p 24509,24470 -F 977 --call-graph fp -- sleep 120
then perf script
to get stack traces.
What I found is that all stack traces from 24509 have main | __libc_start_main
at the bottom but stack traces from 24470 only have main
.
My question is why perf stack trace doesn't have __libc_start_main for one process but have it for the other process even though they are running the same program.
My perf version is 3.10.0-514.36.5.el7.x86_64.debug
.
linux x86-64 glibc perf backtrace
I have a C++ program compiled with -fno-omit-frame-pointer
. I run the same program in two processes and use perf record -p 24509,24470 -F 977 --call-graph fp -- sleep 120
then perf script
to get stack traces.
What I found is that all stack traces from 24509 have main | __libc_start_main
at the bottom but stack traces from 24470 only have main
.
My question is why perf stack trace doesn't have __libc_start_main for one process but have it for the other process even though they are running the same program.
My perf version is 3.10.0-514.36.5.el7.x86_64.debug
.
linux x86-64 glibc perf backtrace
linux x86-64 glibc perf backtrace
edited yesterday
Peter Cordes
115k16174300
115k16174300
asked yesterday
jjyao
100415
100415
Are the two processes both started from the exact same executable, or different builds of it? Are they started with the same options and environment variables? Is this repeatable if you restart the processes and runperf record
again? Or even if you runperf record
again without restarting them?
– Peter Cordes
yesterday
1
Hi @PeterCordes, those two processes are started from the exact same executable. The only difference is that they log to different log files. Also I restarted the process that's missing__libc_start_main
and__libc_start_main
appears after restart. If I don't restart, the process is consistently missing__libc_start_main
.
– jjyao
10 hours ago
If you try repeated restarts, is it sometimes missing__libc_start_main
? Or was that a one-off that you can't repeat at all? Or maybe after running for a long time, something gets overwritten on the stack and the info is lost? And BTW, command line args and the environment are on the stack on process entry, but they're above the stack frames that lead to__libc_start_main
so should have no impact.
– Peter Cordes
9 hours ago
Have you upgraded your systemlibc.so.6
after starting the original process? Maybeperf
is using stack-unwind metadata from the file of that name in the filesystem, but the libc the process still has mapped has different symbol addresses so its.eh_frame
addresses don't match.
– Peter Cordes
9 hours ago
Hi @PeterCordes, you are right. I realized that after running the process for a long time,__libc_start_main
may disappear. That's why I can always see__libc_start_main
if I immediately runperf record
after restarting the process. So my question is why the stack gets overwritten if the process runs for a long time.
– jjyao
6 hours ago
|
show 2 more comments
Are the two processes both started from the exact same executable, or different builds of it? Are they started with the same options and environment variables? Is this repeatable if you restart the processes and runperf record
again? Or even if you runperf record
again without restarting them?
– Peter Cordes
yesterday
1
Hi @PeterCordes, those two processes are started from the exact same executable. The only difference is that they log to different log files. Also I restarted the process that's missing__libc_start_main
and__libc_start_main
appears after restart. If I don't restart, the process is consistently missing__libc_start_main
.
– jjyao
10 hours ago
If you try repeated restarts, is it sometimes missing__libc_start_main
? Or was that a one-off that you can't repeat at all? Or maybe after running for a long time, something gets overwritten on the stack and the info is lost? And BTW, command line args and the environment are on the stack on process entry, but they're above the stack frames that lead to__libc_start_main
so should have no impact.
– Peter Cordes
9 hours ago
Have you upgraded your systemlibc.so.6
after starting the original process? Maybeperf
is using stack-unwind metadata from the file of that name in the filesystem, but the libc the process still has mapped has different symbol addresses so its.eh_frame
addresses don't match.
– Peter Cordes
9 hours ago
Hi @PeterCordes, you are right. I realized that after running the process for a long time,__libc_start_main
may disappear. That's why I can always see__libc_start_main
if I immediately runperf record
after restarting the process. So my question is why the stack gets overwritten if the process runs for a long time.
– jjyao
6 hours ago
Are the two processes both started from the exact same executable, or different builds of it? Are they started with the same options and environment variables? Is this repeatable if you restart the processes and run
perf record
again? Or even if you run perf record
again without restarting them?– Peter Cordes
yesterday
Are the two processes both started from the exact same executable, or different builds of it? Are they started with the same options and environment variables? Is this repeatable if you restart the processes and run
perf record
again? Or even if you run perf record
again without restarting them?– Peter Cordes
yesterday
1
1
Hi @PeterCordes, those two processes are started from the exact same executable. The only difference is that they log to different log files. Also I restarted the process that's missing
__libc_start_main
and __libc_start_main
appears after restart. If I don't restart, the process is consistently missing __libc_start_main
.– jjyao
10 hours ago
Hi @PeterCordes, those two processes are started from the exact same executable. The only difference is that they log to different log files. Also I restarted the process that's missing
__libc_start_main
and __libc_start_main
appears after restart. If I don't restart, the process is consistently missing __libc_start_main
.– jjyao
10 hours ago
If you try repeated restarts, is it sometimes missing
__libc_start_main
? Or was that a one-off that you can't repeat at all? Or maybe after running for a long time, something gets overwritten on the stack and the info is lost? And BTW, command line args and the environment are on the stack on process entry, but they're above the stack frames that lead to __libc_start_main
so should have no impact.– Peter Cordes
9 hours ago
If you try repeated restarts, is it sometimes missing
__libc_start_main
? Or was that a one-off that you can't repeat at all? Or maybe after running for a long time, something gets overwritten on the stack and the info is lost? And BTW, command line args and the environment are on the stack on process entry, but they're above the stack frames that lead to __libc_start_main
so should have no impact.– Peter Cordes
9 hours ago
Have you upgraded your system
libc.so.6
after starting the original process? Maybe perf
is using stack-unwind metadata from the file of that name in the filesystem, but the libc the process still has mapped has different symbol addresses so its .eh_frame
addresses don't match.– Peter Cordes
9 hours ago
Have you upgraded your system
libc.so.6
after starting the original process? Maybe perf
is using stack-unwind metadata from the file of that name in the filesystem, but the libc the process still has mapped has different symbol addresses so its .eh_frame
addresses don't match.– Peter Cordes
9 hours ago
Hi @PeterCordes, you are right. I realized that after running the process for a long time,
__libc_start_main
may disappear. That's why I can always see __libc_start_main
if I immediately run perf record
after restarting the process. So my question is why the stack gets overwritten if the process runs for a long time.– jjyao
6 hours ago
Hi @PeterCordes, you are right. I realized that after running the process for a long time,
__libc_start_main
may disappear. That's why I can always see __libc_start_main
if I immediately run perf record
after restarting the process. So my question is why the stack gets overwritten if the process runs for a long time.– jjyao
6 hours ago
|
show 2 more comments
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53401691%2fperf-stack-trace-is-missing-libc-start-main-for-some-processes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Are the two processes both started from the exact same executable, or different builds of it? Are they started with the same options and environment variables? Is this repeatable if you restart the processes and run
perf record
again? Or even if you runperf record
again without restarting them?– Peter Cordes
yesterday
1
Hi @PeterCordes, those two processes are started from the exact same executable. The only difference is that they log to different log files. Also I restarted the process that's missing
__libc_start_main
and__libc_start_main
appears after restart. If I don't restart, the process is consistently missing__libc_start_main
.– jjyao
10 hours ago
If you try repeated restarts, is it sometimes missing
__libc_start_main
? Or was that a one-off that you can't repeat at all? Or maybe after running for a long time, something gets overwritten on the stack and the info is lost? And BTW, command line args and the environment are on the stack on process entry, but they're above the stack frames that lead to__libc_start_main
so should have no impact.– Peter Cordes
9 hours ago
Have you upgraded your system
libc.so.6
after starting the original process? Maybeperf
is using stack-unwind metadata from the file of that name in the filesystem, but the libc the process still has mapped has different symbol addresses so its.eh_frame
addresses don't match.– Peter Cordes
9 hours ago
Hi @PeterCordes, you are right. I realized that after running the process for a long time,
__libc_start_main
may disappear. That's why I can always see__libc_start_main
if I immediately runperf record
after restarting the process. So my question is why the stack gets overwritten if the process runs for a long time.– jjyao
6 hours ago