lib: Prevent crash then another crash from happening

When a memory operation (malloc/free/... ) causes a crash
and the call to core_handler causes another crash then
instead of actually writing a core dump the alarm is
hit and the daemon in trouble will not cause a core dump.
Modify the shutdown code to just try to dump the buffers
and leave instead of cleaning up after itself.

Back Trace:

(gdb) bt
0  0x00007f17082ec056 in __lll_lock_wait_private () from /lib/x86_64-linux-gnu/libc.so.6
1  0x00007f17082fc8bd in ?? () from /lib/x86_64-linux-gnu/libc.so.6
2  0x00007f17082fee8f in free () from /lib/x86_64-linux-gnu/libc.so.6
3  0x00007f170866c2ea in qfree (mt=<optimized out>, ptr=<optimized out>) at lib/memory.c:141
4  0x00007f17086c156a in zlog_tls_free (arg=0x55584f816fb0) at lib/zlog.c:390
5  zlog_tls_buffer_fini () at lib/zlog.c:346
6  0x00007f1708695e5f in core_handler (signo=11, siginfo=0x7ffd173229f0, context=<optimized out>) at lib/sigevent.c:264
7  <signal handler called>
8  0x00007f17082fd7bc in ?? () from /lib/x86_64-linux-gnu/libc.so.6
9  0x00007f17082ff6e2 in calloc () from /lib/x86_64-linux-gnu/libc.so.6
10 0x00007f1708451e78 in lh_table_new () from /lib/x86_64-linux-gnu/libjson-c.so.5
11 0x00007f170844c979 in json_object_new_object () from /lib/x86_64-linux-gnu/libjson-c.so.5
12 0x000055584e002fd9 in evpn_show_all_routes (vty=vty@entry=0x55584fb5ea00, bgp=bgp@entry=0x55584f82c600, type=<optimized out>, json=json@entry=0x55584f998130, detail=<optimized out>,
    self_orig=<optimized out>) at bgpd/bgp_evpn_vty.c:3192
13 0x000055584e009ed6 in show_bgp_l2vpn_evpn_route (self=<optimized out>, vty=0x55584fb5ea00, argc=6, argv=0x55584f998970) at bgpd/bgp_evpn_vty.c:5048
14 0x00007f170863af60 in cmd_execute_command_real (vline=vline@entry=0x55584fa87cb0, vty=vty@entry=0x55584fb5ea00, cmd=cmd@entry=0x0, up_level=up_level@entry=0, filter=FILTER_RELAXED)
    at lib/command.c:1030
15 0x00007f170863b2be in cmd_execute_command (vline=vline@entry=0x55584fa87cb0, vty=vty@entry=0x55584fb5ea00, cmd=cmd@entry=0x0, vtysh=vtysh@entry=0) at lib/command.c:1089
16 0x00007f170863b550 in cmd_execute (vty=vty@entry=0x55584fb5ea00, cmd=cmd@entry=0x55584fb65160 "sh bgp l2vpn evpn route json", matched=matched@entry=0x0, vtysh=vtysh@entry=0)
    at lib/command.c:1257
17 0x00007f17086acc77 in vty_command (vty=vty@entry=0x55584fb5ea00, buf=0x55584fb65160 "sh bgp l2vpn evpn route json") at lib/vty.c:503
18 0x00007f17086ad444 in vty_execute (vty=vty@entry=0x55584fb5ea00) at lib/vty.c:1266
19 0x00007f17086b06c8 in vtysh_read (thread=<optimized out>) at lib/vty.c:2165
20 0x00007f17086a798d in thread_call (thread=thread@entry=0x7ffd17325ce0) at lib/thread.c:2008
21 0x00007f1708660568 in frr_run (master=0x55584f22a120) at lib/libfrr.c:1223
22 0x000055584dfc8c96 in main (argc=<optimized out>, argv=<optimized out>) at bgpd/bgp_main.c:555

Signed-off-by: Donald Sharp <sharpd@nvidia.com>

diff --git a/lib/sigevent.c b/lib/sigevent.c
index 06f80db4cca8b8cb0f46b722325f43d8b6e7ffc1..3e69f280da2ec4266c58649b9db45521c4164fea 100644 (file)
--- a/lib/sigevent.c
+++ b/lib/sigevent.c
@@ -240,7 +240,17 @@ core_handler(int signo, siginfo_t *siginfo, void *context)
        /* dump memory stats on core */
        log_memstats(stderr, "core_handler");
 
-       zlog_tls_buffer_fini();
+       /*
+        * This is a buffer flush because FRR is going down
+        * hard.  This is especially important if the crash
+        * was caused by a memory operation and if we call
+        * zlog_tls_buffer_fini() then it has memory
+        * operations as well.  This will cause the
+        * core dump to not happen.  BAD MOJO
+        * So this is intentional, let's try to flush
+        * what we can and let the crash happen.
+        */
+       zlog_tls_buffer_flush();
 
        /* give the kernel a chance to generate a coredump */
        sigaddset(&sigset, signo);