Обсуждение: ci: Improve OpenBSD core dump backtrace handling

Hi,

While working on [1], I realized that the core dump output of OpenBSD
has very little information. We had an off-list chat with Thomas and
found that this happens because OpenBSD core dump files do not include
the executable information; so we need to run `lldb` with the
executable path. Attached patch manually finds executables in the
build directory and calls `lldb` with these executables' paths'.

Here is an example:

Master [2]:
```
[11:36:51.769] (lldb) target create --core "/var/crash/postgres.core"
[11:36:51.797] Core file '/var/crash/postgres.core' (x86_64) was loaded.
[11:36:51.797] (lldb) thread backtrace all
[11:36:51.798] * thread #1, stop reason = signal SIGABRT
[11:36:51.798]   * frame #0: 0x0000029d071e14bb
[11:36:51.798]     frame #1: 0x550327f2b69964ee
[11:36:51.798] (lldb) quit
```

Patched [3]:
```
[11:22:11.087] (lldb) target create
"/home/postgres/postgres/build/tmp_install/usr/local/pgsql/bin/postgres"
--core "/var/crash/postgres.core"
[11:22:13.104] Core file '/var/crash/postgres.core' (x86_64) was loaded.
[11:22:13.104] (lldb) thread backtrace all
[11:22:13.105] warning: This version of LLDB has no plugin for the
language "assembler". Inspection of frame variables will be limited.
[11:22:13.213] * thread #1, stop reason = signal SIGABRT
[11:22:13.213]   * frame #0: 0x00000b12e4dfe27b
libc.so.100.3`_thread_sys_thrkill at -:2
[11:22:13.213]     frame #1: 0x00000b12e4db010b
libc.so.100.3`_libc_abort at abort.c:51:8
[11:22:13.213]     frame #2: 0x00000b126d3fd39d
pg_prewarm.so`pg_prewarm(fcinfo=<unavailable>) at pg_prewarm.c:80:2
[11:22:13.213]     frame #3: 0x00000b0ff584add5
postgres`ExecInterpExpr(state=0x00000b12527e4790,
econtext=0x00000b12527e4438, isnull=0x0000000000000000) at
execExprInterp.c:926:8
[11:22:13.213]     frame #4: 0x00000b0ff5896a47 postgres`ExecResult
[inlined] ExecEvalExprNoReturn(state=0x00000b12527e4790,
econtext=0x00000b12527e4438) at executor.h:420:13
[11:22:13.213]     frame #5: 0x00000b0ff5896a3b postgres`ExecResult
[inlined] ExecEvalExprNoReturnSwitchContext(state=0x00000b12527e4790,
econtext=0x00000b12527e4438) at executor.h:461:2
[11:22:13.213]     frame #6: 0x00000b0ff5896a2a postgres`ExecResult
[inlined] ExecProject(projInfo=0x00000b12527e4788) at executor.h:493:2
[11:22:13.213]     frame #7: 0x00000b0ff5896a11
postgres`ExecResult(pstate=<unavailable>) at nodeResult.c:135:10
[11:22:13.213]     frame #8: 0x00000b0ff5855eda
postgres`standard_ExecutorRun [inlined]
ExecProcNode(node=0x00000b12527e4328) at executor.h:316:9
[11:22:13.213]     frame #9: 0x00000b0ff5855ec5
postgres`standard_ExecutorRun [inlined]
ExecutePlan(queryDesc=0x00000b124f332500, operation=CMD_SELECT,
sendTuples=<unavailable>, numberTuples=0, direction=<unavailable>,
dest=0x00000b12e5cae7a0) at execMain.c:1697:10
[11:22:13.213]     frame #10: 0x00000b0ff5855e4a
postgres`standard_ExecutorRun(queryDesc=0x00000b124f332500,
direction=<unavailable>, count=0) at execMain.c:366:3
[11:22:13.213]     frame #11: 0x00000b0ff5a7623b
postgres`PortalRunSelect(portal=0x00000b128d07c100,
forward=<unavailable>, count=0, dest=<unavailable>) at pquery.c:921:4
[11:22:13.213]     frame #12: 0x00000b0ff5a75d9d
postgres`PortalRun(portal=0x00000b128d07c100,
count=9223372036854775807, isTopLevel=<unavailable>,
dest=0x00000b12e5cae7a0, altdest=0x00000b12e5cae7a0,
qc=0x0000759da39141b0) at pquery.c:765:18
[11:22:13.213]     frame #13: 0x00000b0ff5a74ad9
postgres`exec_simple_query(query_string="SELECT pg_prewarm('test',
'read');") at postgres.c:1279:10
[11:22:13.213]     frame #14: 0x00000b0ff5a72332
postgres`PostgresMain(dbname=<unavailable>, username=<unavailable>) at
postgres.c:0
[11:22:13.213]     frame #15: 0x00000b0ff5a6bfad
postgres`BackendMain(startup_data=<unavailable>,
startup_data_len=<unavailable>) at backend_startup.c:124:2
[11:22:13.213]     frame #16: 0x00000b0ff59a35a6
postgres`postmaster_child_launch(child_type=B_BACKEND, child_slot=1,
startup_data=0x0000759da3915030, startup_data_len=24,
client_sock=0x0000759da3914f20) at launch_backend.c:268:3
[11:22:13.213]     frame #17: 0x00000b0ff59a8584 postgres`ServerLoop
[inlined] BackendStartup(client_sock=<unavailable>) at
postmaster.c:3598:8
[11:22:13.213]     frame #18: 0x00000b0ff59a8543 postgres`ServerLoop
at postmaster.c:1713:6
[11:22:13.213]     frame #19: 0x00000b0ff59a60e8
postgres`PostmasterMain(argc=<unavailable>, argv=<unavailable>) at
postmaster.c:1403:11
[11:22:13.213]     frame #20: 0x00000b0ff58c6915 postgres`main(argc=4,
argv=0x0000759da39155c8) at main.c:231:4
[11:22:13.213]     frame #21: 0x00000b0ff565728b postgres`__start + 299
```

Any feedback would be appreciated.

[1] https://postgr.es/m/CAN55FZ12R-LG=wZZ8yLY7+tmVbR=9Y_O5uveDHU0nmYT+AiTjQ@mail.gmail.com
[2] https://cirrus-ci.com/task/6177664672727040
[3] https://cirrus-ci.com/task/5487875443130368

-- 
Regards,
Nazir Bilal Yavuz
Microsoft

On Sat, Oct 25, 2025 at 2:20 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
> Any feedback would be appreciated.

+        filename=$(basename "$corefile")
+        base=$(echo "$filename" | sed 's/\.core.*$//')
+        binary=$(find "$executable_directory" -type f -name "$base"
2>/dev/null | head -n 1)
+
+        if [ -z "$binary" ]; then
+            echo "${base} executable can not be found in
${executable_directory}"
+            continue
+        fi
+
+        lldb "$binary" -c "$corefile" --batch -o 'thread backtrace
all' -o 'quit'

s/can not/cannot/, I don't make the rules...

Maybe if not found it should just run it the old way without the
executable?  We don't really expect other programs to be dumped (well,
sometimes we abort cp etc, something I plan to fix...), and they're
probably stripped anyway, but I guess we might as well try to show as
much information as we can if it happens?  Would it be better to set
PATH to $executable_directory:$PATH and use "which"?