Getting stacktraces on crashes ============================== :toc: The Compiler <mail@qutebrowser.org> When there is a fatal crash in qutebrowser - most of the times a https://en.wikipedia.org/wiki/Segmentation_fault[segfault] - the crash report usually doesn't contain much viable information, as these crashes usually happen inside of the Qt mainloop in C++. To know what the issue is, a https://en.wikipedia.org/wiki/Stack_trace[stack trace] with https://en.wikipedia.org/wiki/Debug_symbol[debugging symbols] is required. The rest of this guide is quite Linux specific, though there is a <<windows,section for Windows>> at the end. Crashes which can be reproduced ------------------------------- If a crash can be reproduced, packages with debugging symbols should be installed, and the crash should be reproduced under gdb. Getting debugging symbols ~~~~~~~~~~~~~~~~~~~~~~~~~ Debian/Ubuntu/... ^^^^^^^^^^^^^^^^^ For Debian based systems (Debian, Ubuntu, Linux Mint, ...), debug information is available in the repositories: ---- # apt-get install python3-pyqt5-dbg python3-pyqt5.qtwebkit-dbg python3-dbg libqt5webkit5-dbg ---- Archlinux ^^^^^^^^^ For Archlinux, no debug information is provided. You can either compile Qt yourself (which will take a few hours even on a modern machine) or use debugging symbols compiled/packaged by me (x86_64 only). To install my pre-built packages ++++++++++++++++++++++++++++++++ First download and sign the key: ---- # pacman-key -r 0xD6A1C70FE80A0C82 $ pacman-key -f 0xD6A1C70FE80A0C82 Key fingerprint = 14AF EC28 70C6 4863 C5C7 ACCB D6A1 C70F E80A 0C82 # pacman-key --lsign-key 0xD6A1C70FE80A0C82 ---- Then edit your `/etc/pacman.conf` to add the repository to the bottom: ---- [qt-debug] Server = https://qutebrowser.org/qt-debug/$arch ---- Then install the packages: ---- # pacman -Suy pyqt5-common-debug python-pyqt5-debug qt5-base-debug qt5-webkit-debug qt5-webengine-debug ---- The `-debug` packages conflict with the non-debug variants - it's safe to remove them. To compile by yourself ++++++++++++++++++++++ Note that building Qt will likely take multiple hours, even on a recent system. I'd also expect it to take around 6 GB of RAM and 30 GB of disk space for a successful compile run. ---- $ git clone https://github.com/qutebrowser/qt-debug-pkgbuild.git $ cd qt-debug-pkgbuild $ export DEBUG_CFLAGS='-ggdb3 -fvar-tracking-assignments -Og' $ export DEBUG_CXXFLAGS='-ggdb3 -fvar-tracking-assignments -Og' $ cd qt5 $ makepkg -si --pkg qt5-base-debug,qt5-webkit-debug,qt5-webengine-debug $ cd ../pyqt5 $ makepkg -si --pkg pyqt5-common-debug,python-pyqt5-debug ---- Getting the stack trace ~~~~~~~~~~~~~~~~~~~~~~~ First install `gdb` on your system if it's not installed already. Then run qutebrowser directly inside gdb like this: ---- $ gdb $(readlink -f $(which python3)) -ex 'run -m qutebrowser --debug' ---- After you reproduce the crash, you should now see something like: ---- Program received signal SIGSEGV, Segmentation fault. ... (gdb) ---- Now enter these commands at the gdb prompt: ---- (gdb) set logging on (gdb) bt full # you might have to press enter a few times until you get the prompt back (gdb) quit ---- This will create a `gdb.txt` in your current directory. Copy the last few lines of the debug log (before you got the gdb prompt) and the full content of `gdb.txt` into the bug report. Please also add some words about what you were doing (or what pages you visited) before the crash happened. Crashes which can NOT be reproduced ----------------------------------- If you cannot reproduce the problem, you need to check if a coredump got written somewhere. You should not install debug symbols as they won't match the generated coredump. First install `gdb` on your system if it's not installed already. Then check the file `/proc/sys/kernel/core_pattern` on your system. If it does not start with a `|` character (pipe), check if there is a file named `core` or `core.NNNN` in the directory from that file, or in the current directory. If so, execute gdb like this: ---- $ gdb $(readlink -f $(which python3)) /path/to/core ---- If your `/proc/sys/kernel/core_pattern` contains something like `|/usr/lib/systemd/systemd-coredump`, use `coredumpctl` to run gdb: ---- $ coredumpctl gdb $(readlink -f $(which python3)) ---- Getting the stack trace ~~~~~~~~~~~~~~~~~~~~~~~ Now enter these commands at the gdb prompt: ---- (gdb) set logging on (gdb) bt # you might have to press enter a few times until you get the prompt back (gdb) quit ---- Copy the content of `gdb.txt` into the bug report. Please also add some words about what you were doing (or what pages you visited) before the crash happened. [[windows]] For Windows ----------- When you see the _qutebrowser.exe has stopped working_ window, do not click "Close the program". Instead, open your task manager, there right-click on `qutebrowser.exe` and select "Create dump file". Remember the path of the dump file displayed there. Now install https://www.microsoft.com/en-us/download/details.aspx?id=49924[DebugDiag] from Microsoft, then run the *DebugDiag 2 Analysis* tool. There, check *CrashHangAnalysis* and add your crash dump via *Add Data files*. Then click *Start analysis*. Close the Internet Explorer which opens when it's done and use the folder-button at the top left to get to the reports. There find the report file and send it to mail@qutebrowser.org.