Merge branch 'master' into stylesheet

.pylintrc

@@ -40,7 +40,11 @@ disable=no-self-use,
         # https://github.com/PyCQA/pylint/issues/1698
         unsupported-membership-test,
         unsupported-assignment-operation,
-        unsubscriptable-object
+        unsubscriptable-object,
+        too-many-boolean-expressions,
+        too-many-locals,
+        too-many-branches,
+        too-many-statements
 
 [BASIC]
 function-rgx=[a-z_][a-z0-9_]{2,50}$

.travis.yml

@@ -51,7 +51,7 @@ matrix:
       env: TESTENV=eslint
       language: node_js
       python: null
-      node_js: node
+      node_js: "lts/*"
   fast_finish: true
 
 cache:

doc/changelog.asciidoc

@@ -32,6 +32,10 @@ Added
 - New `config.source(...)` method for `config.py` to source another file.
 - New `keyhint.radius` option to configure the edge rounding for the key hint
   widget.
+- `:edit-url` now handles the `--private` and `--related` flags, which have the
+  same effect they have with `:open`.
+- New `{line}` and `{column}` replacements for `editor.command` to position the
+  cursor correctly.
 
 Changed
 ~~~~~~~
@@ -65,10 +69,17 @@ Removed
 v1.0.3 (unreleased)
 -------------------
 
+Changed
+~~~~~~~
+
+- Performance improvements for tab rendering
+
 Fixed
 ~~~~~
 
 - Handle accessing a locked sqlite database gracefully
+- Abort pinned tab dialogs properly when a tab is closed e.g. by closing a
+  window.
 
 v1.0.2
 ------

doc/faq.asciidoc

@@ -152,7 +152,7 @@ For QtWebEngine:
   `:set spellcheck.languages "['en-US', 'pl-PL']"`
 
 How do I use Tor with qutebrowser?::
-    Start tor on your machine, and do `:set network proxy socks://localhost:9050/`
+    Start tor on your machine, and do `:set content.proxy socks://localhost:9050/`
     in qutebrowser. Note this won't give you the same amount of fingerprinting
     protection that the Tor Browser does, but it's useful to be able to access
     `.onion` sites.
@@ -162,7 +162,7 @@ Why does J move to the next (right) tab, and K to the previous (left) one?::
     and qutebrowser's keybindings are designed to be compatible with dwb's.
     The rationale behind it is that J is "down" in vim, and K is "up", which
     corresponds nicely to "next"/"previous". It also makes much more sense with
-    vertical tabs (e.g. `:set tabs position left`).
+    vertical tabs (e.g. `:set tabs.position left`).
 
 What's the difference between insert and passthrough mode?::
     They are quite similar, but insert mode has some bindings (like `Ctrl-e` to

doc/help/commands.asciidoc

@@ -60,6 +60,7 @@ It is possible to run or bind multiple commands by separating them with `;;`.
 |<<messages,messages>>|Show a log of past messages.
 |<<navigate,navigate>>|Open typical prev/next links or navigate using the URL path.
 |<<open,open>>|Open a URL in the current/[count]th tab.
+|<<open-editor,open-editor>>|Open an external editor with the currently selected form field.
 |<<print,print>>|Print the current/[count]th tab.
 |<<quickmark-add,quickmark-add>>|Add a new quickmark.
 |<<quickmark-del,quickmark-del>>|Delete a quickmark.
@@ -361,7 +362,7 @@ The index of the download to retry.
 
 [[edit-url]]
 === edit-url
-Syntax: +:edit-url [*--bg*] [*--tab*] [*--window*] ['url']+
+Syntax: +:edit-url [*--bg*] [*--tab*] [*--window*] [*--private*] [*--related*] ['url']+
 
 Navigate to a url formed in an external editor.
 
@@ -374,6 +375,9 @@ The editor which should be launched can be configured via the `editor.command` c
 * +*-b*+, +*--bg*+: Open in a new background tab.
 * +*-t*+, +*--tab*+: Open in a new tab.
 * +*-w*+, +*--window*+: Open in a new window.
+* +*-p*+, +*--private*+: Open a new window in private browsing mode.
+* +*-r*+, +*--related*+: If opening a new tab, position the tab as related to the current one (like clicking on a link).
+
 
 [[fake-key]]
 === fake-key
@@ -654,6 +658,12 @@ The tab index to open the URL in.
 ==== note
 * This command does not split arguments after the last argument and handles quotes literally.
 
+[[open-editor]]
+=== open-editor
+Open an external editor with the currently selected form field.
+
+The editor which should be launched can be configured via the `editor.command` config option.
+
 [[print]]
 === print
 Syntax: +:print [*--preview*] [*--pdf* 'file']+
@@ -1155,7 +1165,6 @@ How many steps to zoom out.
 |<<move-to-start-of-next-block,move-to-start-of-next-block>>|Move the cursor or selection to the start of next block.
 |<<move-to-start-of-prev-block,move-to-start-of-prev-block>>|Move the cursor or selection to the start of previous block.
 |<<nop,nop>>|Do nothing.
-|<<open-editor,open-editor>>|Open an external editor with the currently selected form field.
 |<<prompt-accept,prompt-accept>>|Accept the current prompt.
 |<<prompt-item-focus,prompt-item-focus>>|Shift the focus of the prompt file completion menu to another item.
 |<<prompt-open-download,prompt-open-download>>|Immediately open a download.
@@ -1406,12 +1415,6 @@ How many blocks to move.
 === nop
 Do nothing.
 
-[[open-editor]]
-=== open-editor
-Open an external editor with the currently selected form field.
-
-The editor which should be launched can be configured via the `editor.command` config option.
-
 [[prompt-accept]]
 === prompt-accept
 Syntax: +:prompt-accept ['value']+

doc/help/settings.asciidoc

@@ -1961,7 +1961,12 @@ Default: +pass:[-1]+
 [[editor.command]]
 === editor.command
 The editor (and arguments) to use for the `open-editor` command.
-`{}` gets replaced by the filename of the file to be edited.
+Several placeholders are supported. Placeholders are substituted by the respective value when executing the command.
+`{file}` gets replaced by the filename of the file to be edited.
+`{line}` gets replaced by the line in which the caret is found in the text.
+`{column}` gets replaced by the column in which the caret is found in the text.
+`{line0}` same as `{line}`, but starting from index 0.
+`{column0}` same as `{column}`, but starting from index 0.
 
 Type: <<types,ShellCommand>>
 
@@ -1969,7 +1974,9 @@ Default:
 
 - +pass:[gvim]+
 - +pass:[-f]+
-- +pass:[{}]+
+- +pass:[{file}]+
+- +pass:[-c]+
+- +pass:[normal {line}G{column0}l]+
 
 [[editor.encoding]]
 === editor.encoding

qutebrowser/app.py

@@ -417,7 +417,6 @@ def _init_modules(args, crash_handler):
         args: The argparse namespace.
         crash_handler: The CrashHandler instance.
     """
-    # pylint: disable=too-many-statements
     log.init.debug("Initializing save manager...")
     save_manager = savemanager.SaveManager(qApp)
     objreg.register('save-manager', save_manager)
@@ -661,9 +660,9 @@ class Quitter:
         try:
             args, cwd = self._get_restart_args(pages, session, override_args)
             if cwd is None:
-                subprocess.Popen(args)
+                subprocess.run(args)
             else:
-                subprocess.Popen(args, cwd=cwd)
+                subprocess.run(args, cwd=cwd)
         except OSError:
             log.destroy.exception("Failed to restart")
             return False

qutebrowser/browser/browsertab.py

@@ -19,6 +19,7 @@
 
 """Base class for a wrapper over QWebView/QWebEngineView."""
 
+import enum
 import itertools
 
 import attr
@@ -74,7 +75,7 @@ class UnsupportedOperationError(WebTabError):
     """Raised when an operation is not supported with the given backend."""
 
 
-TerminationStatus = usertypes.enum('TerminationStatus', [
+TerminationStatus = enum.Enum('TerminationStatus', [
     'normal',
     'abnormal',  # non-zero exit status
     'crashed',   # e.g. segfault

qutebrowser/browser/commands.py

@@ -1618,13 +1618,14 @@ class CommandDispatcher:
             return
         assert isinstance(text, str), text
 
+        caret_position = elem.caret_position()
+
         ed = editor.ExternalEditor(self._tabbed_browser)
         ed.editing_finished.connect(functools.partial(
             self.on_editing_finished, elem))
-        ed.edit(text)
+        ed.edit(text, caret_position)
 
-    @cmdutils.register(instance='command-dispatcher', hide=True,
+    @cmdutils.register(instance='command-dispatcher', scope='window')
-                       scope='window')
     def open_editor(self):
         """Open an external editor with the currently selected form field.
 
@@ -2112,7 +2113,8 @@ class CommandDispatcher:
         self._current_widget().clear_ssl_errors()
 
     @cmdutils.register(instance='command-dispatcher', scope='window')
-    def edit_url(self, url=None, bg=False, tab=False, window=False):
+    def edit_url(self, url=None, bg=False, tab=False, window=False,
+                 private=False, related=False):
         """Navigate to a url formed in an external editor.
 
         The editor which should be launched can be configured via the
@@ -2123,6 +2125,9 @@ class CommandDispatcher:
             bg: Open in a new background tab.
             tab: Open in a new tab.
             window: Open in a new window.
+            private: Open a new window in private browsing mode.
+            related: If opening a new tab, position the tab as related to the
+                     current one (like clicking on a link).
         """
         cmdutils.check_exclusive((tab, bg, window), 'tbw')
 
@@ -2133,7 +2138,7 @@ class CommandDispatcher:
         # Passthrough for openurl args (e.g. -t, -b, -w)
         ed.editing_finished.connect(functools.partial(
             self._open_if_changed, old_url=old_url, bg=bg, tab=tab,
-            window=window))
+            window=window, private=private, related=related))
 
         ed.edit(url or old_url)
 
@@ -2158,7 +2163,7 @@ class CommandDispatcher:
         self._tabbed_browser.jump_mark(key)
 
     def _open_if_changed(self, url=None, old_url=None, bg=False, tab=False,
-                         window=False):
+                         window=False, private=False, related=False):
         """Open a URL unless it's already open in the tab.
 
         Args:
@@ -2167,9 +2172,13 @@ class CommandDispatcher:
             bg: Open in a new background tab.
             tab: Open in a new tab.
             window: Open in a new window.
+            private: Open a new window in private browsing mode.
+            related: If opening a new tab, position the tab as related to the
+                     current one (like clicking on a link).
         """
-        if bg or tab or window or url != old_url:
+        if bg or tab or window or private or related or url != old_url:
-            self.openurl(url=url, bg=bg, tab=tab, window=window)
+            self.openurl(url=url, bg=bg, tab=tab, window=window,
+                         private=private, related=related)
 
     @cmdutils.register(instance='command-dispatcher', scope='window')
     def fullscreen(self, leave=False):

qutebrowser/browser/downloads.py

@@ -27,6 +27,7 @@ import collections
 import functools
 import pathlib
 import tempfile
+import enum
 
 import sip
 from PyQt5.QtCore import (pyqtSlot, pyqtSignal, Qt, QObject, QModelIndex,
@@ -38,8 +39,7 @@ from qutebrowser.utils import (usertypes, standarddir, utils, message, log,
                                qtutils)
 
 
-ModelRole = usertypes.enum('ModelRole', ['item'], start=Qt.UserRole,
+ModelRole = enum.IntEnum('ModelRole', ['item'], start=Qt.UserRole)
-                           is_int=True)
 
 
 # Remember the last used directory

qutebrowser/browser/hints.py

@@ -24,6 +24,7 @@ import functools
 import math
 import re
 import html
+import enum
 from string import ascii_lowercase
 
 import attr
@@ -37,10 +38,9 @@ from qutebrowser.commands import userscripts, cmdexc, cmdutils, runners
 from qutebrowser.utils import usertypes, log, qtutils, message, objreg, utils
 
 
-Target = usertypes.enum('Target', ['normal', 'current', 'tab', 'tab_fg',
+Target = enum.Enum('Target', ['normal', 'current', 'tab', 'tab_fg', 'tab_bg',
-                                   'tab_bg', 'window', 'yank', 'yank_primary',
+                              'window', 'yank', 'yank_primary', 'run', 'fill',
-                                   'run', 'fill', 'hover', 'download',
+                              'hover', 'download', 'userscript', 'spawn'])
-                                   'userscript', 'spawn'])
 
 
 class HintingError(Exception):

qutebrowser/browser/webelem.py

@@ -24,6 +24,7 @@ Module attributes:
     SELECTORS: CSS selectors for different groups of elements.
 """
 
+import enum
 import collections.abc
 
 from PyQt5.QtCore import QUrl, Qt, QEvent, QTimer
@@ -35,7 +36,7 @@ from qutebrowser.mainwindow import mainwindow
 from qutebrowser.utils import log, usertypes, utils, qtutils, objreg
 
 
-Group = usertypes.enum('Group', ['all', 'links', 'images', 'url', 'inputs'])
+Group = enum.Enum('Group', ['all', 'links', 'images', 'url', 'inputs'])
 
 
 SELECTORS = {

qutebrowser/browser/webengine/webengineelem.py

@@ -47,6 +47,7 @@ class WebEngineElement(webelem.AbstractWebElement):
             'class_name': str,
             'rects': list,
             'attributes': dict,
+            'caret_position': int,
         }
         assert set(js_dict.keys()).issubset(js_dict_types.keys())
         for name, typ in js_dict_types.items():
@@ -132,6 +133,10 @@ class WebEngineElement(webelem.AbstractWebElement):
     def set_value(self, value):
         self._js_call('set_value', value)
 
+    def caret_position(self):
+        """Get the text caret position for the current element."""
+        return self._js_dict.get('caret_position', 0)
+
     def insert_text(self, text):
         if not self.is_editable(strict=True):
             raise webelem.Error("Element is not editable!")

qutebrowser/browser/webkit/webkitelem.py

@@ -126,6 +126,14 @@ class WebKitElement(webelem.AbstractWebElement):
             value = javascript.string_escape(value)
             self._elem.evaluateJavaScript("this.value='{}'".format(value))
 
+    def caret_position(self):
+        """Get the text caret position for the current element."""
+        self._check_vanished()
+        pos = self._elem.evaluateJavaScript('this.selectionStart')
+        if pos is None:
+            return 0
+        return int(pos)
+
     def insert_text(self, text):
         self._check_vanished()
         if not self.is_editable(strict=True):

qutebrowser/commands/command.py

@@ -81,8 +81,6 @@ class Command:
                  deprecated=False, no_cmd_split=False,
                  star_args_optional=False, scope='global', backend=None,
                  no_replace_variables=False):
-        # I really don't know how to solve this in a better way, I tried.
-        # pylint: disable=too-many-locals
         if modes is not None and not_modes is not None:
             raise ValueError("Only modes or not_modes can be given!")
         if modes is not None:

qutebrowser/completion/completer.py

@@ -43,7 +43,6 @@ class Completer(QObject):
 
     Attributes:
         _cmd: The statusbar Command object this completer belongs to.
-        _ignore_change: Whether to ignore the next completion update.
         _timer: The timer used to trigger the completion update.
         _last_cursor_pos: The old cursor position so we avoid double completion
                           updates.
@@ -54,7 +53,6 @@ class Completer(QObject):
     def __init__(self, cmd, parent=None):
         super().__init__(parent)
         self._cmd = cmd
-        self._ignore_change = False
         self._timer = QTimer()
         self._timer.setSingleShot(True)
         self._timer.setInterval(0)
@@ -178,13 +176,15 @@ class Completer(QObject):
             text = self._quote(text)
         model = self._model()
         if model.count() == 1 and config.val.completion.quick:
-            # If we only have one item, we want to apply it immediately
+            # If we only have one item, we want to apply it immediately and go
-            # and go on to the next part.
+            # on to the next part, unless we are quick-completing the part
-            self._change_completed_part(text, before, after, immediate=True)
+            # after maxsplit, so that we don't keep offering completions
+            # (see issue #1519)
             if maxsplit is not None and maxsplit < len(before):
-                # If we are quick-completing the part after maxsplit, don't
+                self._change_completed_part(text, before, after)
-                # keep offering completions (see issue #1519)
+            else:
-                self._ignore_change = True
+                self._change_completed_part(text, before, after,
+                                            immediate=True)
         else:
             self._change_completed_part(text, before, after)
 
@@ -219,12 +219,6 @@ class Completer(QObject):
     @pyqtSlot()
     def _update_completion(self):
         """Check if completions are available and activate them."""
-        if self._ignore_change:
-            log.completion.debug("Ignoring completion update because "
-                                 "ignore_change is True.")
-            self._ignore_change = False
-            return
-
         completion = self.parent()
 
         if self._cmd.prefix() != ':':

qutebrowser/config/configdata.yml

@@ -766,11 +766,23 @@ editor.command:
   type:
     name: ShellCommand
     placeholder: true
-  default: ["gvim", "-f", "{}"]
+  default: ["gvim", "-f", "{file}", "-c", "normal {line}G{column0}l"]
   desc: >-
     The editor (and arguments) to use for the `open-editor` command.
 
-    `{}` gets replaced by the filename of the file to be edited.
+    Several placeholders are supported. Placeholders are substituted by the
+    respective value when executing the command.
+
+    `{file}` gets replaced by the filename of the file to be edited.
+
+    `{line}` gets replaced by the line in which the caret is found in the text.
+
+    `{column}` gets replaced by the column in which the caret is found in the text.
+
+    `{line0}` same as `{line}`, but starting from index 0.
+
+    `{column0}` same as `{column}`, but starting from index 0.
+
 
 editor.encoding:
   type: Encoding

qutebrowser/config/configdiff.py

@@ -755,6 +755,6 @@ def get_diff():
     lexer = pygments.lexers.DiffLexer()
     formatter = pygments.formatters.HtmlFormatter(
         full=True, linenos='table',
-        title='Config diff')
+        title='Diffing pre-1.0 default config with pre-1.0 modified config')
     # pylint: enable=no-member
     return pygments.highlight(conf_diff + key_diff, lexer, formatter)

qutebrowser/config/configtypes.py

@@ -1341,9 +1341,12 @@ class ShellCommand(List):
         if not value:
             return value
 
-        if self.placeholder and '{}' not in ' '.join(value):
+        if (self.placeholder and
+                '{}' not in ' '.join(value) and
+                '{file}' not in ' '.join(value)):
             raise configexc.ValidationError(value, "needs to contain a "
-                                            "{}-placeholder.")
+                                            "{}-placeholder or a "
+                                            "{file}-placeholder.")
         return value
 
 

qutebrowser/javascript/.eslintrc.yaml

@@ -2,7 +2,7 @@ env:
     browser: true
 
 parserOptions:
-    ecmaVersion: 3
+    ecmaVersion: 6
 
 extends:
     "eslint:all"
@@ -13,13 +13,9 @@ rules:
      padded-blocks: ["error", "never"]
      space-before-function-paren: ["error", "never"]
      no-underscore-dangle: "off"
-     no-var: "off"
-     vars-on-top: "off"
-     newline-after-var: "off"
      camelcase: "off"
      require-jsdoc: "off"
      func-style: ["error", "declaration"]
-     newline-before-return: "off"
      init-declarations: "off"
      no-plusplus: "off"
      no-extra-parens: off

qutebrowser/javascript/history.js

@@ -21,22 +21,22 @@
 
 window.loadHistory = (function() {
     // Date of last seen item.
-    var lastItemDate = null;
+    let lastItemDate = null;
 
     // Each request for new items includes the time of the last item and an
     // offset. The offset is equal to the number of items from the previous
     // request that had time=nextTime, and causes the next request to skip
     // those items to avoid duplicates.
-    var nextTime = null;
+    let nextTime = null;
-    var nextOffset = 0;
+    let nextOffset = 0;
 
     // The URL to fetch data from.
-    var DATA_URL = "qute://history/data";
+    const DATA_URL = "qute://history/data";
 
     // Various fixed elements
-    var EOF_MESSAGE = document.getElementById("eof");
+    const EOF_MESSAGE = document.getElementById("eof");
-    var LOAD_LINK = document.getElementById("load");
+    const LOAD_LINK = document.getElementById("load");
-    var HIST_CONTAINER = document.getElementById("hist-container");
+    const HIST_CONTAINER = document.getElementById("hist-container");
 
     /**
      * Finds or creates the session table>tbody to which item with given date
@@ -47,17 +47,17 @@ window.loadHistory = (function() {
      */
     function getSessionNode(date) {
         // Find/create table
-        var tableId = ["hist", date.getDate(), date.getMonth(),
+        const tableId = ["hist", date.getDate(), date.getMonth(),
             date.getYear()].join("-");
-        var table = document.getElementById(tableId);
+        let table = document.getElementById(tableId);
         if (table === null) {
             table = document.createElement("table");
             table.id = tableId;
 
             // Caption contains human-readable date
-            var caption = document.createElement("caption");
+            const caption = document.createElement("caption");
             caption.className = "date";
-            var options = {
+            const options = {
                 "weekday": "long",
                 "year": "numeric",
                 "month": "long",
@@ -71,7 +71,7 @@ window.loadHistory = (function() {
         }
 
         // Find/create tbody
-        var tbody = table.lastChild;
+        let tbody = table.lastChild;
         if (tbody.tagName !== "TBODY") {
             tbody = document.createElement("tbody");
             table.appendChild(tbody);
@@ -80,10 +80,10 @@ window.loadHistory = (function() {
         // Create session-separator and new tbody if necessary
         if (tbody.lastChild !== null && lastItemDate !== null &&
                 window.GAP_INTERVAL > 0) {
-            var interval = lastItemDate.getTime() - date.getTime();
+            const interval = lastItemDate.getTime() - date.getTime();
             if (interval > window.GAP_INTERVAL) {
                 // Add session-separator
-                var sessionSeparator = document.createElement("td");
+                const sessionSeparator = document.createElement("td");
                 sessionSeparator.className = "session-separator";
                 sessionSeparator.colSpan = 2;
                 sessionSeparator.innerHTML = "§";
@@ -108,20 +108,20 @@ window.loadHistory = (function() {
      * @returns {Element} the completed tr.
      */
     function makeHistoryRow(itemUrl, itemTitle, itemTime) {
-        var row = document.createElement("tr");
+        const row = document.createElement("tr");
 
-        var title = document.createElement("td");
+        const title = document.createElement("td");
         title.className = "title";
-        var link = document.createElement("a");
+        const link = document.createElement("a");
         link.href = itemUrl;
         link.innerHTML = itemTitle;
-        var host = document.createElement("span");
+        const host = document.createElement("span");
         host.className = "hostname";
         host.innerHTML = link.hostname;
         title.appendChild(link);
         title.appendChild(host);
 
-        var time = document.createElement("td");
+        const time = document.createElement("td");
         time.className = "time";
         time.innerHTML = itemTime;
 
@@ -139,11 +139,11 @@ window.loadHistory = (function() {
      * @returns {void}
      */
     function getJSON(url, callback) {
-        var xhr = new XMLHttpRequest();
+        const xhr = new XMLHttpRequest();
         xhr.open("GET", url, true);
         xhr.responseType = "json";
-        xhr.onload = function() {
+        xhr.onload = () => {
-            var status = xhr.status;
+            const status = xhr.status;
             callback(status, xhr.response);
         };
         xhr.send();
@@ -172,10 +172,10 @@ window.loadHistory = (function() {
         nextTime = history[history.length - 1].time;
         nextOffset = 0;
 
-        for (var i = 0, len = history.length; i < len; i++) {
+        for (let i = 0, len = history.length; i < len; i++) {
-            var item = history[i];
+            const item = history[i];
             // python's time.time returns seconds, but js Date expects ms
-            var currentItemDate = new Date(item.time * 1000);
+            const currentItemDate = new Date(item.time * 1000);
             getSessionNode(currentItemDate).appendChild(makeHistoryRow(
                 item.url, item.title, currentItemDate.toLocaleTimeString()
             ));
@@ -191,7 +191,7 @@ window.loadHistory = (function() {
      * @return {void}
      */
     function loadHistory() {
-        var url = DATA_URL.concat("?offset=", nextOffset.toString());
+        let url = DATA_URL.concat("?offset=", nextOffset.toString());
         if (nextTime === null) {
             getJSON(url, receiveHistory);
         } else {

qutebrowser/javascript/position_caret.js

@@ -27,16 +27,15 @@
  */
 
 "use strict";
-
 (function() {
     // FIXME:qtwebengine integrate this with other window._qutebrowser code?
     function isElementInViewport(node) {  // eslint-disable-line complexity
-        var i;
+        let i;
-        var boundingRect = (node.getClientRects()[0] ||
+        let boundingRect = (node.getClientRects()[0] ||
                             node.getBoundingClientRect());
 
         if (boundingRect.width <= 1 && boundingRect.height <= 1) {
-            var rects = node.getClientRects();
+            const rects = node.getClientRects();
             for (i = 0; i < rects.length; i++) {
                 if (rects[i].width > rects[0].height &&
                         rects[i].height > rects[0].height) {
@@ -51,8 +50,8 @@
             return null;
         }
         if (boundingRect.width <= 1 || boundingRect.height <= 1) {
-            var children = node.children;
+            const children = node.children;
-            var visibleChildNode = false;
+            let visibleChildNode = false;
             for (i = 0; i < children.length; ++i) {
                 boundingRect = (children[i].getClientRects()[0] ||
                                 children[i].getBoundingClientRect());
@@ -69,7 +68,7 @@
                 boundingRect.left + boundingRect.width < -10) {
             return null;
         }
-        var computedStyle = window.getComputedStyle(node, null);
+        const computedStyle = window.getComputedStyle(node, null);
         if (computedStyle.visibility !== "visible" ||
                 computedStyle.display === "none" ||
                 node.hasAttribute("disabled") ||
@@ -81,27 +80,27 @@
     }
 
     function positionCaret() {
-        var walker = document.createTreeWalker(document.body, 4, null);
+        const walker = document.createTreeWalker(document.body, 4, null);
-        var node;
+        let node;
-        var textNodes = [];
+        const textNodes = [];
-        var el;
+        let el;
         while ((node = walker.nextNode())) {
             if (node.nodeType === 3 && node.data.trim() !== "") {
                 textNodes.push(node);
             }
         }
-        for (var i = 0; i < textNodes.length; i++) {
+        for (let i = 0; i < textNodes.length; i++) {
-            var element = textNodes[i].parentElement;
+            const element = textNodes[i].parentElement;
             if (isElementInViewport(element.parentElement)) {
                 el = element;
                 break;
             }
         }
         if (el !== undefined) {
-            var range = document.createRange();
+            const range = document.createRange();
             range.setStart(el, 0);
             range.setEnd(el, 0);
-            var sel = window.getSelection();
+            const sel = window.getSelection();
             sel.removeAllRanges();
             sel.addRange(range);
         }

qutebrowser/javascript/scroll.js

@@ -20,19 +20,19 @@
 "use strict";
 
 window._qutebrowser.scroll = (function() {
-    var funcs = {};
+    const funcs = {};
 
-    funcs.to_perc = function(x, y) {
+    funcs.to_perc = (x, y) => {
-        var x_px = window.scrollX;
+        let x_px = window.scrollX;
-        var y_px = window.scrollY;
+        let y_px = window.scrollY;
 
-        var width = Math.max(
+        const width = Math.max(
             document.body.scrollWidth,
             document.body.offsetWidth,
             document.documentElement.scrollWidth,
             document.documentElement.offsetWidth
         );
-        var height = Math.max(
+        const height = Math.max(
             document.body.scrollHeight,
             document.body.offsetHeight,
             document.documentElement.scrollHeight,
@@ -65,9 +65,9 @@ window._qutebrowser.scroll = (function() {
         window.scroll(x_px, y_px);
     };
 
-    funcs.delta_page = function(x, y) {
+    funcs.delta_page = (x, y) => {
-        var dx = window.innerWidth * x;
+        const dx = window.innerWidth * x;
-        var dy = window.innerHeight * y;
+        const dy = window.innerHeight * y;
         window.scrollBy(dx, dy);
     };
 

qutebrowser/javascript/webelem.js

@@ -37,22 +37,37 @@
 "use strict";
 
 window._qutebrowser.webelem = (function() {
-    var funcs = {};
+    const funcs = {};
-    var elements = [];
+    const elements = [];
 
     function serialize_elem(elem) {
         if (!elem) {
             return null;
         }
 
-        var id = elements.length;
+        const id = elements.length;
         elements[id] = elem;
 
-        var out = {
+        // InvalidStateError will be thrown if elem doesn't have selectionStart
+        let caret_position = 0;
+        try {
+            caret_position = elem.selectionStart;
+        } catch (err) {
+            if (err instanceof DOMException &&
+                    err.name === "InvalidStateError") {
+                // nothing to do, caret_position is already 0
+            } else {
+                // not the droid we're looking for
+                throw err;
+            }
+        }
+
+        const out = {
             "id": id,
             "value": elem.value,
             "outer_xml": elem.outerHTML,
             "rects": [],  // Gets filled up later
+            "caret_position": caret_position,
         };
 
         // https://github.com/qutebrowser/qutebrowser/issues/2569
@@ -77,16 +92,16 @@ window._qutebrowser.webelem = (function() {
             out.text = elem.text;
         }  // else: don't add the text at all
 
-        var attributes = {};
+        const attributes = {};
-        for (var i = 0; i < elem.attributes.length; ++i) {
+        for (let i = 0; i < elem.attributes.length; ++i) {
-            var attr = elem.attributes[i];
+            const attr = elem.attributes[i];
             attributes[attr.name] = attr.value;
         }
         out.attributes = attributes;
 
-        var client_rects = elem.getClientRects();
+        const client_rects = elem.getClientRects();
-        for (var k = 0; k < client_rects.length; ++k) {
+        for (let k = 0; k < client_rects.length; ++k) {
-            var rect = client_rects[k];
+            const rect = client_rects[k];
             out.rects.push({
                 "top": rect.top,
                 "right": rect.right,
@@ -111,8 +126,8 @@ window._qutebrowser.webelem = (function() {
         // the cVim implementation here?
         // https://github.com/1995eaton/chromium-vim/blob/1.2.85/content_scripts/dom.js#L74-L134
 
-        var win = elem.ownerDocument.defaultView;
+        const win = elem.ownerDocument.defaultView;
-        var rect = elem.getBoundingClientRect();
+        let rect = elem.getBoundingClientRect();
 
         if (!rect ||
                 rect.top > window.innerHeight ||
@@ -127,7 +142,7 @@ window._qutebrowser.webelem = (function() {
             return false;
         }
 
-        var style = win.getComputedStyle(elem, null);
+        const style = win.getComputedStyle(elem, null);
         if (style.getPropertyValue("visibility") !== "visible" ||
                 style.getPropertyValue("display") === "none" ||
                 style.getPropertyValue("opacity") === "0") {
@@ -144,11 +159,11 @@ window._qutebrowser.webelem = (function() {
         return true;
     }
 
-    funcs.find_css = function(selector, only_visible) {
+    funcs.find_css = (selector, only_visible) => {
-        var elems = document.querySelectorAll(selector);
+        const elems = document.querySelectorAll(selector);
-        var out = [];
+        const out = [];
 
-        for (var i = 0; i < elems.length; ++i) {
+        for (let i = 0; i < elems.length; ++i) {
             if (!only_visible || is_visible(elems[i])) {
                 out.push(serialize_elem(elems[i]));
             }
@@ -157,13 +172,13 @@ window._qutebrowser.webelem = (function() {
         return out;
     };
 
-    funcs.find_id = function(id) {
+    funcs.find_id = (id) => {
-        var elem = document.getElementById(id);
+        const elem = document.getElementById(id);
         return serialize_elem(elem);
     };
 
-    funcs.find_focused = function() {
+    funcs.find_focused = () => {
-        var elem = document.activeElement;
+        const elem = document.activeElement;
 
         if (!elem || elem === document.body) {
             // "When there is no selection, the active element is the page's
@@ -174,43 +189,43 @@ window._qutebrowser.webelem = (function() {
         return serialize_elem(elem);
     };
 
-    funcs.find_at_pos = function(x, y) {
+    funcs.find_at_pos = (x, y) => {
         // FIXME:qtwebengine
         // If the element at the specified point belongs to another document
         // (for example, an iframe's subdocument), the subdocument's parent
         // element is returned (the iframe itself).
 
-        var elem = document.elementFromPoint(x, y);
+        const elem = document.elementFromPoint(x, y);
         return serialize_elem(elem);
     };
 
     // Function for returning a selection to python (so we can click it)
-    funcs.find_selected_link = function() {
+    funcs.find_selected_link = () => {
-        var elem = window.getSelection().anchorNode;
+        const elem = window.getSelection().anchorNode;
         if (!elem) {
             return null;
         }
         return serialize_elem(elem.parentNode);
     };
 
-    funcs.set_value = function(id, value) {
+    funcs.set_value = (id, value) => {
         elements[id].value = value;
     };
 
-    funcs.insert_text = function(id, text) {
+    funcs.insert_text = (id, text) => {
-        var elem = elements[id];
+        const elem = elements[id];
         elem.focus();
         document.execCommand("insertText", false, text);
     };
 
-    funcs.set_attribute = function(id, name, value) {
+    funcs.set_attribute = (id, name, value) => {
         elements[id].setAttribute(name, value);
     };
 
-    funcs.remove_blank_target = function(id) {
+    funcs.remove_blank_target = (id) => {
-        var elem = elements[id];
+        let elem = elements[id];
         while (elem !== null) {
-            var tag = elem.tagName.toLowerCase();
+            const tag = elem.tagName.toLowerCase();
             if (tag === "a" || tag === "area") {
                 if (elem.getAttribute("target") === "_blank") {
                     elem.setAttribute("target", "_top");
@@ -221,18 +236,18 @@ window._qutebrowser.webelem = (function() {
         }
     };
 
-    funcs.click = function(id) {
+    funcs.click = (id) => {
-        var elem = elements[id];
+        const elem = elements[id];
         elem.click();
     };
 
-    funcs.focus = function(id) {
+    funcs.focus = (id) => {
-        var elem = elements[id];
+        const elem = elements[id];
         elem.focus();
     };
 
-    funcs.move_cursor_to_end = function(id) {
+    funcs.move_cursor_to_end = (id) => {
-        var elem = elements[id];
+        const elem = elements[id];
         elem.selectionStart = elem.value.length;
         elem.selectionEnd = elem.value.length;
     };

qutebrowser/keyinput/basekeyparser.py

@@ -19,6 +19,7 @@
 
 """Base class for vim-like key sequence parser."""
 
+import enum
 import re
 import unicodedata
 
@@ -75,8 +76,8 @@ class BaseKeyParser(QObject):
     do_log = True
     passthrough = False
 
-    Match = usertypes.enum('Match', ['partial', 'definitive', 'other', 'none'])
+    Match = enum.Enum('Match', ['partial', 'definitive', 'other', 'none'])
-    Type = usertypes.enum('Type', ['chain', 'special'])
+    Type = enum.Enum('Type', ['chain', 'special'])
 
     def __init__(self, win_id, parent=None, supports_count=None,
                  supports_chains=False):

qutebrowser/keyinput/modeparsers.py

@@ -24,6 +24,7 @@ Module attributes:
 """
 
 import traceback
+import enum
 
 from PyQt5.QtCore import pyqtSlot, Qt
 
@@ -34,7 +35,7 @@ from qutebrowser.utils import usertypes, log, message, objreg, utils
 
 
 STARTCHARS = ":/?"
-LastPress = usertypes.enum('LastPress', ['none', 'filtertext', 'keystring'])
+LastPress = enum.Enum('LastPress', ['none', 'filtertext', 'keystring'])
 
 
 class NormalKeyParser(keyparser.CommandKeyParser):

qutebrowser/mainwindow/statusbar/bar.py

@@ -19,6 +19,7 @@
 
 """The main statusbar widget."""
 
+import enum
 import attr
 from PyQt5.QtCore import pyqtSignal, pyqtSlot, pyqtProperty, Qt, QSize, QTimer
 from PyQt5.QtWidgets import QWidget, QHBoxLayout, QStackedLayout, QSizePolicy
@@ -46,7 +47,7 @@ class ColorFlags:
         passthrough: If we're currently in passthrough-mode.
     """
 
-    CaretMode = usertypes.enum('CaretMode', ['off', 'on', 'selection'])
+    CaretMode = enum.Enum('CaretMode', ['off', 'on', 'selection'])
     prompt = attr.ib(False)
     insert = attr.ib(False)
     command = attr.ib(False)

qutebrowser/mainwindow/statusbar/text.py

@@ -19,10 +19,12 @@
 
 """Text displayed in the statusbar."""
 
+import enum
+
 from PyQt5.QtCore import pyqtSlot
 
 from qutebrowser.mainwindow.statusbar import textbase
-from qutebrowser.utils import usertypes, log
+from qutebrowser.utils import log
 
 
 class Text(textbase.TextBase):
@@ -37,7 +39,7 @@ class Text(textbase.TextBase):
         available. If not, the permanent text is shown.
     """
 
-    Text = usertypes.enum('Text', ['normal', 'temp'])
+    Text = enum.Enum('Text', ['normal', 'temp'])
 
     def __init__(self, parent=None):
         super().__init__(parent)

qutebrowser/mainwindow/statusbar/url.py

@@ -19,6 +19,8 @@
 
 """URL displayed in the statusbar."""
 
+import enum
+
 from PyQt5.QtCore import pyqtSlot, pyqtProperty, Qt, QUrl
 
 from qutebrowser.mainwindow.statusbar import textbase
@@ -27,8 +29,8 @@ from qutebrowser.utils import usertypes, urlutils
 
 
 # Note this has entries for success/error/warn from widgets.webview:LoadStatus
-UrlType = usertypes.enum('UrlType', ['success', 'success_https', 'error',
+UrlType = enum.Enum('UrlType', ['success', 'success_https', 'error', 'warn',
-                                     'warn', 'hover', 'normal'])
+                                'hover', 'normal'])
 
 
 class UrlText(textbase.TextBase):

qutebrowser/mainwindow/tabbedbrowser.py

@@ -259,13 +259,14 @@ class TabbedBrowser(tabwidget.TabWidget):
     def tab_close_prompt_if_pinned(self, tab, force, yes_action):
         """Helper method for tab_close.
 
-        If tab is pinned, prompt. If everything is good, run yes_action.
+        If tab is pinned, prompt. If not, run yes_action.
+        If tab is destroyed, abort question.
         """
         if tab.data.pinned and not force:
             message.confirm_async(
                 title='Pinned Tab',
                 text="Are you sure you want to close a pinned tab?",
-                yes_action=yes_action, default=False)
+                yes_action=yes_action, default=False, abort_on=[tab.destroyed])
         else:
             yes_action()
 

qutebrowser/mainwindow/tabwidget.py

@@ -20,6 +20,7 @@
 """The tab widget used for TabbedBrowser from browser.py."""
 
 import functools
+import enum
 
 import attr
 from PyQt5.QtCore import (pyqtSignal, pyqtSlot, Qt, QSize, QRect, QPoint,
@@ -34,8 +35,8 @@ from qutebrowser.config import config
 from qutebrowser.misc import objects
 
 
-PixelMetrics = usertypes.enum('PixelMetrics', ['icon_padding'],
+PixelMetrics = enum.IntEnum('PixelMetrics', ['icon_padding'],
-                              start=QStyle.PM_CustomBase, is_int=True)
+                            start=QStyle.PM_CustomBase)
 
 
 class TabWidget(QTabWidget):
@@ -336,7 +337,7 @@ class TabBar(QTabBar):
         return self.parent().currentWidget()
 
     @pyqtSlot(str)
-    def _on_config_changed(self, option):
+    def _on_config_changed(self, option: str):
         if option == 'fonts.tabs':
             self._set_font()
         elif option == 'tabs.favicons.scale':
@@ -351,6 +352,12 @@ class TabBar(QTabBar):
         if option.startswith('colors.tabs.'):
             self.update()
 
+        # Clear _minimum_tab_size_hint_helper cache when appropriate
+        if option in ["tabs.indicator_padding",
+                      "tabs.padding",
+                      "tabs.width.indicator"]:
+            self._minimum_tab_size_hint_helper.cache_clear()
+
     def _on_show_switching_delay_changed(self):
         """Set timer interval when tabs.show_switching_delay got changed."""
         self._auto_hide_timer.setInterval(config.val.tabs.show_switching_delay)
@@ -459,7 +466,7 @@ class TabBar(QTabBar):
             return
         super().mousePressEvent(e)
 
-    def minimumTabSizeHint(self, index, ellipsis: bool = True):
+    def minimumTabSizeHint(self, index, ellipsis: bool = True) -> QSize:
         """Set the minimum tab size to indicator/icon/... text.
 
         Args:
@@ -469,38 +476,47 @@ class TabBar(QTabBar):
         Return:
             A QSize of the smallest tab size we can make.
         """
-        text = '\u2026' if ellipsis else self.tabText(index)
+        icon = self.tabIcon(index)
+        extent = self.style().pixelMetric(QStyle.PM_TabBarIconSize, None, self)
+        if icon.isNull():
+            icon_width = 0
+        else:
+            icon_width = icon.actualSize(QSize(extent, extent)).width()
+        return self._minimum_tab_size_hint_helper(self.tabText(index),
+                                                  icon_width,
+                                                  ellipsis)
+
+    @functools.lru_cache(maxsize=2**9)
+    def _minimum_tab_size_hint_helper(self, tab_text: str,
+                                      icon_width: int,
+                                      ellipsis: bool) -> QSize:
+        """Helper function to cache tab results.
+
+        Config values accessed in here should be added to _on_config_changed to
+        ensure cache is flushed when needed.
+        """
+        text = '\u2026' if ellipsis else tab_text
         # Don't ever shorten if text is shorter than the ellipsis
         text_width = min(self.fontMetrics().width(text),
-                         self.fontMetrics().width(self.tabText(index)))
+                         self.fontMetrics().width(tab_text))
-        icon = self.tabIcon(index)
         padding = config.val.tabs.padding
         indicator_padding = config.val.tabs.indicator_padding
         padding_h = padding.left + padding.right
         padding_h += indicator_padding.left + indicator_padding.right
         padding_v = padding.top + padding.bottom
-        if icon.isNull():
-            icon_size = QSize(0, 0)
-        else:
-            extent = self.style().pixelMetric(QStyle.PM_TabBarIconSize, None,
-                                              self)
-            icon_size = icon.actualSize(QSize(extent, extent))
         height = self.fontMetrics().height() + padding_v
-        width = (text_width + icon_size.width() +
+        width = (text_width + icon_width +
                  padding_h + config.val.tabs.width.indicator)
         return QSize(width, height)
 
-    def _tab_total_width_pinned(self):
+    def _pinned_statistics(self) -> (int, int):
-        """Get the current total width of pinned tabs.
+        """Get the number of pinned tabs and the total width of pinned tabs."""
-
+        pinned_list = [idx for idx in range(self.count())
-        This width is calculated assuming no shortening due to ellipsis."""
+                       if self._tab_pinned(idx)]
-        return sum(self.minimumTabSizeHint(idx, ellipsis=False).width()
+        pinned_count = len(pinned_list)
-            for idx in range(self.count())
+        pinned_width = sum(self.minimumTabSizeHint(idx, ellipsis=False).width()
-            if self._tab_pinned(idx))
+                           for idx in pinned_list)
-
+        return (pinned_count, pinned_width)
-    def _pinnedCount(self) -> int:
-        """Get the number of pinned tabs."""
-        return sum(self._tab_pinned(idx) for idx in range(self.count()))
 
     def _tab_pinned(self, index: int) -> bool:
         """Return True if tab is pinned."""
@@ -539,8 +555,8 @@ class TabBar(QTabBar):
             return QSize()
         else:
             pinned = self._tab_pinned(index)
-            no_pinned_count = self.count() - self._pinnedCount()
+            pinned_count, pinned_width = self._pinned_statistics()
-            pinned_width = self._tab_total_width_pinned()
+            no_pinned_count = self.count() - pinned_count
             no_pinned_width = self.width() - pinned_width
 
             if pinned:

qutebrowser/misc/backendproblem.py

@@ -25,6 +25,7 @@ import functools
 import html
 import ctypes
 import ctypes.util
+import enum
 
 import attr
 from PyQt5.QtCore import Qt
@@ -37,10 +38,10 @@ from qutebrowser.utils import usertypes, objreg, version, qtutils, log, utils
 from qutebrowser.misc import objects, msgbox
 
 
-_Result = usertypes.enum(
+_Result = enum.IntEnum(
     '_Result',
     ['quit', 'restart', 'restart_webkit', 'restart_webengine'],
-    is_int=True, start=QDialog.Accepted + 1)
+    start=QDialog.Accepted + 1)
 
 
 @attr.s

qutebrowser/misc/crashdialog.py

@@ -27,6 +27,7 @@ import getpass
 import fnmatch
 import traceback
 import datetime
+import enum
 
 import pkg_resources
 from PyQt5.QtCore import pyqtSlot, Qt, QSize
@@ -35,13 +36,13 @@ from PyQt5.QtWidgets import (QDialog, QLabel, QTextEdit, QPushButton,
                              QDialogButtonBox, QApplication, QMessageBox)
 
 import qutebrowser
-from qutebrowser.utils import version, log, utils, objreg, usertypes
+from qutebrowser.utils import version, log, utils, objreg
 from qutebrowser.misc import (miscwidgets, autoupdate, msgbox, httpclient,
                               pastebin)
 from qutebrowser.config import config, configfiles
 
 
-Result = usertypes.enum('Result', ['restore', 'no_restore'], is_int=True,
+Result = enum.IntEnum('Result', ['restore', 'no_restore'],
                       start=QDialog.Accepted + 1)
 
 

qutebrowser/misc/editor.py

@@ -96,11 +96,12 @@ class ExternalEditor(QObject):
     def on_proc_error(self, _err):
         self._cleanup()
 
-    def edit(self, text):
+    def edit(self, text, caret_position=0):
         """Edit a given text.
 
         Args:
             text: The initial text to edit.
+            caret_position: The position of the caret in the text.
         """
         if self._filename is not None:
             raise ValueError("Already editing a file!")
@@ -121,7 +122,9 @@ class ExternalEditor(QObject):
             return
 
         self._remove_file = True
-        self._start_editor()
+
+        line, column = self._calc_line_and_column(text, caret_position)
+        self._start_editor(line=line, column=column)
 
     def edit_file(self, filename):
         """Edit the file with the given filename."""
@@ -129,13 +132,82 @@ class ExternalEditor(QObject):
         self._remove_file = False
         self._start_editor()
 
-    def _start_editor(self):
+    def _start_editor(self, line=1, column=1):
-        """Start the editor with the file opened as self._filename."""
+        """Start the editor with the file opened as self._filename.
+
+        Args:
+            line: the line number to pass to the editor
+            column: the column number to pass to the editor
+        """
         self._proc = guiprocess.GUIProcess(what='editor', parent=self)
         self._proc.finished.connect(self.on_proc_closed)
         self._proc.error.connect(self.on_proc_error)
         editor = config.val.editor.command
         executable = editor[0]
-        args = [arg.replace('{}', self._filename) for arg in editor[1:]]
+
+        args = [self._sub_placeholder(arg, line, column) for arg in editor[1:]]
         log.procs.debug("Calling \"{}\" with args {}".format(executable, args))
         self._proc.start(executable, args)
+
+    def _calc_line_and_column(self, text, caret_position):
+        r"""Calculate line and column numbers given a text and caret position.
+
+        Both line and column are 1-based indexes, because that's what most
+        editors use as line and column starting index.  By "most" we mean at
+        least vim, nvim, gvim, emacs, atom, sublimetext, notepad++, brackets,
+        visual studio, QtCreator and so on.
+
+        To find the line we just count how many newlines there are before the
+        caret and add 1.
+
+        To find the column we calculate the difference between the caret and
+        the last newline before the caret.
+
+        For example in the text `aaa\nbb|bbb` (| represents the caret):
+        caret_position = 6
+        text[:caret_position] = `aaa\nbb`
+        text[:caret_position].count('\n') = 1
+        caret_position - text[:caret_position].rfind('\n') = 3
+
+        Thus line, column = 2, 3, and the caret is indeed in the second
+        line, third column
+
+        Args:
+            text: the text for which the numbers must be calculated
+            caret_position: the position of the caret in the text
+
+        Return:
+            A (line, column) tuple of (int, int)
+        """
+        line = text[:caret_position].count('\n') + 1
+        column = caret_position - text[:caret_position].rfind('\n')
+        return line, column
+
+    def _sub_placeholder(self, arg, line, column):
+        """Substitute a single placeholder.
+
+        If the `arg` input to this function is a valid placeholder it will
+        be substituted with the appropriate value, otherwise it will be left
+        unchanged.
+
+        Args:
+            arg: an argument of editor.command.
+            line: the previously-calculated line number for the text caret.
+            column: the previously-calculated column number for the text caret.
+
+        Return:
+            The substituted placeholder or the original argument.
+        """
+        replacements = {
+            '{}': self._filename,
+            '{file}': self._filename,
+            '{line}': str(line),
+            '{line0}': str(line-1),
+            '{column}': str(column),
+            '{column0}': str(column-1)
+        }
+
+        for old, new in replacements.items():
+            arg = arg.replace(old, new)
+
+        return arg

qutebrowser/misc/split.py

@@ -57,7 +57,6 @@ class ShellLexer:
 
     def __iter__(self):  # pragma: no mccabe
         """Read a raw token from the input stream."""
-        # pylint: disable=too-many-branches,too-many-statements
         self.reset()
         for nextchar in self.string:
             if self.state == ' ':

qutebrowser/misc/utilcmds.py

@@ -182,9 +182,17 @@ def debug_cache_stats():
     except ImportError:
         pass
 
+    tabbed_browser = objreg.get('tabbed-browser', scope='window',
+                                window='last-focused')
+    # pylint: disable=protected-access
+    tab_bar = tabbed_browser.tabBar()
+    tabbed_browser_info = tab_bar._minimum_tab_size_hint_helper.cache_info()
+    # pylint: enable=protected-access
+
     log.misc.debug('is_valid_prefix: {}'.format(prefix_info))
     log.misc.debug('_render_stylesheet: {}'.format(render_stylesheet_info))
     log.misc.debug('history: {}'.format(history_info))
+    log.misc.debug('tab width cache: {}'.format(tabbed_browser_info))
 
 
 @cmdutils.register(debug=True)

qutebrowser/utils/docutils.py

@@ -24,9 +24,10 @@ import sys
 import inspect
 import os.path
 import collections
+import enum
 
 import qutebrowser
-from qutebrowser.utils import usertypes, log, utils
+from qutebrowser.utils import log, utils
 
 
 def is_git_repo():
@@ -75,7 +76,7 @@ class DocstringParser:
         arg_descs: A dict of argument names to their descriptions
     """
 
-    State = usertypes.enum('State', ['short', 'desc', 'desc_hidden',
+    State = enum.Enum('State', ['short', 'desc', 'desc_hidden',
                                 'arg_start', 'arg_inside', 'misc'])
 
     def __init__(self, func):

qutebrowser/utils/standarddir.py

@@ -24,17 +24,18 @@ import sys
 import shutil
 import os.path
 import contextlib
+import enum
 
 from PyQt5.QtCore import QStandardPaths
 from PyQt5.QtWidgets import QApplication
 
-from qutebrowser.utils import log, debug, usertypes, message, utils
+from qutebrowser.utils import log, debug, message, utils
 
 # The cached locations
 _locations = {}
 
 
-Location = usertypes.enum('Location', ['config', 'auto_config',
+Location = enum.Enum('Location', ['config', 'auto_config',
                                   'data', 'system_data',
                                   'cache', 'download', 'runtime'])
 

qutebrowser/utils/usertypes.py

@@ -25,7 +25,7 @@ Module attributes:
 
 import operator
 import collections.abc
-import enum as pyenum
+import enum
 
 from PyQt5.QtCore import pyqtSignal, pyqtSlot, QObject, QTimer
 
@@ -35,22 +35,6 @@ from qutebrowser.utils import log, qtutils, utils
 _UNSET = object()
 
 
-def enum(name, items, start=1, is_int=False):
-    """Factory for simple enumerations.
-
-    Args:
-        name: Name of the enum
-        items: Iterable of items to be sequentially enumerated.
-        start: The number to use for the first value.
-               We use 1 as default so enum members are always True.
-        is_init: True if the enum should be a Python IntEnum
-    """
-    enums = [(v, i) for (i, v) in enumerate(items, start)]
-    base = pyenum.IntEnum if is_int else pyenum.Enum
-    base = pyenum.unique(base)
-    return base(name, enums)
-
-
 class NeighborList(collections.abc.Sequence):
 
     """A list of items which saves its current position.
@@ -65,7 +49,7 @@ class NeighborList(collections.abc.Sequence):
         _mode: The current mode.
     """
 
-    Modes = enum('Modes', ['edge', 'exception'])
+    Modes = enum.Enum('Modes', ['edge', 'exception'])
 
     def __init__(self, items=None, default=_UNSET, mode=Modes.exception):
         """Constructor.
@@ -221,45 +205,46 @@ class NeighborList(collections.abc.Sequence):
 
 
 # The mode of a Question.
-PromptMode = enum('PromptMode', ['yesno', 'text', 'user_pwd', 'alert',
+PromptMode = enum.Enum('PromptMode', ['yesno', 'text', 'user_pwd', 'alert',
                                       'download'])
 
 
 # Where to open a clicked link.
-ClickTarget = enum('ClickTarget', ['normal', 'tab', 'tab_bg', 'window',
+ClickTarget = enum.Enum('ClickTarget', ['normal', 'tab', 'tab_bg', 'window',
                                         'hover'])
 
 
 # Key input modes
-KeyMode = enum('KeyMode', ['normal', 'hint', 'command', 'yesno', 'prompt',
+KeyMode = enum.Enum('KeyMode', ['normal', 'hint', 'command', 'yesno', 'prompt',
                                 'insert', 'passthrough', 'caret', 'set_mark',
                                 'jump_mark', 'record_macro', 'run_macro'])
 
 
 # Exit statuses for errors. Needs to be an int for sys.exit.
-Exit = enum('Exit', ['ok', 'reserved', 'exception', 'err_ipc', 'err_init',
+Exit = enum.IntEnum('Exit', ['ok', 'reserved', 'exception', 'err_ipc',
-                     'err_config', 'err_key_config'], is_int=True, start=0)
+                             'err_init', 'err_config', 'err_key_config'],
+                    start=0)
 
 
 # Load status of a tab
-LoadStatus = enum('LoadStatus', ['none', 'success', 'success_https', 'error',
+LoadStatus = enum.Enum('LoadStatus', ['none', 'success', 'success_https',
-                                 'warn', 'loading'])
+                                      'error', 'warn', 'loading'])
 
 
 # Backend of a tab
-Backend = enum('Backend', ['QtWebKit', 'QtWebEngine'])
+Backend = enum.Enum('Backend', ['QtWebKit', 'QtWebEngine'])
 
 
 # JS world for QtWebEngine
-JsWorld = enum('JsWorld', ['main', 'application', 'user', 'jseval'])
+JsWorld = enum.Enum('JsWorld', ['main', 'application', 'user', 'jseval'])
 
 
 # Log level of a JS message. This needs to match up with the keys allowed for
 # the content.javascript.log setting.
-JsLogLevel = enum('JsLogLevel', ['unknown', 'info', 'warning', 'error'])
+JsLogLevel = enum.Enum('JsLogLevel', ['unknown', 'info', 'warning', 'error'])
 
 
-MessageLevel = enum('MessageLevel', ['error', 'warning', 'info'])
+MessageLevel = enum.Enum('MessageLevel', ['error', 'warning', 'info'])
 
 
 class Question(QObject):

qutebrowser/utils/utils.py

@@ -882,7 +882,7 @@ def yaml_load(f):
     end = datetime.datetime.now()
 
     delta = (end - start).total_seconds()
-    deadline = 3 if 'CI' in os.environ else 2
+    deadline = 5 if 'CI' in os.environ else 2
     if delta > deadline:  # pragma: no cover
         log.misc.warning(
             "YAML load took unusually long, please report this at "

qutebrowser/utils/version.py

@@ -27,6 +27,7 @@ import platform
 import subprocess
 import importlib
 import collections
+import enum
 import pkg_resources
 
 import attr
@@ -63,7 +64,7 @@ class DistributionInfo:
     pretty = attr.ib()
 
 
-Distribution = usertypes.enum(
+Distribution = enum.Enum(
     'Distribution', ['unknown', 'ubuntu', 'debian', 'void', 'arch',
                      'gentoo', 'fedora', 'opensuse', 'linuxmint', 'manjaro'])
 
@@ -151,12 +152,14 @@ def _git_str_subprocess(gitpath):
         return None
     try:
         # https://stackoverflow.com/questions/21017300/21017394#21017394
-        commit_hash = subprocess.check_output(
+        commit_hash = subprocess.run(
             ['git', 'describe', '--match=NeVeRmAtCh', '--always', '--dirty'],
-            cwd=gitpath).decode('UTF-8').strip()
+            cwd=gitpath, check=True,
-        date = subprocess.check_output(
+            stdout=subprocess.PIPE).stdout.decode('UTF-8').strip()
+        date = subprocess.run(
             ['git', 'show', '-s', '--format=%ci', 'HEAD'],
-            cwd=gitpath).decode('UTF-8').strip()
+            cwd=gitpath, check=True,
+            stdout=subprocess.PIPE).stdout.decode('UTF-8').strip()
         return '{} ({})'.format(commit_hash, date)
     except (subprocess.CalledProcessError, OSError):
         return None

scripts/asciidoc2html.py

@@ -117,7 +117,6 @@ class AsciiDoc:
 
     def _build_website_file(self, root, filename):
         """Build a single website file."""
-        # pylint: disable=too-many-locals,too-many-statements
         src = os.path.join(root, filename)
         src_basename = os.path.basename(src)
         parts = [self._args.website[0]]
@@ -225,7 +224,7 @@ class AsciiDoc:
             return self._args.asciidoc
 
         try:
-            subprocess.call(['asciidoc'], stdout=subprocess.DEVNULL,
+            subprocess.run(['asciidoc'], stdout=subprocess.DEVNULL,
                            stderr=subprocess.DEVNULL)
         except OSError:
             pass
@@ -233,7 +232,7 @@ class AsciiDoc:
             return ['asciidoc']
 
         try:
-            subprocess.call(['asciidoc.py'], stdout=subprocess.DEVNULL,
+            subprocess.run(['asciidoc.py'], stdout=subprocess.DEVNULL,
                            stderr=subprocess.DEVNULL)
         except OSError:
             pass
@@ -259,7 +258,7 @@ class AsciiDoc:
         try:
             env = os.environ.copy()
             env['HOME'] = self._homedir
-            subprocess.check_call(cmdline, env=env)
+            subprocess.run(cmdline, check=True, env=env)
         except (subprocess.CalledProcessError, OSError) as e:
             self._failed = True
             utils.print_col(str(e), 'red')

scripts/dev/build_release.py

@@ -50,7 +50,7 @@ def call_script(name, *args, python=sys.executable):
         python: The python interpreter to use.
     """
     path = os.path.join(os.path.dirname(__file__), os.pardir, name)
-    subprocess.check_call([python, path] + list(args))
+    subprocess.run([python, path] + list(args), check=True)
 
 
 def call_tox(toxenv, *args, python=sys.executable):
@@ -64,9 +64,9 @@ def call_tox(toxenv, *args, python=sys.executable):
     env = os.environ.copy()
     env['PYTHON'] = python
     env['PATH'] = os.environ['PATH'] + os.pathsep + os.path.dirname(python)
-    subprocess.check_call(
+    subprocess.run(
         [sys.executable, '-m', 'tox', '-vv', '-e', toxenv] + list(args),
-        env=env)
+        env=env, check=True)
 
 
 def run_asciidoc2html(args):
@@ -89,8 +89,9 @@ def _maybe_remove(path):
 
 def smoke_test(executable):
     """Try starting the given qutebrowser executable."""
-    subprocess.check_call([executable, '--no-err-windows', '--nowindow',
+    subprocess.run([executable, '--no-err-windows', '--nowindow',
-                           '--temp-basedir', 'about:blank', ':later 500 quit'])
+                    '--temp-basedir', 'about:blank', ':later 500 quit'],
+                   check=True)
 
 
 def patch_mac_app():
@@ -178,7 +179,7 @@ def build_mac():
     utils.print_title("Patching .app")
     patch_mac_app()
     utils.print_title("Building .dmg")
-    subprocess.check_call(['make', '-f', 'scripts/dev/Makefile-dmg'])
+    subprocess.run(['make', '-f', 'scripts/dev/Makefile-dmg'], check=True)
 
     dmg_name = 'qutebrowser-{}.dmg'.format(qutebrowser.__version__)
     os.rename('qutebrowser.dmg', dmg_name)
@@ -187,14 +188,14 @@ def build_mac():
 
     try:
         with tempfile.TemporaryDirectory() as tmpdir:
-            subprocess.check_call(['hdiutil', 'attach', dmg_name,
+            subprocess.run(['hdiutil', 'attach', dmg_name,
-                                   '-mountpoint', tmpdir])
+                            '-mountpoint', tmpdir], check=True)
             try:
                 binary = os.path.join(tmpdir, 'qutebrowser.app', 'Contents',
                                       'MacOS', 'qutebrowser')
                 smoke_test(binary)
             finally:
-                subprocess.call(['hdiutil', 'detach', tmpdir])
+                subprocess.run(['hdiutil', 'detach', tmpdir])
     except PermissionError as e:
         print("Failed to remove tempdir: {}".format(e))
 
@@ -242,13 +243,13 @@ def build_windows():
     patch_windows(out_64)
 
     utils.print_title("Building installers")
-    subprocess.check_call(['makensis.exe',
+    subprocess.run(['makensis.exe',
                     '/DVERSION={}'.format(qutebrowser.__version__),
-                           'misc/qutebrowser.nsi'])
+                    'misc/qutebrowser.nsi'], check=True)
-    subprocess.check_call(['makensis.exe',
+    subprocess.run(['makensis.exe',
                     '/DX64',
                     '/DVERSION={}'.format(qutebrowser.__version__),
-                           'misc/qutebrowser.nsi'])
+                    'misc/qutebrowser.nsi'], check=True)
 
     name_32 = 'qutebrowser-{}-win32.exe'.format(qutebrowser.__version__)
     name_64 = 'qutebrowser-{}-amd64.exe'.format(qutebrowser.__version__)
@@ -292,12 +293,12 @@ def build_sdist():
 
     _maybe_remove('dist')
 
-    subprocess.check_call([sys.executable, 'setup.py', 'sdist'])
+    subprocess.run([sys.executable, 'setup.py', 'sdist'], check=True)
     dist_files = os.listdir(os.path.abspath('dist'))
     assert len(dist_files) == 1
 
     dist_file = os.path.join('dist', dist_files[0])
-    subprocess.check_call(['gpg', '--detach-sign', '-a', dist_file])
+    subprocess.run(['gpg', '--detach-sign', '-a', dist_file], check=True)
 
     tar = tarfile.open(dist_file)
     by_ext = collections.defaultdict(list)
@@ -366,7 +367,7 @@ def github_upload(artifacts, tag):
 def pypi_upload(artifacts):
     """Upload the given artifacts to PyPI using twine."""
     filenames = [a[0] for a in artifacts]
-    subprocess.check_call(['twine', 'upload'] + filenames)
+    subprocess.run(['twine', 'upload'] + filenames, check=True)
 
 
 def main():

scripts/dev/check_coverage.py

@@ -285,8 +285,8 @@ def main_check():
             print(msg.text)
         print()
         filters = ','.join('qutebrowser/' + msg.filename for msg in messages)
-        subprocess.check_call([sys.executable, '-m', 'coverage', 'report',
+        subprocess.run([sys.executable, '-m', 'coverage', 'report',
-                               '--show-missing', '--include', filters])
+                        '--show-missing', '--include', filters], check=True)
         print()
         print("To debug this, run 'tox -e py36-pyqt59-cov' "
               "(or py35-pyqt59-cov) locally and check htmlcov/index.html")
@@ -312,9 +312,9 @@ def main_check_all():
     for test_file, src_file in PERFECT_FILES:
         if test_file is None:
             continue
-        subprocess.check_call(
+        subprocess.run(
             [sys.executable, '-m', 'pytest', '--cov', 'qutebrowser',
-             '--cov-report', 'xml', test_file])
+             '--cov-report', 'xml', test_file], check=True)
         with open('coverage.xml', encoding='utf-8') as f:
             messages = check(f, [(test_file, src_file)])
         os.remove('coverage.xml')

scripts/dev/check_doc_changes.py

@@ -24,7 +24,8 @@ import sys
 import subprocess
 import os
 
-code = subprocess.call(['git', '--no-pager', 'diff', '--exit-code', '--stat'])
+code = subprocess.run(['git', '--no-pager', 'diff',
+                       '--exit-code', '--stat']).returncode
 
 if os.environ.get('TRAVIS_PULL_REQUEST', 'false') != 'false':
     if code != 0:
@@ -42,6 +43,6 @@ if code != 0:
     if 'TRAVIS' in os.environ:
         print()
         print("travis_fold:start:gitdiff")
-        subprocess.call(['git', '--no-pager', 'diff'])
+        subprocess.run(['git', '--no-pager', 'diff'])
         print("travis_fold:end:gitdiff")
 sys.exit(code)

scripts/dev/gen_resources.py

@@ -23,4 +23,4 @@
 import subprocess
 
 with open('qutebrowser/resources.py', 'w', encoding='utf-8') as f:
-    subprocess.check_call(['pyrcc5', 'qutebrowser.rcc'], stdout=f)
+    subprocess.run(['pyrcc5', 'qutebrowser.rcc'], stdout=f, check=True)

scripts/dev/get_coredumpctl_traces.py

@@ -84,7 +84,8 @@ def parse_coredumpctl_line(line):
 def get_info(pid):
     """Get and parse "coredumpctl info" output for the given PID."""
     data = {}
-    output = subprocess.check_output(['coredumpctl', 'info', str(pid)])
+    output = subprocess.run(['coredumpctl', 'info', str(pid)], check=True,
+                            stdout=subprocess.PIPE).stdout
     output = output.decode('utf-8')
     for line in output.split('\n'):
         if not line.strip():
@@ -117,12 +118,12 @@ def dump_infos_gdb(parsed):
     """Dump all needed infos for the given crash using gdb."""
     with tempfile.TemporaryDirectory() as tempdir:
         coredump = os.path.join(tempdir, 'dump')
-        subprocess.check_call(['coredumpctl', 'dump', '-o', coredump,
+        subprocess.run(['coredumpctl', 'dump', '-o', coredump,
-                               str(parsed.pid)])
+                        str(parsed.pid)], check=True)
-        subprocess.check_call(['gdb', parsed.exe, coredump,
+        subprocess.run(['gdb', parsed.exe, coredump,
                         '-ex', 'info threads',
                         '-ex', 'thread apply all bt full',
-                               '-ex', 'quit'])
+                        '-ex', 'quit'], check=True)
 
 
 def dump_infos(parsed):
@@ -143,7 +144,7 @@ def check_prerequisites():
     """Check if coredumpctl/gdb are installed."""
     for binary in ['coredumpctl', 'gdb']:
         try:
-            subprocess.check_call([binary, '--version'])
+            subprocess.run([binary, '--version'], check=True)
         except FileNotFoundError:
             print("{} is needed to run this script!".format(binary),
                   file=sys.stderr)
@@ -158,7 +159,8 @@ def main():
                         action='store_true')
     args = parser.parse_args()
 
-    coredumps = subprocess.check_output(['coredumpctl', 'list'])
+    coredumps = subprocess.run(['coredumpctl', 'list'], check=True,
+                               stdout=subprocess.PIPE).stdout
     lines = coredumps.decode('utf-8').split('\n')
     for line in lines[1:]:
         if not line.strip():

scripts/dev/misc_checks.py

@@ -62,7 +62,8 @@ def check_git():
         print()
         return False
     untracked = []
-    gitst = subprocess.check_output(['git', 'status', '--porcelain'])
+    gitst = subprocess.run(['git', 'status', '--porcelain'], check=True,
+                           stdout=subprocess.PIPE).stdout
     gitst = gitst.decode('UTF-8').strip()
     for line in gitst.splitlines():
         s, name = line.split(maxsplit=1)

scripts/dev/recompile_requirements.py

@@ -116,9 +116,11 @@ def main():
 
         with tempfile.TemporaryDirectory() as tmpdir:
             pip_bin = os.path.join(tmpdir, 'bin', 'pip')
-            subprocess.check_call(['virtualenv', tmpdir])
+            subprocess.run(['virtualenv', tmpdir], check=True)
-            subprocess.check_call([pip_bin, 'install', '-r', filename])
+            subprocess.run([pip_bin, 'install', '-r', filename], check=True)
-            reqs = subprocess.check_output([pip_bin, 'freeze']).decode('utf-8')
+            reqs = subprocess.run([pip_bin, 'freeze'], check=True,
+                                  stdout=subprocess.PIPE
+                                  ).stdout.decode('utf-8')
 
         with open(filename, 'r', encoding='utf-8') as f:
             comments = read_comments(f)

scripts/dev/run_profile.py

@@ -76,14 +76,15 @@ def main():
         pass
     elif args.profile_tool == 'gprof2dot':
         # yep, shell=True. I know what I'm doing.
-        subprocess.call('gprof2dot -f pstats {} | dot -Tpng | feh -F -'.format(
+        subprocess.run(
+            'gprof2dot -f pstats {} | dot -Tpng | feh -F -'.format(
                 shlex.quote(profilefile)), shell=True)
     elif args.profile_tool == 'kcachegrind':
         callgraphfile = os.path.join(tempdir, 'callgraph')
-        subprocess.call(['pyprof2calltree', '-k', '-i', profilefile,
+        subprocess.run(['pyprof2calltree', '-k', '-i', profilefile,
                         '-o', callgraphfile])
     elif args.profile_tool == 'snakeviz':
-        subprocess.call(['snakeviz', profilefile])
+        subprocess.run(['snakeviz', profilefile])
 
     shutil.rmtree(tempdir)
 

scripts/dev/run_pylint_on_tests.py

@@ -73,7 +73,7 @@ def main():
     env = os.environ.copy()
     env['PYTHONPATH'] = os.pathsep.join(pythonpath)
 
-    ret = subprocess.call(['pylint'] + args, env=env)
+    ret = subprocess.run(['pylint'] + args, env=env).returncode
     return ret
 
 

scripts/dev/segfault_test.py

@@ -92,22 +92,22 @@ def main():
         utils.print_bold("==== {} ====".format(page))
         if test_harfbuzz:
             print("With system harfbuzz:")
-        ret = subprocess.call([sys.executable, '-c', SCRIPT, page])
+        ret = subprocess.run([sys.executable, '-c', SCRIPT, page]).returncode
         print_ret(ret)
         retvals.append(ret)
         if test_harfbuzz:
             print("With QT_HARFBUZZ=old:")
             env = dict(os.environ)
             env['QT_HARFBUZZ'] = 'old'
-            ret = subprocess.call([sys.executable, '-c', SCRIPT, page],
+            ret = subprocess.run([sys.executable, '-c', SCRIPT, page],
-                                  env=env)
+                                 env=env).returncode
             print_ret(ret)
             retvals.append(ret)
             print("With QT_HARFBUZZ=new:")
             env = dict(os.environ)
             env['QT_HARFBUZZ'] = 'new'
-            ret = subprocess.call([sys.executable, '-c', SCRIPT, page],
+            ret = subprocess.run([sys.executable, '-c', SCRIPT, page],
-                                  env=env)
+                                 env=env).returncode
             print_ret(ret)
             retvals.append(ret)
     if all(r == 0 for r in retvals):

scripts/dev/src2asciidoc.py

@@ -529,9 +529,9 @@ def regenerate_cheatsheet():
     ]
 
     for filename, x, y in files:
-        subprocess.check_call(['inkscape', '-e', filename, '-b', 'white',
+        subprocess.run(['inkscape', '-e', filename, '-b', 'white',
                         '-w', str(x), '-h', str(y),
-                               'misc/cheatsheet.svg'])
+                        'misc/cheatsheet.svg'], check=True)
 
 
 def main():

scripts/importer.py

@@ -28,50 +28,176 @@ Currently only importing bookmarks from Netscape Bookmark files is supported.
 
 import argparse
 
+browser_default_input_format = {
+    'chromium': 'netscape',
+    'ie': 'netscape',
+    'firefox': 'netscape',
+    'seamonkey': 'netscape',
+    'palemoon': 'netscape'
+}
+
 
 def main():
     args = get_args()
-    if args.browser in ['chromium', 'firefox', 'ie']:
+    bookmark_types = []
-        import_netscape_bookmarks(args.bookmarks, args.bookmark_format)
+    output_format = None
+    input_format = args.input_format
+    if args.search_output:
+        bookmark_types = ['search']
+        if args.oldconfig:
+            output_format = 'oldsearch'
+        else:
+            output_format = 'search'
+    else:
+        if args.bookmark_output:
+            output_format = 'bookmark'
+        elif args.quickmark_output:
+            output_format = 'quickmark'
+        if args.import_bookmarks:
+            bookmark_types.append('bookmark')
+        if args.import_keywords:
+            bookmark_types.append('keyword')
+    if not bookmark_types:
+        bookmark_types = ['bookmark', 'keyword']
+    if not output_format:
+        output_format = 'quickmark'
+    if not input_format:
+        if args.browser:
+            input_format = browser_default_input_format[args.browser]
+        else:
+            #default to netscape
+            input_format = 'netscape'
+
+    import_function = {'netscape': import_netscape_bookmarks}
+    import_function[input_format](args.bookmarks, bookmark_types,
+                                  output_format)
 
 
 def get_args():
     """Get the argparse parser."""
     parser = argparse.ArgumentParser(
         epilog="To import bookmarks from Chromium, Firefox or IE, "
-               "export them to HTML in your browsers bookmark manager. "
+        "export them to HTML in your browsers bookmark manager. ")
-               "By default, this script will output in a quickmarks format.")
+    parser.add_argument(
-    parser.add_argument('browser', help="Which browser? (chromium, firefox)",
+        'browser',
-                        choices=['chromium', 'firefox', 'ie'],
+        help="Which browser? {%(choices)s}",
+        choices=browser_default_input_format.keys(),
+        nargs='?',
         metavar='browser')
-    parser.add_argument('-b', help="Output in bookmark format.",
+    parser.add_argument(
-                        dest='bookmark_format', action='store_true',
+        '-i',
-                        default=False, required=False)
+        '--input-format',
+        help='Which input format? (overrides browser default; "netscape" if '
+        'neither given)',
+        choices=set(browser_default_input_format.values()),
+        required=False)
+    parser.add_argument(
+        '-b',
+        '--bookmark-output',
+        help="Output in bookmark format.",
+        action='store_true',
+        default=False,
+        required=False)
+    parser.add_argument(
+        '-q',
+        '--quickmark-output',
+        help="Output in quickmark format (default).",
+        action='store_true',
+        default=False,
+        required=False)
+    parser.add_argument(
+        '-s',
+        '--search-output',
+        help="Output config.py search engine format (negates -B and -K)",
+        action='store_true',
+        default=False,
+        required=False)
+    parser.add_argument(
+        '--oldconfig',
+        help="Output search engine format for old qutebrowser.conf format",
+        default=False,
+        action='store_true',
+        required=False)
+    parser.add_argument(
+        '-B',
+        '--import-bookmarks',
+        help="Import plain bookmarks (can be combiend with -K)",
+        action='store_true',
+        default=False,
+        required=False)
+    parser.add_argument(
+        '-K',
+        '--import-keywords',
+        help="Import keywords (can be combined with -B)",
+        action='store_true',
+        default=False,
+        required=False)
     parser.add_argument('bookmarks', help="Bookmarks file (html format)")
     args = parser.parse_args()
     return args
 
 
-def import_netscape_bookmarks(bookmarks_file, is_bookmark_format):
+def search_escape(url):
+    """Escape URLs such that preexisting { and } are handled properly.
+
+    Will obviously trash a properly-formatted Qutebrowser URL.
+    """
+    return url.replace('{', '{{').replace('}', '}}')
+
+
+def import_netscape_bookmarks(bookmarks_file, bookmark_types, output_format):
     """Import bookmarks from a NETSCAPE-Bookmark-file v1.
 
-    Generated by Chromium, Firefox, IE and possibly more browsers
+    Generated by Chromium, Firefox, IE and possibly more browsers. Not all
+    export all possible bookmark types:
+        - Firefox mostly works with everything
+        - Chrome doesn't support keywords at all; searches are a separate
+          database
     """
     import bs4
     with open(bookmarks_file, encoding='utf-8') as f:
         soup = bs4.BeautifulSoup(f, 'html.parser')
-
+    bookmark_query = {
-    html_tags = soup.findAll('a')
+        'search': lambda tag: (
-    if is_bookmark_format:
+            (tag.name == 'a') and
-        output_template = '{tag[href]} {tag.string}'
+            ('shortcuturl' in tag.attrs) and
-    else:
+            ('%s' in tag['href'])),
-        output_template = '{tag.string} {tag[href]}'
+        'keyword': lambda tag: (
-
+            (tag.name == 'a') and
+            ('shortcuturl' in tag.attrs) and
+            ('%s' not in tag['href'])),
+        'bookmark': lambda tag: (
+            (tag.name == 'a') and
+            ('shortcuturl' not in tag.attrs) and
+            (tag.string)),
+    }
+    output_template = {
+        'search': {
+            'search':
+            "c.url.searchengines['{tag[shortcuturl]}'] = "
+            "'{tag[href]}' #{tag.string}"
+        },
+        'oldsearch': {
+            'search': '{tag[shortcuturl]} = {tag[href]} #{tag.string}',
+        },
+        'bookmark': {
+            'bookmark': '{tag[href]} {tag.string}',
+            'keyword': '{tag[href]} {tag.string}'
+        },
+        'quickmark': {
+            'bookmark': '{tag.string} {tag[href]}',
+            'keyword': '{tag[shortcuturl]} {tag[href]}'
+        }
+    }
     bookmarks = []
-    for tag in html_tags:
+    for typ in bookmark_types:
+        tags = soup.findAll(bookmark_query[typ])
+        for tag in tags:
+            if typ == 'search':
+                tag['href'] = search_escape(tag['href']).replace('%s', '{}')
             if tag['href'] not in bookmarks:
-            bookmarks.append(output_template.format(tag=tag))
+                bookmarks.append(
-
+                    output_template[output_format][typ].format(tag=tag))
     for bookmark in bookmarks:
         print(bookmark)
 

scripts/link_pyqt.py

@@ -46,12 +46,14 @@ def run_py(executable, *code):
             f.write('\n'.join(code))
         cmd = [executable, filename]
         try:
-            ret = subprocess.check_output(cmd, universal_newlines=True)
+            ret = subprocess.run(cmd, universal_newlines=True, check=True,
+                                 stdout=subprocess.PIPE).stdout
         finally:
             os.remove(filename)
     else:
         cmd = [executable, '-c', '\n'.join(code)]
-        ret = subprocess.check_output(cmd, universal_newlines=True)
+        ret = subprocess.run(cmd, universal_newlines=True, check=True,
+                             stdout=subprocess.PIPE).stdout
     return ret.rstrip()
 
 

scripts/setupcommon.py

@@ -51,12 +51,14 @@ def _git_str():
         return None
     try:
         # https://stackoverflow.com/questions/21017300/21017394#21017394
-        commit_hash = subprocess.check_output(
+        commit_hash = subprocess.run(
             ['git', 'describe', '--match=NeVeRmAtCh', '--always', '--dirty'],
-            cwd=BASEDIR).decode('UTF-8').strip()
+            cwd=BASEDIR, check=True,
-        date = subprocess.check_output(
+            stdout=subprocess.PIPE).stdout.decode('UTF-8').strip()
+        date = subprocess.run(
             ['git', 'show', '-s', '--format=%ci', 'HEAD'],
-            cwd=BASEDIR).decode('UTF-8').strip()
+            cwd=BASEDIR, check=True,
+            stdout=subprocess.PIPE).stdout.decode('UTF-8').strip()
         return '{} ({})'.format(commit_hash, date)
     except (subprocess.CalledProcessError, OSError):
         return None

tests/end2end/features/editor.feature

@@ -2,6 +2,9 @@
 
 Feature: Opening external editors
 
+    Background:
+        Given I have a fresh instance
+
     ## :edit-url
 
     Scenario: Editing a URL
@@ -20,6 +23,16 @@ Feature: Opening external editors
             - data/numbers/1.txt
             - data/numbers/2.txt (active)
 
+    Scenario: Editing a URL with -rt
+        When I set tabs.new_position.related to prev
+        And I open data/numbers/1.txt
+        And I set up a fake editor replacing "1.txt" by "2.txt"
+        And I run :edit-url -rt
+        Then data/numbers/2.txt should be loaded
+        And the following tabs should be open:
+            - data/numbers/2.txt (active)
+            - data/numbers/1.txt
+
     Scenario: Editing a URL with -b
         When I run :tab-only
         And I open data/numbers/1.txt
@@ -49,6 +62,26 @@ Feature: Opening external editors
                 - active: true
                   url: http://localhost:*/data/numbers/2.txt
 
+    Scenario: Editing a URL with -p
+        When I open data/numbers/1.txt in a new tab
+        And I run :tab-only
+        And I set up a fake editor replacing "1.txt" by "2.txt"
+        And I run :edit-url -p
+        Then data/numbers/2.txt should be loaded
+        And the session should look like:
+            windows:
+            - tabs:
+              - active: true
+                history:
+                - active: true
+                  url: http://localhost:*/data/numbers/1.txt
+            - tabs:
+              - active: true
+                history:
+                - active: true
+                  url: http://localhost:*/data/numbers/2.txt
+              private: true
+
     Scenario: Editing a URL with -t and -b
         When I run :edit-url -t -b
         Then the error "Only one of -t/-b/-w can be given!" should be shown

tests/end2end/features/tabs.feature

@@ -713,6 +713,7 @@ Feature: Tab management
         Then the following tabs should be open:
             - data/hello.txt (active)
 
+    @flaky
     Scenario: Double-undo with single tab on tabs.last_close default page
         Given I have a fresh instance
         When I open about:blank

tests/end2end/features/test_qutescheme_bdd.py

@@ -47,10 +47,10 @@ def update_documentation():
         return
 
     try:
-        subprocess.call(['asciidoc'], stdout=subprocess.DEVNULL,
+        subprocess.run(['asciidoc'], stdout=subprocess.DEVNULL,
                        stderr=subprocess.DEVNULL)
     except OSError:
         pytest.skip("Docs outdated and asciidoc unavailable!")
 
     update_script = os.path.join(script_path, 'asciidoc2html.py')
-    subprocess.call([sys.executable, update_script])
+    subprocess.run([sys.executable, update_script])

tests/end2end/test_invocations.py

@@ -259,14 +259,13 @@ def test_command_on_start(request, quteproc_new):
 
 def test_launching_with_python2():
     try:
-        proc = subprocess.Popen(['python2', '-m', 'qutebrowser',
+        proc = subprocess.run(['python2', '-m', 'qutebrowser',
                                '--no-err-windows'], stderr=subprocess.PIPE)
     except FileNotFoundError:
         pytest.skip("python2 not found")
-    _stdout, stderr = proc.communicate()
     assert proc.returncode == 1
     error = "At least Python 3.5 is required to run qutebrowser"
-    assert stderr.decode('ascii').startswith(error)
+    assert proc.stderr.decode('ascii').startswith(error)
 
 
 def test_initial_private_browsing(request, quteproc_new):

tests/unit/browser/webkit/test_webkitelem.py

@@ -53,7 +53,6 @@ def get_webelem(geometry=None, frame=None, *, null=False, style=None,
                         evaluateJavaScript.
         zoom_text_only: Whether zoom.text_only is set in the config
     """
-    # pylint: disable=too-many-locals,too-many-branches
     elem = mock.Mock()
     elem.isNull.return_value = null
     elem.geometry.return_value = geometry

tests/unit/commands/test_argparser.py

@@ -20,15 +20,15 @@
 """Tests for qutebrowser.commands.argparser."""
 
 import inspect
+import enum
 
 import pytest
 from PyQt5.QtCore import QUrl
 
 from qutebrowser.commands import argparser, cmdexc
-from qutebrowser.utils import usertypes
 
 
-Enum = usertypes.enum('Enum', ['foo', 'foo_bar'])
+Enum = enum.Enum('Enum', ['foo', 'foo_bar'])
 
 
 class TestArgumentParser:

tests/unit/commands/test_cmdutils.py

@@ -25,6 +25,7 @@ import sys
 import logging
 import types
 import typing
+import enum
 
 import pytest
 
@@ -243,7 +244,7 @@ class TestRegister:
         else:
             assert pos_args == [('arg', 'arg')]
 
-    Enum = usertypes.enum('Test', ['x', 'y'])
+    Enum = enum.Enum('Test', ['x', 'y'])
 
     @pytest.mark.parametrize('typ, inp, choices, expected', [
         (int, '42', None, 42),

tests/unit/completion/test_completer.py

@@ -274,7 +274,11 @@ def test_on_selection_changed(before, newtxt, after, completer_obj,
     check(True, 2, after_txt, after_pos)
 
     # quick-completing a single item should move the cursor ahead by 1 and add
-    # a trailing space if at the end of the cmd string
+    # a trailing space if at the end of the cmd string, unless the command has
+    # maxsplit < len(before) (such as :open in these tests)
+    if after_txt.startswith(':open'):
+        return
+
     after_pos += 1
     if after_pos > len(after_txt):
         after_txt += ' '
@@ -299,6 +303,11 @@ def test_quickcomplete_flicker(status_command_stub, completer_obj,
     config_stub.val.completion.quick = True
 
     _set_cmd_prompt(status_command_stub, ':open |')
+    completer_obj.schedule_completion_update()
+    assert completion_widget_stub.set_model.called
+    completion_widget_stub.set_model.reset_mock()
+
+    # selecting a completion should not re-set the model
     completer_obj.on_selection_changed('http://example.com')
     completer_obj.schedule_completion_update()
     assert not completion_widget_stub.set_model.called

tests/unit/config/test_configcommands.py

@@ -60,8 +60,9 @@ class TestSet:
     @pytest.mark.parametrize('option, old_value, inp, new_value', [
         ('url.auto_search', 'naive', 'dns', 'dns'),
         # https://github.com/qutebrowser/qutebrowser/issues/2962
-        ('editor.command', ['gvim', '-f', '{}'], '[emacs, "{}"]',
+        ('editor.command',
-         ['emacs', '{}']),
+         ['gvim', '-f', '{file}', '-c', 'normal {line}G{column0}l'],
+         '[emacs, "{}"]', ['emacs', '{}']),
     ])
     def test_set_simple(self, monkeypatch, commands, config_stub,
                         temp, option, old_value, inp, new_value):

tests/unit/config/test_configinit.py

@@ -111,7 +111,6 @@ class TestEarlyInit:
     def test_autoconfig_yml(self, init_patch, config_tmpdir, caplog, args,
                             load_autoconfig, config_py, invalid_yaml):
         """Test interaction between config.py and autoconfig.yml."""
-        # pylint: disable=too-many-locals,too-many-branches
         # Prepare files
         autoconfig_file = config_tmpdir / 'autoconfig.yml'
         config_py_file = config_tmpdir / 'config.py'

tests/unit/config/test_configtypes.py

@@ -1815,9 +1815,12 @@ class TestShellCommand:
 
     @pytest.mark.parametrize('kwargs, val, expected', [
         ({}, '[foobar]', ['foobar']),
-        ({'placeholder': '{}'}, '[foo, "{}", bar]', ['foo', '{}', 'bar']),
+        ({'placeholder': True}, '[foo, "{}", bar]', ['foo', '{}', 'bar']),
-        ({'placeholder': '{}'}, '["foo{}bar"]', ['foo{}bar']),
+        ({'placeholder': True}, '["foo{}bar"]', ['foo{}bar']),
-        ({'placeholder': '{}'}, '[foo, "bar {}"]', ['foo', 'bar {}']),
+        ({'placeholder': True}, '[foo, "bar {}"]', ['foo', 'bar {}']),
+        ({'placeholder': True}, '[f, "{file}", b]', ['f', '{file}', 'b']),
+        ({'placeholder': True}, '["f{file}b"]', ['f{file}b']),
+        ({'placeholder': True}, '[f, "b {file}"]', ['f', 'b {file}']),
     ])
     def test_valid(self, klass, kwargs, val, expected):
         cmd = klass(**kwargs)
@@ -1825,8 +1828,15 @@ class TestShellCommand:
         assert cmd.to_py(expected) == expected
 
     @pytest.mark.parametrize('kwargs, val', [
-        ({'placeholder': '{}'}, '[foo, bar]'),
+        ({'placeholder': True}, '[foo, bar]'),
-        ({'placeholder': '{}'}, '[foo, "{", "}", bar'),
+        ({'placeholder': True}, '[foo, "{", "}", bar'),
+        ({'placeholder': True}, '[foo, bar]'),
+        ({'placeholder': True}, '[foo, "{fi", "le}", bar'),
+
+        # Like valid ones but with wrong placeholder
+        ({'placeholder': True}, '[f, "{wrong}", b]'),
+        ({'placeholder': True}, '["f{wrong}b"]'),
+        ({'placeholder': True}, '[f, "b {wrong}"]'),
     ])
     def test_from_str_invalid(self, klass, kwargs, val):
         with pytest.raises(configexc.ValidationError):

tests/unit/mainwindow/test_tabwidget.py

@@ -52,3 +52,16 @@ class TestTabWidget:
 
         with qtbot.waitExposed(widget):
             widget.show()
+
+    def test_update_tab_titles_benchmark(self, benchmark, widget,
+                                         qtbot, fake_web_tab):
+        """Benchmark for update_tab_titles."""
+        widget.addTab(fake_web_tab(), 'foobar')
+        widget.addTab(fake_web_tab(), 'foobar2')
+        widget.addTab(fake_web_tab(), 'foobar3')
+        widget.addTab(fake_web_tab(), 'foobar4')
+
+        with qtbot.waitExposed(widget):
+            widget.show()
+
+        benchmark(widget._update_tab_titles)

tests/unit/misc/test_checkpyver.py

@@ -36,15 +36,14 @@ TEXT = (r"At least Python 3.5 is required to run qutebrowser, but it's "
 def test_python2():
     """Run checkpyver with python 2."""
     try:
-        proc = subprocess.Popen(
+        proc = subprocess.run(
             ['python2', checkpyver.__file__, '--no-err-windows'],
             stdout=subprocess.PIPE,
             stderr=subprocess.PIPE)
-        stdout, stderr = proc.communicate()
     except FileNotFoundError:
         pytest.skip("python2 not found")
-    assert not stdout
+    assert not proc.stdout
-    stderr = stderr.decode('utf-8')
+    stderr = proc.stderr.decode('utf-8')
     assert re.match(TEXT, stderr), stderr
     assert proc.returncode == 1
 

tests/unit/misc/test_editor.py

@@ -177,3 +177,18 @@ def test_modify(qtbot, editor, initial_text, edited_text):
         editor._proc.finished.emit(0, QProcess.NormalExit)
 
     assert blocker.args == [edited_text]
+
+
+@pytest.mark.parametrize('text, caret_position, result', [
+    ('', 0, (1, 1)),
+    ('a', 0, (1, 1)),
+    ('a\nb', 1, (1, 2)),
+    ('a\nb', 2, (2, 1)),
+    ('a\nb', 3, (2, 2)),
+    ('a\nbb\nccc', 4, (2, 3)),
+    ('a\nbb\nccc', 5, (3, 1)),
+    ('a\nbb\nccc', 8, (3, 4)),
+])
+def test_calculation(editor, text, caret_position, result):
+    """Test calculation for line and column given text and caret_position."""
+    assert editor._calc_line_and_column(text, caret_position) == result

tests/unit/utils/test_standarddir.py

@@ -555,8 +555,9 @@ def test_no_qapplication(qapp, tmpdir):
     pyfile = tmpdir / 'sub.py'
     pyfile.write_text(textwrap.dedent(sub_code), encoding='ascii')
 
-    output = subprocess.check_output([sys.executable, str(pyfile)] + sys.path,
+    output = subprocess.run([sys.executable, str(pyfile)] + sys.path,
-                                     universal_newlines=True)
+                            universal_newlines=True,
+                            check=True, stdout=subprocess.PIPE).stdout
     sub_locations = json.loads(output)
 
     standarddir._init_dirs()

tests/unit/utils/test_version.py

@@ -299,8 +299,8 @@ class TestGitStr:
 def _has_git():
     """Check if git is installed."""
     try:
-        subprocess.check_call(['git', '--version'], stdout=subprocess.DEVNULL,
+        subprocess.run(['git', '--version'], stdout=subprocess.DEVNULL,
-                              stderr=subprocess.DEVNULL)
+                       stderr=subprocess.DEVNULL, check=True)
     except (OSError, subprocess.CalledProcessError):
         return False
     else:
@@ -337,12 +337,13 @@ class TestGitStrSubprocess:
                 # If we don't call this with shell=True it might fail under
                 # some environments on Windows...
                 # http://bugs.python.org/issue24493
-                subprocess.check_call(
+                subprocess.run(
                     'git -C "{}" {}'.format(tmpdir, ' '.join(args)),
-                    env=env, shell=True)
+                    env=env, check=True, shell=True)
             else:
-                subprocess.check_call(
+                subprocess.run(
-                    ['git', '-C', str(tmpdir)] + list(args), env=env)
+                    ['git', '-C', str(tmpdir)] + list(args),
+                    check=True, env=env)
 
         (tmpdir / 'file').write_text("Hello World!", encoding='utf-8')
         _git('init')
@@ -368,14 +369,14 @@ class TestGitStrSubprocess:
         subprocess.CalledProcessError(1, 'foobar')
     ])
     def test_exception(self, exc, mocker, tmpdir):
-        """Test with subprocess.check_output raising an exception.
+        """Test with subprocess.run raising an exception.
 
         Args:
             exc: The exception to raise.
         """
         m = mocker.patch('qutebrowser.utils.version.os')
         m.path.isdir.return_value = True
-        mocker.patch('qutebrowser.utils.version.subprocess.check_output',
+        mocker.patch('qutebrowser.utils.version.subprocess.run',
                      side_effect=exc)
         ret = version._git_str_subprocess(str(tmpdir))
         assert ret is None

tests/unit/utils/usertypes/test_enum.py

@@ -1,74 +0,0 @@
-# vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
-
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
-#
-# This file is part of qutebrowser.
-#
-# qutebrowser is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# qutebrowser is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
-
-"""Tests for the Enum class."""
-
-from qutebrowser.utils import usertypes
-
-import pytest
-
-
-@pytest.fixture
-def enum():
-    return usertypes.enum('Enum', ['one', 'two'])
-
-
-def test_values(enum):
-    """Test if enum members resolve to the right values."""
-    assert enum.one.value == 1
-    assert enum.two.value == 2
-
-
-def test_name(enum):
-    """Test .name mapping."""
-    assert enum.one.name == 'one'
-    assert enum.two.name == 'two'
-
-
-def test_unknown(enum):
-    """Test invalid values which should raise an AttributeError."""
-    with pytest.raises(AttributeError):
-        _ = enum.three  # flake8: disable=F841
-
-
-def test_start():
-    """Test the start= argument."""
-    e = usertypes.enum('Enum', ['three', 'four'], start=3)
-    assert e.three.value == 3
-    assert e.four.value == 4
-
-
-def test_exit():
-    """Make sure the exit status enum is correct."""
-    assert usertypes.Exit.ok == 0
-    assert usertypes.Exit.reserved == 1
-
-
-def test_is_int():
-    """Test the is_int argument."""
-    int_enum = usertypes.enum('Enum', ['item'], is_int=True)
-    no_int_enum = usertypes.enum('Enum', ['item'])
-    assert isinstance(int_enum.item, int)
-    assert not isinstance(no_int_enum.item, int)
-
-
-def test_unique():
-    """Make sure elements need to be unique."""
-    with pytest.raises(TypeError):
-        usertypes.enum('Enum', ['item', 'item'])