Merge branch 'master' into issue#2785

.appveyor.yml

@@ -7,7 +7,7 @@ environment:
     PYTHONUNBUFFERED: 1
     PYTHON: C:\Python36\python.exe
     matrix:
-    - TESTENV: py36-pyqt59
+    - TESTENV: py36-pyqt510
     - TESTENV: pylint
 
 install:

.github/CODEOWNERS

@@ -8,3 +8,5 @@ tests/unit/completion/*  @rcorre
 tests/unit/misc/test_sql.py  @rcorre
 
 qutebrowser/config/configdata.yml  @mschilli87
+
+qutebrowser/javascript/caret.js @artur-shaik

.gitignore

@@ -25,6 +25,7 @@ __pycache__
 /.tox
 /testresults.html
 /.cache
+/.pytest_cache
 /.testmondata
 /.hypothesis
 /.mypy_cache

.travis.yml

@@ -21,6 +21,17 @@ matrix:
       env: TESTENV=py35-pyqt59
     - os: linux
       env: TESTENV=py36-pyqt59-cov
+    - os: linux
+      env: TESTENV=py36-pyqt510
+      # We need a newer Xvfb as a WORKAROUND for:
+      # https://bugreports.qt.io/browse/QTBUG-64928
+      sudo: required
+      addons:
+         apt:
+           sources:
+           - sourceline: "deb http://us.archive.ubuntu.com/ubuntu/ xenial main universe"
+           packages:
+           - xvfb
     - os: osx
       env: TESTENV=py36 OSX=sierra
       osx_image: xcode9.2

MANIFEST.in

@@ -8,7 +8,7 @@ graft icons
 graft doc/img
 graft misc/apparmor
 graft misc/userscripts
-recursive-include scripts *.py *.sh
+recursive-include scripts *.py *.sh *.js
 include qutebrowser/utils/testfile
 include qutebrowser/git-commit-id
 include LICENSE doc/* README.asciidoc

README.asciidoc

@@ -114,7 +114,7 @@ The following software and libraries are required to run qutebrowser:
 * http://fdik.org/pyPEG/[pyPEG2]
 * http://jinja.pocoo.org/[jinja2]
 * http://pygments.org/[pygments]
-* http://pyyaml.org/wiki/PyYAML[PyYAML]
+* https://github.com/yaml/pyyaml[PyYAML]
 * http://www.attrs.org/[attrs]
 
 The following libraries are optional:

doc/changelog.asciidoc

@@ -21,7 +21,18 @@ v1.2.0 (unreleased)
 Added
 ~~~~~
 
+- QtWebEngine: Caret/visual mode is now supported.
 - A new `qute://bindings` page, opened by `:bind`, shows all keybindings.
+- `:session-load` has a new `--delete` flag which deletes the
+  session after loading it.
+- QtWebEngine: Retrying downloads is now supported with Qt 5.10 or newer.
+- QtWebEngine: Hinting and other features inside same-origin frames is now
+  supported.
+- New `cycle-inputs.js` script in `scripts/` which can be used with `:jseval -f`
+  to cycle through inputs.
+- New `--no-last` flag for `:tab-focus` to not focus the last tab when focusing
+  the currently focused one.
+- New `--edit` flag for `:view-source` to open the source in an external editor.
 
 Changed
 ~~~~~~~
@@ -31,19 +42,57 @@ Changed
 - Deleting a prefix (`:`, `/` or `?`) via backspace now leaves command mode.
 - Angular 1 elements now get hints assigned.
 - `:tab-only` with pinned tabs now still closes unpinned tabs.
+- GreaseMonkey `@include` and `@exclude` now support
+  regex matches. With QtWebEngine and Qt 5.8 and newer, Qt handles the matching,
+  but similar functionality was added in Qt 5.11.
+- The sqlite history now uses write-ahead logging which should be
+  a performance and stability improvement.
+- The `url.incdec_segments` option now also can take `port` as possible segment.
+- QtWebEngine: `:view-source` now uses Chromium's `view-source:` scheme.
+- Tabs now show their full title as tooltip.
+- When an editor is spawned with `:open-editor` and `:config-edit`, the changes
+  are now applied as soon as the file is saved in the editor.
+- When there are multiple unknown keys in a autoconfig.yml, they now all get
+  reported in one error.
+- New `tabs.mode_on_change` setting which replaces
+  `tabs.persist_mode_on_change`. It can now be set to `restore` which remembers
+  input modes (input/passthrough) per tab.
 
 Fixed
 ~~~~~
 
-- Improved fullscreen handling with Qt 5.10.
+- QtWebEngine: Improved fullscreen handling with Qt 5.10.
+- QtWebEngine: Hinting and scrolling now works properly on special
+  `view-source:` pages.
+- QtWebKit: `:view-source` now displays a valid URL.
+- URLs containing ampersands and other special chars are now shown
+  correctly when filtering them in the completion.
+- `:bookmark-add "" foo` can now be used to save the current URL with a custom
+  title.
+- `:spawn -o` now waits until the process has finished before trying to show the
+  output. Previously, it incorrectly showed the previous output immediately.
+- QtWebEngine: Qt download objects are now cleaned up properly when a download
+  is removed.
+- Suspended pages now should always load the correct page when being un-suspended.
 
-v1.1.1 (unreleased)
+Removed
--------------------
+~~~~~~~
+
+- `QUTE_SELECTED_HTML` is now not set for userscripts anymore except when called
+  via hints.
+- The `qutebrowser_viewsource` userscript has been removed as `:view-source
+  --edit` can now be used.
+- The `tabs.persist_mode_on_change` setting has been removed and replaced by
+  `tabs.mode_on_change`.
+
+v1.1.1
+------
 
 Fixed
 ~~~~~
 
 - The Makefile now actually works.
+- Fixed crashes with Qt 5.10 when closing a tab before it finished loading.
 
 v1.1.0
 ------
@@ -1124,7 +1173,7 @@ Added
 - New `:fake-key` command to send a fake keypress to a website or to
   qutebrowser.
 - New `--mhtml` argument for `:download` to download a page including all
-  ressources as MHTML file.
+  resources as MHTML file.
 - New option `tabs -> title-alignment` to change the alignment of tab titles.
 
 Changed
@@ -1324,7 +1373,7 @@ Added
 - New argument `--no-err-windows` to suppress all error windows.
 - New arguments `--top-navigate` and `--bottom-navigate` (`-t`/`-b`) for `:scroll-page` to specify a navigation action (e.g. automatically go to the next page when arriving at the bottom).
 - New flag `-d`/`--detach` for `:spawn` to detach the spawned process so it's not closed when qutebrowser is.
-- New flag `-v`/`--verbose` for `:spawn` to print informations when the process started/exited successfully.
+- New flag `-v`/`--verbose` for `:spawn` to print information when the process started/exited successfully.
 - Many new color settings (foreground setting for every background setting).
 - New setting `ui -> modal-js-dialog` to use the standard modal dialogs for javascript questions instead of using the statusbar.
 - New setting `colors -> webpage.bg` to set the background color to use for websites which don't set one.

doc/contributing.asciidoc

@@ -375,7 +375,7 @@ The following logging levels are available for every logger:
 |error     |There was an issue and some kind of operation was abandoned.
 |warning   |There was an issue but the operation can continue running.
 |info      |General informational messages.
-|debug     |Verbose debugging informations.
+|debug     |Verbose debugging information.
 |=======================================================================
 
 [[commands]]

doc/faq.asciidoc

@@ -216,6 +216,29 @@ And then re-emerging qtwebengine with: +
     
     emerge -1 qtwebengine
 
+Unable to view DRM content (Netflix, Spotify, etc.).::
+    You will need to install `widevine` and set `qt.args` to point to it.
+    Qt 5.9 currently only supports widevine up to Chrome version 61.
++
+On Arch, simply install `qt5-webengine-widevine` from the AUR and run:
++
+----
+:set qt.args '["ppapi-widevine-path=/usr/lib/qt/plugins/ppapi/libwidevinecdmadapter.so"]'
+:restart
+----
++
+For other distributions, download the chromium tarball and widevine-cdm zip from
+https://aur.archlinux.org/packages/qt5-webengine-widevine/[the AUR page],
+extract `libwidevinecdmadapter.so` and `libwidevinecdm.so` files, respectively,
+and move them to the `ppapi` plugin directory in your Qt library directory (create it if it does not exist).
++
+Lastly, set your `qt.args` to point to that directory and restart qutebrowser:
++
+----
+:set qt.args '["ppapi-widevine-path=/usr/lib64/qt5/plugins/ppapi/libwidevinecdmadapter.so"]'
+:restart
+----
+
 My issue is not listed.::
     If you experience any segfaults or crashes, you can report the issue in
     https://github.com/qutebrowser/qutebrowser/issues[the issue tracker] or

doc/help/commands.asciidoc

@@ -1,6 +1,7 @@
 // DO NOT EDIT THIS FILE DIRECTLY!
 // It is autogenerated by running:
 //   $ python3 scripts/dev/src2asciidoc.py
+// vim: readonly:
 
 = Commands
 
@@ -175,7 +176,8 @@ Save the current page as a bookmark, or a specific url.
 If no url and title are provided, then save the current page as a bookmark. If a url and title have been provided, then save the given url as a bookmark with the provided title. You can view all saved bookmarks on the link:qute://bookmarks[bookmarks page].
 
 ==== positional arguments
-* +'url'+: url to save as a bookmark. If None, use url of current page.
+* +'url'+: url to save as a bookmark. If not given, use url of current page.
+
 * +'title'+: title of the new bookmark.
 
 ==== optional arguments
@@ -742,7 +744,13 @@ This tries to automatically click on typical _Previous Page_ or _Next Page_ link
  - `next`: Open a _next_ link.
  - `up`: Go up a level in the current URL.
  - `increment`: Increment the last number in the URL.
+ Uses the
+ link:settings.html#url.incdec_segments[url.incdec_segments]
+ config option.
  - `decrement`: Decrement the last number in the URL.
+ Uses the
+ link:settings.html#url.incdec_segments[url.incdec_segments]
+ config option.
  
 
 
@@ -1067,7 +1075,7 @@ Delete a session.
 
 [[session-load]]
 === session-load
-Syntax: +:session-load [*--clear*] [*--temp*] [*--force*] 'name'+
+Syntax: +:session-load [*--clear*] [*--temp*] [*--force*] [*--delete*] 'name'+
 
 Load a session.
 
@@ -1079,6 +1087,7 @@ Load a session.
 * +*-t*+, +*--temp*+: Don't set the current session for :session-save.
 * +*-f*+, +*--force*+: Force loading internal sessions (starting with an underline).
 
+* +*-d*+, +*--delete*+: Delete the saved session once it has loaded.
 
 [[session-save]]
 === session-save
@@ -1203,7 +1212,7 @@ The tab index to close
 
 [[tab-focus]]
 === tab-focus
-Syntax: +:tab-focus ['index']+
+Syntax: +:tab-focus [*--no-last*] ['index']+
 
 Select the tab given as argument/[count].
 
@@ -1215,6 +1224,9 @@ If neither count nor index are given, it behaves like tab-next. If both are give
  last tab.
 
 
+==== optional arguments
+* +*-n*+, +*--no-last*+: Whether to avoid focusing last tab if already focused.
+
 ==== count
 The tab index to focus, starting with 1.
 
@@ -1314,8 +1326,13 @@ Show version information.
 
 [[view-source]]
 === view-source
+Syntax: +:view-source [*--edit*]+
+
 Show the source of the current page in a new tab.
 
+==== optional arguments
+* +*-e*+, +*--edit*+: Edit the source in the editor instead of opening a tab.
+
 [[window-only]]
 === window-only
 Close all windows except for the current one.

doc/help/configuring.asciidoc

@@ -254,7 +254,7 @@ Getting the config directory
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 If you need to get the qutebrowser config directory, you can do so by reading
-`config.configdir`. Similarily, you can get the qutebrowser data directory via
+`config.configdir`. Similarly, you can get the qutebrowser data directory via
 `config.datadir`.
 
 This gives you a https://docs.python.org/3/library/pathlib.html[`pathlib.Path`
@@ -366,6 +366,8 @@ You can use something like this to read colors from an `~/.Xresources` file:
 
 [source,python]
 ----
+import subprocess
+
 def read_xresources(prefix):
     props = {}
     x = subprocess.run(['xrdb', '-query'], stdout=subprocess.PIPE)
@@ -376,7 +378,7 @@ def read_xresources(prefix):
     return props
 
 xresources = read_xresources('*')
-c.colors.statusbar.normal.bg = xresources['*background']
+c.colors.statusbar.normal.bg = xresources['*.background']
 ----
 
 Avoiding flake8 errors

doc/help/settings.asciidoc

@@ -1,6 +1,7 @@
 // DO NOT EDIT THIS FILE DIRECTLY!
 // It is autogenerated by running:
 //   $ python3 scripts/dev/src2asciidoc.py
+// vim: readonly:
 
 = Setting reference
 
@@ -237,11 +238,11 @@
 |<<tabs.indicator.padding,tabs.indicator.padding>>|Padding (in pixels) for tab indicators.
 |<<tabs.indicator.width,tabs.indicator.width>>|Width (in pixels) of the progress indicator (0 to disable).
 |<<tabs.last_close,tabs.last_close>>|How to behave when the last tab is closed.
+|<<tabs.mode_on_change,tabs.mode_on_change>>|When switching tabs, what input mode is applied.
 |<<tabs.mousewheel_switching,tabs.mousewheel_switching>>|Switch between tabs using the mouse wheel.
 |<<tabs.new_position.related,tabs.new_position.related>>|Position of new tabs opened from another tab.
 |<<tabs.new_position.unrelated,tabs.new_position.unrelated>>|Position of new tabs which aren't opened from another tab.
 |<<tabs.padding,tabs.padding>>|Padding (in pixels) around text for tabs.
-|<<tabs.persist_mode_on_change,tabs.persist_mode_on_change>>|Stay in insert/passthrough mode when switching tabs.
 |<<tabs.pinned.shrink,tabs.pinned.shrink>>|Shrink pinned tabs down to their contents.
 |<<tabs.position,tabs.position>>|Position of the tab bar.
 |<<tabs.select_on_remove,tabs.select_on_remove>>|Which tab to select when the focused tab is removed.
@@ -1579,7 +1580,7 @@ Default: +pass:[true]+
 [[content.headers.referer]]
 === content.headers.referer
 When to send the Referer header.
-The Referer header tells websites from which website you were coming from when visting them.
+The Referer header tells websites from which website you were coming from when visiting them.
 
 Type: <<types,String>>
 
@@ -2773,6 +2774,20 @@ Valid values:
 
 Default: +pass:[ignore]+
 
+[[tabs.mode_on_change]]
+=== tabs.mode_on_change
+When switching tabs, what input mode is applied.
+
+Type: <<types,String>>
+
+Valid values:
+
+ * +persist+: Retain the current mode.
+ * +restore+: Restore previously saved mode.
+ * +normal+: Always revert to normal mode.
+
+Default: +pass:[normal]+
+
 [[tabs.mousewheel_switching]]
 === tabs.mousewheel_switching
 Switch between tabs using the mouse wheel.
@@ -2824,14 +2839,6 @@ Default:
 - +pass:[right]+: +pass:[5]+
 - +pass:[top]+: +pass:[0]+
 
-[[tabs.persist_mode_on_change]]
-=== tabs.persist_mode_on_change
-Stay in insert/passthrough mode when switching tabs.
-
-Type: <<types,Bool>>
-
-Default: +pass:[false]+
-
 [[tabs.pinned.shrink]]
 === tabs.pinned.shrink
 Shrink pinned tabs down to their contents.
@@ -2993,6 +3000,7 @@ Type: <<types,FlagList>>
 Valid values:
 
  * +host+
+ * +port+
  * +path+
  * +query+
  * +anchor+

doc/stacktrace.asciidoc

@@ -37,7 +37,7 @@ is available in the repositories:
 Archlinux
 ^^^^^^^^^
 
-For Archlinux, no debug informations are provided. You can either compile Qt
+For Archlinux, no debug information is provided. You can either compile Qt
 yourself (which will take a few hours even on a modern machine) or use
 debugging symbols compiled/packaged by me (x86_64 only).
 

doc/userscripts.asciidoc

@@ -45,8 +45,6 @@ In `command` mode:
 - `QUTE_URL`: The current URL.
 - `QUTE_TITLE`: The title of the current page.
 - `QUTE_SELECTED_TEXT`: The text currently selected on the page.
-- `QUTE_SELECTED_HTML` The HTML currently selected on the page (not supported
-  with QtWebEngine).
 
 In `hints` mode:
 

misc/requirements/requirements-codecov.txt

@@ -1,9 +1,9 @@
 # This file is automatically generated by scripts/dev/recompile_requirements.py
 
-certifi==2017.11.5
+certifi==2018.1.18
 chardet==3.0.4
-codecov==2.0.13
+codecov==2.0.15
-coverage==4.4.2
+coverage==4.5
 idna==2.6
 requests==2.18.4
 urllib3==1.22

misc/requirements/requirements-flake8.txt

@@ -2,7 +2,7 @@
 
 attrs==17.4.0
 flake8==3.5.0
-flake8-bugbear==17.12.0
+flake8-bugbear==18.2.0
 flake8-builtins==1.0.post0
 flake8-comprehensions==1.4.1
 flake8-copyright==0.2.0

misc/requirements/requirements-pip.txt

@@ -3,6 +3,6 @@
 appdirs==1.4.3
 packaging==16.8
 pyparsing==2.2.0
-setuptools==38.4.0
+setuptools==38.5.0
 six==1.11.0
 wheel==0.30.0

misc/requirements/requirements-pylint-master.txt

@@ -1,11 +1,11 @@
 # This file is automatically generated by scripts/dev/recompile_requirements.py
 
 -e git+https://github.com/PyCQA/astroid.git#egg=astroid
-certifi==2017.11.5
+certifi==2018.1.18
 chardet==3.0.4
 github3.py==0.9.6
 idna==2.6
-isort==4.2.15
+isort==4.3.2
 lazy-object-proxy==1.3.1
 mccabe==0.6.1
 -e git+https://github.com/PyCQA/pylint.git#egg=pylint

misc/requirements/requirements-pylint.txt

@@ -1,14 +1,14 @@
 # This file is automatically generated by scripts/dev/recompile_requirements.py
 
-astroid==1.6.0
+astroid==1.6.1
-certifi==2017.11.5
+certifi==2018.1.18
 chardet==3.0.4
 github3.py==0.9.6
 idna==2.6
-isort==4.2.15
+isort==4.3.2
 lazy-object-proxy==1.3.1
 mccabe==0.6.1
-pylint==1.8.1
+pylint==1.8.2
 ./scripts/dev/pylint_checkers
 requests==2.18.4
 six==1.11.0

misc/requirements/requirements-pyqt.txt

@@ -1,4 +1,4 @@
 # This file is automatically generated by scripts/dev/recompile_requirements.py
 
-PyQt5==5.9.2
+PyQt5==5.10
-sip==4.19.6
+sip==4.19.7

misc/requirements/requirements-tests.txt

@@ -5,24 +5,25 @@ beautifulsoup4==4.6.0
 cheroot==6.0.0
 click==6.7
 # colorama==0.3.9
-coverage==4.4.2
+coverage==4.5
 EasyProcess==0.2.3
 fields==5.0.0
 Flask==0.12.2
 glob2==0.6
 hunter==2.0.2
-hypothesis==3.44.16
+hypothesis==3.44.25
 itsdangerous==0.24
 # Jinja2==2.10
 Mako==1.0.7
 # MarkupSafe==1.0
+more-itertools==4.1.0
 parse==1.8.2
 parse-type==0.4.2
 pluggy==0.6.0
 py==1.5.2
 py-cpuinfo==3.3.0
-pytest==3.3.1  # rq.filter: != 3.3.2
+pytest==3.4.0
-pytest-bdd==2.19.0
+pytest-bdd==2.20.0
 pytest-benchmark==3.1.1
 pytest-cov==2.5.1
 pytest-faulthandler==1.3.1

misc/requirements/requirements-tests.txt-raw

@@ -4,7 +4,7 @@ coverage
 Flask
 hunter
 hypothesis
-pytest==3.3.1
+pytest
 pytest-bdd
 pytest-benchmark
 pytest-cov
@@ -19,4 +19,3 @@ pytest-xvfb
 vulture
 
 #@ ignore: Jinja2, MarkupSafe, colorama
-#@ filter: pytest != 3.3.2

misc/userscripts/qutebrowser_viewsource

@@ -1,33 +0,0 @@
-#!/usr/bin/env bash
-
-# Copyright 2015 Zach-Button <zachrey.button@gmail.com>
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
-#
-# This file is part of qutebrowser.
-#
-# qutebrowser is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# qutebrowser is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with qutebrowser.  If not, see <http://www.gnu.org/licenses/>.
-
-#
-# This script fetches the unprocessed HTML source for a page and opens it in vim.
-#	:bind gf spawn --userscript qutebrowser_viewsource
-#
-# Caveat: Does not use authentication of any kind. Add it in if you want it to.
-#
-
-path=$(mktemp --tmpdir qutebrowser_XXXXXXXX.html)
-
-curl "$QUTE_URL" > "$path"
-urxvt -e vim "$path"
-
-rm "$path"

pytest.ini

@@ -1,4 +1,5 @@
 [pytest]
+log_level = NOTSET
 addopts = --strict -rfEw --faulthandler-timeout=90 --instafail --pythonwarnings error --benchmark-columns=Min,Max,Median
 testpaths = tests
 markers =
@@ -25,6 +26,7 @@ markers =
     this: Used to mark tests during development
     no_invalid_lines: Don't fail on unparseable lines in end2end tests
     issue2478: Tests which are broken on Windows with QtWebEngine, https://github.com/qutebrowser/qutebrowser/issues/2478
+    issue3572: Tests which are broken with QtWebEngine and Qt 5.10, https://github.com/qutebrowser/qutebrowser/issues/3572
     fake_os: Fake utils.is_* to a fake operating system
     unicode_locale: Tests which need an unicode locale to work
 qt_log_level_fail = WARNING

qutebrowser.py

@@ -2,7 +2,7 @@
 
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2015 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/__init__.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -26,7 +26,7 @@ __copyright__ = "Copyright 2014-2017 Florian Bruhin (The Compiler)"
 __license__ = "GPL"
 __maintainer__ = __author__
 __email__ = "mail@qutebrowser.org"
-__version_info__ = (1, 1, 0)
+__version_info__ = (1, 1, 1)
 __version__ = '.'.join(str(e) for e in __version_info__)
 __description__ = "A keyboard-driven, vim-like browser based on PyQt5."
 

qutebrowser/__main__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python3
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/app.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/__init__.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/adblock.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/browsertab.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -94,21 +94,21 @@ class TabData:
         keep_icon: Whether the (e.g. cloned) icon should not be cleared on page
                    load.
         inspector: The QWebInspector used for this webview.
-        viewing_source: Set if we're currently showing a source view.
         override_target: Override for open_target for fake clicks (like hints).
                          Only used for QtWebKit.
         pinned: Flag to pin the tab.
         fullscreen: Whether the tab has a video shown fullscreen currently.
         netrc_used: Whether netrc authentication was performed.
+        input_mode: current input mode for the tab.
     """
 
     keep_icon = attr.ib(False)
-    viewing_source = attr.ib(False)
     inspector = attr.ib(None)
     override_target = attr.ib(None)
     pinned = attr.ib(False)
     fullscreen = attr.ib(False)
     netrc_used = attr.ib(False)
+    input_mode = attr.ib(usertypes.KeyMode.normal)
 
 
 class AbstractAction:
@@ -123,8 +123,9 @@ class AbstractAction:
     action_class = None
     action_base = None
 
-    def __init__(self):
+    def __init__(self, tab):
         self._widget = None
+        self._tab = tab
 
     def exit_fullscreen(self):
         """Exit the fullscreen mode."""
@@ -141,6 +142,10 @@ class AbstractAction:
             raise WebTabError("{} is not a valid web action!".format(name))
         self._widget.triggerPageAction(member)
 
+    def show_source(self):
+        """Show the source of the current page in a new tab."""
+        raise NotImplementedError
+
 
 class AbstractPrinting:
 
@@ -247,10 +252,10 @@ class AbstractZoom(QObject):
         _default_zoom_changed: Whether the zoom was changed from the default.
     """
 
-    def __init__(self, win_id, parent=None):
+    def __init__(self, tab, parent=None):
         super().__init__(parent)
+        self._tab = tab
         self._widget = None
-        self._win_id = win_id
         self._default_zoom_changed = False
         self._init_neighborlist()
         config.instance.changed.connect(self._on_config_changed)
@@ -326,10 +331,9 @@ class AbstractCaret(QObject):
 
     """Attribute of AbstractTab for caret browsing."""
 
-    def __init__(self, win_id, tab, mode_manager, parent=None):
+    def __init__(self, tab, mode_manager, parent=None):
         super().__init__(parent)
         self._tab = tab
-        self._win_id = win_id
         self._widget = None
         self.selection_enabled = False
         mode_manager.entered.connect(self._on_mode_entered)
@@ -392,10 +396,7 @@ class AbstractCaret(QObject):
     def drop_selection(self):
         raise NotImplementedError
 
-    def has_selection(self):
+    def selection(self, callback):
-        raise NotImplementedError
-
-    def selection(self, html=False):
         raise NotImplementedError
 
     def follow_selected(self, *, tab=False):
@@ -641,16 +642,6 @@ class AbstractTab(QWidget):
         tab_registry[self.tab_id] = self
         objreg.register('tab', self, registry=self.registry)
 
-        # self.history = AbstractHistory(self)
-        # self.scroller = AbstractScroller(self, parent=self)
-        # self.caret = AbstractCaret(win_id=win_id, tab=self,
-        #                            mode_manager=mode_manager, parent=self)
-        # self.zoom = AbstractZoom(win_id=win_id)
-        # self.search = AbstractSearch(parent=self)
-        # self.printing = AbstractPrinting()
-        # self.elements = AbstractElements(self)
-        # self.action = AbstractAction()
-
         self.data = TabData()
         self._layout = miscwidgets.WrapperLayout(self)
         self._widget = None
@@ -725,7 +716,6 @@ class AbstractTab(QWidget):
     def _on_load_started(self):
         self._progress = 0
         self._has_ssl_errors = False
-        self.data.viewing_source = False
         self._set_load_status(usertypes.LoadStatus.loading)
         self.load_started.emit()
 
@@ -751,6 +741,10 @@ class AbstractTab(QWidget):
 
     @pyqtSlot(bool)
     def _on_load_finished(self, ok):
+        if sip.isdeleted(self._widget):
+            # https://github.com/qutebrowser/qutebrowser/issues/3498
+            return
+
         sess_manager = objreg.get('session-manager')
         sess_manager.save_autosave()
 
@@ -813,7 +807,7 @@ class AbstractTab(QWidget):
         raise NotImplementedError
 
     def dump_async(self, callback, *, plain=False):
-        """Dump the current page to a file ascync.
+        """Dump the current page's html asynchronously.
 
         The given callback will be called with the result when dumping is
         complete.

qutebrowser/browser/commands.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -26,12 +26,9 @@ import functools
 import typing
 
 from PyQt5.QtWidgets import QApplication, QTabBar, QDialog
-from PyQt5.QtCore import Qt, QUrl, QEvent, QUrlQuery
+from PyQt5.QtCore import pyqtSlot, Qt, QUrl, QEvent, QUrlQuery
 from PyQt5.QtGui import QKeyEvent
 from PyQt5.QtPrintSupport import QPrintDialog, QPrintPreviewDialog
-import pygments
-import pygments.lexers
-import pygments.formatters
 
 from qutebrowser.commands import userscripts, cmdexc, cmdutils, runners
 from qutebrowser.config import config, configdata
@@ -638,7 +635,13 @@ class CommandDispatcher:
                 - `next`: Open a _next_ link.
                 - `up`: Go up a level in the current URL.
                 - `increment`: Increment the last number in the URL.
+                  Uses the
+                  link:settings.html#url.incdec_segments[url.incdec_segments]
+                  config option.
                 - `decrement`: Decrement the last number in the URL.
+                  Uses the
+                  link:settings.html#url.incdec_segments[url.incdec_segments]
+                  config option.
 
             tab: Open in a new tab.
             bg: Open in a background tab.
@@ -849,14 +852,21 @@ class CommandDispatcher:
             s = self._yank_url(what)
             what = 'URL'  # For printing
         elif what == 'selection':
+            def _selection_callback(s):
+                if not s:
+                    message.info("Nothing to yank")
+                    return
+                self._yank_to_target(s, sel, what, keep)
+
             caret = self._current_widget().caret
-            s = caret.selection()
+            caret.selection(callback=_selection_callback)
-            if not caret.has_selection() or not s:
+            return
-                message.info("Nothing to yank")
-                return
         else:  # pragma: no cover
             raise ValueError("Invalid value {!r} for `what'.".format(what))
 
+        self._yank_to_target(s, sel, what, keep)
+
+    def _yank_to_target(self, s, sel, what, keep):
         if sel and utils.supports_selection():
             target = "primary selection"
         else:
@@ -1099,7 +1109,8 @@ class CommandDispatcher:
     @cmdutils.register(instance='command-dispatcher', scope='window')
     @cmdutils.argument('index', choices=['last'])
     @cmdutils.argument('count', count=True)
-    def tab_focus(self, index: typing.Union[str, int] = None, count=None):
+    def tab_focus(self, index: typing.Union[str, int] = None,
+                  count=None, no_last=False):
         """Select the tab given as argument/[count].
 
         If neither count nor index are given, it behaves like tab-next.
@@ -1111,13 +1122,14 @@ class CommandDispatcher:
                    Negative indices count from the end, such that -1 is the
                    last tab.
             count: The tab index to focus, starting with 1.
+            no_last: Whether to avoid focusing last tab if already focused.
         """
         index = count if count is not None else index
 
         if index == 'last':
             self._tab_focus_last()
             return
-        elif index == self._current_index() + 1:
+        elif not no_last and index == self._current_index() + 1:
             self._tab_focus_last(show_error=False)
             return
         elif index is None:
@@ -1207,9 +1219,29 @@ class CommandDispatcher:
 
         log.procs.debug("Executing {} with args {}, userscript={}".format(
             cmd, args, userscript))
+
+        @pyqtSlot()
+        def _on_proc_finished():
+            if output:
+                tb = objreg.get('tabbed-browser', scope='window',
+                                window='last-focused')
+                tb.openurl(QUrl('qute://spawn-output'), newtab=True)
+
         if userscript:
+            def _selection_callback(s):
+                try:
+                    runner = self._run_userscript(s, cmd, args, verbose)
+                    runner.finished.connect(_on_proc_finished)
+                except cmdexc.CommandError as e:
+                    message.error(str(e))
+
             # ~ expansion is handled by the userscript module.
-            self._run_userscript(cmd, *args, verbose=verbose)
+            # dirty hack for async call because of:
+            # https://bugreports.qt.io/browse/QTBUG-53134
+            # until it fixed or blocked async call implemented:
+            # https://github.com/qutebrowser/qutebrowser/issues/3327
+            caret = self._current_widget().caret
+            caret.selection(callback=_selection_callback)
         else:
             cmd = os.path.expanduser(cmd)
             proc = guiprocess.GUIProcess(what='command', verbose=verbose,
@@ -1218,18 +1250,14 @@ class CommandDispatcher:
                 proc.start_detached(cmd, args)
             else:
                 proc.start(cmd, args)
-
+            proc.finished.connect(_on_proc_finished)
-        if output:
-            tabbed_browser = objreg.get('tabbed-browser', scope='window',
-                                        window='last-focused')
-            tabbed_browser.openurl(QUrl('qute://spawn-output'), newtab=True)
 
     @cmdutils.register(instance='command-dispatcher', scope='window')
     def home(self):
         """Open main startpage in current tab."""
         self.openurl(config.val.url.start_pages[0])
 
-    def _run_userscript(self, cmd, *args, verbose=False):
+    def _run_userscript(self, selection, cmd, args, verbose):
         """Run a userscript given as argument.
 
         Args:
@@ -1239,21 +1267,15 @@ class CommandDispatcher:
         """
         env = {
             'QUTE_MODE': 'command',
+            'QUTE_SELECTED_TEXT': selection,
         }
 
         idx = self._current_index()
         if idx != -1:
             env['QUTE_TITLE'] = self._tabbed_browser.page_title(idx)
 
-        tab = self._tabbed_browser.currentWidget()
-        if tab is not None and tab.caret.has_selection():
-            env['QUTE_SELECTED_TEXT'] = tab.caret.selection()
-            try:
-                env['QUTE_SELECTED_HTML'] = tab.caret.selection(html=True)
-            except browsertab.UnsupportedOperationError:
-                pass
-
         # FIXME:qtwebengine: If tab is None, run_async will fail!
+        tab = self._tabbed_browser.currentWidget()
 
         try:
             url = self._tabbed_browser.current_url()
@@ -1263,10 +1285,11 @@ class CommandDispatcher:
             env['QUTE_URL'] = url.toString(QUrl.FullyEncoded)
 
         try:
-            userscripts.run_async(tab, cmd, *args, win_id=self._win_id,
+            runner = userscripts.run_async(
-                                  env=env, verbose=verbose)
+                tab, cmd, *args, win_id=self._win_id, env=env, verbose=verbose)
         except userscripts.Error as e:
             raise cmdexc.CommandError(e)
+        return runner
 
     @cmdutils.register(instance='command-dispatcher', scope='window')
     def quickmark_save(self):
@@ -1328,7 +1351,8 @@ class CommandDispatcher:
         link:qute://bookmarks[bookmarks page].
 
         Args:
-            url: url to save as a bookmark. If None, use url of current page.
+            url: url to save as a bookmark. If not given, use url of current
+                 page.
             title: title of the new bookmark.
             toggle: remove the bookmark instead of raising an error if it
                     already exists.
@@ -1337,7 +1361,7 @@ class CommandDispatcher:
             raise cmdexc.CommandError('Title must be provided if url has '
                                       'been provided')
         bookmark_manager = objreg.get('bookmark-manager')
-        if url is None:
+        if not url:
             url = self._current_url()
         else:
             try:
@@ -1483,34 +1507,26 @@ class CommandDispatcher:
             )
 
     @cmdutils.register(instance='command-dispatcher', scope='window')
-    def view_source(self):
+    def view_source(self, edit=False):
-        """Show the source of the current page in a new tab."""
+        """Show the source of the current page in a new tab.
-        tab = self._current_widget()
-        if tab.data.viewing_source:
-            raise cmdexc.CommandError("Already viewing source!")
 
+        Args:
+            edit: Edit the source in the editor instead of opening a tab.
+        """
+        tab = self._current_widget()
         try:
             current_url = self._current_url()
         except cmdexc.CommandError as e:
             message.error(str(e))
             return
+        if current_url.scheme() == 'view-source':
+            raise cmdexc.CommandError("Already viewing source!")
 
-        def show_source_cb(source):
+        if edit:
-            """Show source as soon as it's ready."""
+            ed = editor.ExternalEditor(self._tabbed_browser)
-            # WORKAROUND for https://github.com/PyCQA/pylint/issues/491
+            tab.dump_async(ed.edit)
-            # pylint: disable=no-member
+        else:
-            lexer = pygments.lexers.HtmlLexer()
+            tab.action.show_source()
-            formatter = pygments.formatters.HtmlFormatter(
-                full=True, linenos='table',
-                title='Source for {}'.format(current_url.toDisplayString()))
-            # pylint: enable=no-member
-            highlighted = pygments.highlight(source, lexer, formatter)
-
-            new_tab = self._tabbed_browser.tabopen()
-            new_tab.set_html(highlighted)
-            new_tab.data.viewing_source = True
-
-        tab.dump_async(show_source_cb)
 
     @cmdutils.register(instance='command-dispatcher', scope='window',
                        debug=True)
@@ -1616,9 +1632,11 @@ class CommandDispatcher:
 
         caret_position = elem.caret_position()
 
-        ed = editor.ExternalEditor(self._tabbed_browser)
+        ed = editor.ExternalEditor(watch=True, parent=self._tabbed_browser)
-        ed.editing_finished.connect(functools.partial(
+        ed.file_updated.connect(functools.partial(
-            self.on_editing_finished, elem))
+            self.on_file_updated, elem))
+        ed.editing_finished.connect(lambda: mainwindow.raise_window(
+            objreg.last_focused_window(), alert=False))
         ed.edit(text, caret_position)
 
     @cmdutils.register(instance='command-dispatcher', scope='window')
@@ -1631,10 +1649,10 @@ class CommandDispatcher:
         tab = self._current_widget()
         tab.elements.find_focused(self._open_editor_cb)
 
-    def on_editing_finished(self, elem, text):
+    def on_file_updated(self, elem, text):
         """Write the editor text into the form field and clean up tempfile.
 
-        Callback for GUIProcess when the editor was closed.
+        Callback for GUIProcess when the edited text was updated.
 
         Args:
             elem: The WebElementWrapper which was modified.
@@ -1647,8 +1665,6 @@ class CommandDispatcher:
         except webelem.Error as e:
             raise cmdexc.CommandError(str(e))
 
-        mainwindow.raise_window(objreg.last_focused_window(), alert=False)
-
     @cmdutils.register(instance='command-dispatcher', maxsplit=0,
                        scope='window')
     def insert_text(self, text):
@@ -2141,7 +2157,7 @@ class CommandDispatcher:
         ed = editor.ExternalEditor(self._tabbed_browser)
 
         # Passthrough for openurl args (e.g. -t, -b, -w)
-        ed.editing_finished.connect(functools.partial(
+        ed.file_updated.connect(functools.partial(
             self._open_if_changed, old_url=old_url, bg=bg, tab=tab,
             window=window, private=private, related=related))
 

qutebrowser/browser/downloads.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/downloadview.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/greasemonkey.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2017-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -23,7 +23,6 @@ import re
 import os
 import json
 import fnmatch
-import functools
 import glob
 
 import attr
@@ -135,7 +134,7 @@ class GreasemonkeyManager(QObject):
     Signals:
         scripts_reloaded: Emitted when scripts are reloaded from disk.
             Any cached or already-injected scripts should be
-            considered obselete.
+            considered obsolete.
     """
 
     scripts_reloaded = pyqtSignal()
@@ -178,10 +177,10 @@ class GreasemonkeyManager(QObject):
                 elif script.run_at == 'document-idle':
                     self._run_idle.append(script)
                 else:
-                    log.greasemonkey.warning("Script {} has invalid run-at "
+                    if script.run_at:
-                                             "defined, defaulting to "
+                        log.greasemonkey.warning(
-                                             "document-end"
+                            "Script {} has invalid run-at defined, "
-                                             .format(script_path))
+                            "defaulting to document-end".format(script_path))
                     # Default as per
                     # https://wiki.greasespot.net/Metadata_Block#.40run-at
                     self._run_end.append(script)
@@ -196,11 +195,23 @@ class GreasemonkeyManager(QObject):
         """
         if url.scheme() not in self.greaseable_schemes:
             return MatchingScripts(url, [], [], [])
-        match = functools.partial(fnmatch.fnmatch,
+
-                                  url.toString(QUrl.FullyEncoded))
+        string_url = url.toString(QUrl.FullyEncoded)
+
+        def _match(pattern):
+            # For include and exclude rules if they start and end with '/' they
+            # should be treated as a (ecma syntax) regular expression.
+            if pattern.startswith('/') and pattern.endswith('/'):
+                matches = re.search(pattern[1:-1], string_url, flags=re.I)
+                return matches is not None
+
+            # Otherwise they are glob expressions.
+            return fnmatch.fnmatch(string_url, pattern)
+
         tester = (lambda script:
-                  any(match(pat) for pat in script.includes) and
+                  any(_match(pat) for pat in script.includes) and
-                  not any(match(pat) for pat in script.excludes))
+                  not any(_match(pat) for pat in script.excludes))
+
         return MatchingScripts(
             url,
             [script for script in self._run_start if tester(script)],

qutebrowser/browser/hints.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/history.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2015-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -172,7 +172,7 @@ class WebHistory(sql.SqlTable):
     @pyqtSlot(QUrl, QUrl, str)
     def add_from_tab(self, url, requested_url, title):
         """Add a new history entry as slot, called from a BrowserTab."""
-        if any(url.scheme() == 'data' or
+        if any(url.scheme() in ('data', 'view-source') or
                (url.scheme(), url.host()) == ('qute', 'back')
                for url in (url, requested_url)):
             return

qutebrowser/browser/inspector.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2015-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/mouse.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/navigate.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/network/pac.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/network/proxy.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -61,6 +61,9 @@ class ProxyFactory(QNetworkProxyFactory):
         """
         proxy = config.val.content.proxy
         if proxy is configtypes.SYSTEM_PROXY:
+            # On Linux, use "export http_proxy=socks5://host:port" to manually
+            # set system proxy.
+            # ref. http://doc.qt.io/qt-5/qnetworkproxyfactory.html#systemProxyForQuery
             proxies = QNetworkProxyFactory.systemProxyForQuery(query)
         elif isinstance(proxy, pac.PACFetcher):
             proxies = proxy.resolve(query)

qutebrowser/browser/pdfjs.py

@@ -1,7 +1,7 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
 # Copyright 2015 Daniel Schadt
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/qtnetworkdownloads.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/qutescheme.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/shared.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/signalfilter.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/urlmarks.py

@@ -1,7 +1,7 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
-# Copyright 2015-2017 Antoni Boucher <bouanto@zoho.com>
+# Copyright 2015-2018 Antoni Boucher <bouanto@zoho.com>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webelem.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/__init__.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/certificateerror.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/interceptor.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/spell.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2017 Michal Siedlaczek <michal.siedlaczek@gmail.com>
+# Copyright 2017-2018 Michal Siedlaczek <michal.siedlaczek@gmail.com>
 
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/tabhistory.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2015-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/webenginedownloads.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -45,6 +45,10 @@ class DownloadItem(downloads.AbstractDownloadItem):
         qt_item.downloadProgress.connect(self.stats.on_download_progress)
         qt_item.stateChanged.connect(self._on_state_changed)
 
+        # Ensure wrapped qt_item is deleted manually when the wrapper object
+        # is deleted. See https://github.com/qutebrowser/qutebrowser/issues/3373
+        self.destroyed.connect(self._qt_item.deleteLater)
+
     def _is_page_download(self):
         """Check if this item is a page (i.e. mhtml) download."""
         return (self._qt_item.savePageFormat() !=
@@ -96,9 +100,15 @@ class DownloadItem(downloads.AbstractDownloadItem):
         self._qt_item.cancel()
 
     def retry(self):
-        # https://bugreports.qt.io/browse/QTBUG-56840
+        state = self._qt_item.state()
-        raise downloads.UnsupportedOperationError(
+        assert state == QWebEngineDownloadItem.DownloadInterrupted, state
-            "Retrying downloads is unsupported with QtWebEngine")
+
+        try:
+            self._qt_item.resume()
+        except AttributeError:
+            raise downloads.UnsupportedOperationError(
+                "Retrying downloads is unsupported with QtWebEngine on "
+                "Qt/PyQt < 5.10")
 
     def _get_open_filename(self):
         return self._filename

qutebrowser/browser/webengine/webengineelem.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/webengineinspector.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2015-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/webenginequtescheme.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/webenginesettings.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webengine/webenginetab.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -21,6 +21,7 @@
 
 import math
 import functools
+import sys
 import html as html_utils
 
 import sip
@@ -98,6 +99,19 @@ class WebEngineAction(browsertab.AbstractAction):
         """Save the current page."""
         self._widget.triggerPageAction(QWebEnginePage.SavePage)
 
+    def show_source(self):
+        try:
+            self._widget.triggerPageAction(QWebEnginePage.ViewSource)
+        except AttributeError:
+            # Qt < 5.8
+            tb = objreg.get('tabbed-browser', scope='window',
+                            window=self._tab.win_id)
+            urlstr = self._tab.url().toString(QUrl.RemoveUserInfo)
+            # The original URL becomes the path of a view-source: URL
+            # (without a host), but query/fragment should stay.
+            url = QUrl('view-source:' + urlstr)
+            tb.tabopen(url, background=False, related=True)
+
 
 class WebEnginePrinting(browsertab.AbstractPrinting):
 
@@ -201,70 +215,87 @@ class WebEngineCaret(browsertab.AbstractCaret):
 
     @pyqtSlot(usertypes.KeyMode)
     def _on_mode_entered(self, mode):
-        pass
+        if mode != usertypes.KeyMode.caret:
+            return
+
+        self._tab.run_js_async(
+            javascript.assemble('caret', 'setPlatform', sys.platform))
+        self._js_call('setInitialCursor')
 
     @pyqtSlot(usertypes.KeyMode)
     def _on_mode_left(self):
-        pass
+        self.drop_selection()
+        self._js_call('disableCaret')
 
     def move_to_next_line(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveDown')
 
     def move_to_prev_line(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveUp')
 
     def move_to_next_char(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveRight')
 
     def move_to_prev_char(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveLeft')
 
     def move_to_end_of_word(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveToEndOfWord')
 
     def move_to_next_word(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveToNextWord')
 
     def move_to_prev_word(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveToPreviousWord')
 
     def move_to_start_of_line(self):
-        log.stub()
+        self._js_call('moveToStartOfLine')
 
     def move_to_end_of_line(self):
-        log.stub()
+        self._js_call('moveToEndOfLine')
 
     def move_to_start_of_next_block(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveToStartOfNextBlock')
 
     def move_to_start_of_prev_block(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveToStartOfPrevBlock')
 
     def move_to_end_of_next_block(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveToEndOfNextBlock')
 
     def move_to_end_of_prev_block(self, count=1):
-        log.stub()
+        for _ in range(count):
+            self._js_call('moveToEndOfPrevBlock')
 
     def move_to_start_of_document(self):
-        log.stub()
+        self._js_call('moveToStartOfDocument')
 
     def move_to_end_of_document(self):
-        log.stub()
+        self._js_call('moveToEndOfDocument')
 
     def toggle_selection(self):
-        log.stub()
+        self._js_call('toggleSelection')
 
     def drop_selection(self):
-        log.stub()
+        self._js_call('dropSelection')
 
-    def has_selection(self):
+    def selection(self, callback):
-        return self._widget.hasSelection()
+        # Not using selectedText() as WORKAROUND for
-
+        # https://bugreports.qt.io/browse/QTBUG-53134
-    def selection(self, html=False):
+        # Even on Qt 5.10 selectedText() seems to work poorly, see
-        if html:
+        # https://github.com/qutebrowser/qutebrowser/issues/3523
-            raise browsertab.UnsupportedOperationError
+        self._tab.run_js_async(javascript.assemble('caret', 'getSelection'),
-        return self._widget.selectedText()
+                               callback)
 
     def _follow_selected_cb(self, js_elem, tab=False):
         """Callback for javascript which clicks the selected element.
@@ -308,6 +339,10 @@ class WebEngineCaret(browsertab.AbstractCaret):
             self._tab.run_js_async(js_code, lambda jsret:
                                    self._follow_selected_cb(jsret, tab))
 
+    def _js_call(self, command):
+        self._tab.run_js_async(
+            javascript.assemble('caret', command))
+
 
 class WebEngineScroller(browsertab.AbstractScroller):
 
@@ -557,13 +592,13 @@ class WebEngineTab(browsertab.AbstractTab):
                                        private=private)
         self.history = WebEngineHistory(self)
         self.scroller = WebEngineScroller(self, parent=self)
-        self.caret = WebEngineCaret(win_id=win_id, mode_manager=mode_manager,
+        self.caret = WebEngineCaret(mode_manager=mode_manager,
                                     tab=self, parent=self)
-        self.zoom = WebEngineZoom(win_id=win_id, parent=self)
+        self.zoom = WebEngineZoom(tab=self, parent=self)
         self.search = WebEngineSearch(parent=self)
         self.printing = WebEnginePrinting()
-        self.elements = WebEngineElements(self)
+        self.elements = WebEngineElements(tab=self)
-        self.action = WebEngineAction()
+        self.action = WebEngineAction(tab=self)
         self._set_widget(widget)
         self._connect_signals()
         self.backend = usertypes.Backend.QtWebEngine
@@ -577,9 +612,12 @@ class WebEngineTab(browsertab.AbstractTab):
             'window._qutebrowser = window._qutebrowser || {};',
             utils.read_file('javascript/scroll.js'),
             utils.read_file('javascript/webelem.js'),
+            utils.read_file('javascript/caret.js'),
         ])
         script = QWebEngineScript()
-        script.setInjectionPoint(QWebEngineScript.DocumentCreation)
+        # We can't use DocumentCreation here as WORKAROUND for
+        # https://bugreports.qt.io/browse/QTBUG-66011
+        script.setInjectionPoint(QWebEngineScript.DocumentReady)
         script.setSourceCode(js_code)
 
         page = self._widget.page()
@@ -597,6 +635,9 @@ class WebEngineTab(browsertab.AbstractTab):
 
     @pyqtSlot()
     def _restore_zoom(self):
+        if sip.isdeleted(self._widget):
+            # https://github.com/qutebrowser/qutebrowser/issues/3498
+            return
         if self._saved_zoom is None:
             return
         self.zoom.set_factor(self._saved_zoom)

qutebrowser/browser/webengine/webview.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/__init__.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/cache.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/certificateerror.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/cookies.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/http.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/mhtml.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Daniel Schadt
+# Copyright 2015-2018 Daniel Schadt
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/network/filescheme.py

@@ -1,7 +1,7 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
-# Copyright 2015-2017 Antoni Boucher (antoyo) <bouanto@zoho.com>
+# Copyright 2015-2018 Antoni Boucher (antoyo) <bouanto@zoho.com>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/network/networkmanager.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/network/networkreply.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # Based on the Eric5 helpviewer,
 # Copyright (c) 2009 - 2014 Detlev Offenbach <detlev@die-offenbachs.de>

qutebrowser/browser/webkit/network/schemehandler.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # Based on the Eric5 helpviewer,
 # Copyright (c) 2009 - 2014 Detlev Offenbach <detlev@die-offenbachs.de>

qutebrowser/browser/webkit/network/webkitqutescheme.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/rfc6266.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/tabhistory.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2015-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/webkitelem.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/webkithistory.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2015-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/webkitinspector.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2015-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2015-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/webkitsettings.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/webkittab.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2016-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2016-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -23,6 +23,10 @@ import re
 import functools
 import xml.etree.ElementTree
 
+import pygments
+import pygments.lexers
+import pygments.formatters
+
 import sip
 from PyQt5.QtCore import (pyqtSlot, Qt, QEvent, QUrl, QPoint, QTimer, QSizeF,
                           QSize)
@@ -50,6 +54,29 @@ class WebKitAction(browsertab.AbstractAction):
         """Save the current page."""
         raise browsertab.UnsupportedOperationError
 
+    def show_source(self):
+
+        def show_source_cb(source):
+            """Show source as soon as it's ready."""
+            # WORKAROUND for https://github.com/PyCQA/pylint/issues/491
+            # pylint: disable=no-member
+            lexer = pygments.lexers.HtmlLexer()
+            formatter = pygments.formatters.HtmlFormatter(
+                full=True, linenos='table')
+            # pylint: enable=no-member
+            highlighted = pygments.highlight(source, lexer, formatter)
+
+            tb = objreg.get('tabbed-browser', scope='window',
+                            window=self._tab.win_id)
+            new_tab = tb.tabopen(background=False, related=True)
+            # The original URL becomes the path of a view-source: URL
+            # (without a host), but query/fragment should stay.
+            url = QUrl('view-source:' + urlstr)
+            new_tab.set_html(highlighted, url)
+
+        urlstr = self._tab.url().toString(QUrl.RemoveUserInfo)
+        self._tab.dump_async(show_source_cb)
+
 
 class WebKitPrinting(browsertab.AbstractPrinting):
 
@@ -161,7 +188,7 @@ class WebKitCaret(browsertab.AbstractCaret):
 
         settings = self._widget.settings()
         settings.setAttribute(QWebSettings.CaretBrowsingEnabled, True)
-        self.selection_enabled = bool(self.selection())
+        self.selection_enabled = self._widget.hasSelection()
 
         if self._widget.isVisible():
             # Sometimes the caret isn't immediately visible, but unfocusing
@@ -174,7 +201,7 @@ class WebKitCaret(browsertab.AbstractCaret):
             #
             # Note: We can't use hasSelection() here, as that's always
             # true in caret mode.
-            if not self.selection():
+            if not self.selection_enabled:
                 self._widget.page().currentFrame().evaluateJavaScript(
                     utils.read_file('javascript/position_caret.js'))
 
@@ -327,23 +354,16 @@ class WebKitCaret(browsertab.AbstractCaret):
     def toggle_selection(self):
         self.selection_enabled = not self.selection_enabled
         mainwindow = objreg.get('main-window', scope='window',
-                                window=self._win_id)
+                                window=self._tab.win_id)
         mainwindow.status.set_mode_active(usertypes.KeyMode.caret, True)
 
     def drop_selection(self):
         self._widget.triggerPageAction(QWebPage.MoveToNextChar)
 
-    def has_selection(self):
+    def selection(self, callback):
-        return self._widget.hasSelection()
+        callback(self._widget.selectedText())
-
-    def selection(self, html=False):
-        if html:
-            return self._widget.selectedHtml()
-        return self._widget.selectedText()
 
     def follow_selected(self, *, tab=False):
-        if not self.has_selection():
-            return
         if QWebSettings.globalSettings().testAttribute(
                 QWebSettings.JavascriptEnabled):
             if tab:
@@ -351,7 +371,9 @@ class WebKitCaret(browsertab.AbstractCaret):
             self._tab.run_js_async(
                 'window.getSelection().anchorNode.parentNode.click()')
         else:
-            selection = self.selection(html=True)
+            selection = self._widget.selectedHtml()
+            if not selection:
+                return
             try:
                 selected_element = xml.etree.ElementTree.fromstring(
                     '<html>{}</html>'.format(selection)).find('a')
@@ -615,13 +637,13 @@ class WebKitTab(browsertab.AbstractTab):
             self._make_private(widget)
         self.history = WebKitHistory(self)
         self.scroller = WebKitScroller(self, parent=self)
-        self.caret = WebKitCaret(win_id=win_id, mode_manager=mode_manager,
+        self.caret = WebKitCaret(mode_manager=mode_manager,
                                  tab=self, parent=self)
-        self.zoom = WebKitZoom(win_id=win_id, parent=self)
+        self.zoom = WebKitZoom(tab=self, parent=self)
         self.search = WebKitSearch(parent=self)
         self.printing = WebKitPrinting()
-        self.elements = WebKitElements(self)
+        self.elements = WebKitElements(tab=self)
-        self.action = WebKitAction()
+        self.action = WebKitAction(tab=self)
         self._set_widget(widget)
         self._connect_signals()
         self.backend = usertypes.Backend.QtWebKit

qutebrowser/browser/webkit/webpage.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/browser/webkit/webview.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/commands/__init__.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/commands/argparser.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/commands/cmdexc.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/commands/cmdutils.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -105,7 +105,8 @@ class register:  # noqa: N801,N806 pylint: disable=invalid-name
         else:
             assert isinstance(self._name, str), self._name
             name = self._name
-        log.commands.vdebug("Registering command {}".format(name))
+        log.commands.vdebug("Registering command {} (from {}:{})".format(
+            name, func.__module__, func.__qualname__))
         if name in cmd_dict:
             raise ValueError("{} is already registered!".format(name))
         cmd = command.Command(name=name, instance=self._instance,

qutebrowser/commands/command.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -193,10 +193,10 @@ class Command:
         return False
 
     def _inspect_func(self):
-        """Inspect the function to get useful informations from it.
+        """Inspect the function to get useful information from it.
 
         Sets instance attributes (desc, type_conv, name_conv) based on the
-        informations.
+        information.
 
         Return:
             How many user-visible arguments the command has.

qutebrowser/commands/runners.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/commands/userscripts.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -446,3 +446,4 @@ def run_async(tab, cmd, *args, win_id, env, verbose=False):
     runner.prepare_run(cmd_path, *args, env=env, verbose=verbose)
     tab.dump_async(runner.store_html)
     tab.dump_async(runner.store_text, plain=True)
+    return runner

qutebrowser/completion/__init__.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/completer.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/completiondelegate.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #
@@ -203,9 +203,9 @@ class CompletionItemDelegate(QStyledItemDelegate):
             columns_to_filter = index.model().columns_to_filter(index)
             if index.column() in columns_to_filter and pattern:
                 repl = r'<span class="highlight">\g<0></span>'
-                text = re.sub(re.escape(pattern).replace(r'\ ', r'|'),
+                pat = html.escape(re.escape(pattern)).replace(r'\ ', r'|')
-                              repl, html.escape(self._opt.text),
+                txt = html.escape(self._opt.text)
-                              flags=re.IGNORECASE)
+                text = re.sub(pat, repl, txt, flags=re.IGNORECASE)
                 self._doc.setHtml(text)
             else:
                 self._doc.setPlainText(self._opt.text)

qutebrowser/completion/completionwidget.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/__init__.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/completionmodel.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2017 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
+# Copyright 2017-2018 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/configmodel.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/histcategory.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2017 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
+# Copyright 2017-2018 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/listcategory.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2017 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
+# Copyright 2017-2018 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/miscmodels.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/urlmodel.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2014-2017 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
+# Copyright 2014-2018 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
 #
 # This file is part of qutebrowser.
 #

qutebrowser/completion/models/util.py

@@ -1,6 +1,6 @@
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
-# Copyright 2017 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
+# Copyright 2017-2018 Ryan Roden-Corrent (rcorre) <ryan@rcorre.net>
 #
 # This file is part of qutebrowser.
 #