Merge branch 'master' of https://github.com/The-Compiler/qutebrowser into pintab

.travis.yml

@@ -45,6 +45,7 @@ matrix:
     - os: osx
       env: TESTENV=py35 OSX=elcapitan
       osx_image: xcode7.3
+  fast_finish: true
 
 cache:
   directories:

CHANGELOG.asciidoc

@@ -50,6 +50,8 @@ Added
 - New `cast` userscript to show a video on a Google Chromecast
 - New `:run-with-count` command which replaces the (undocumented) `:count:command` syntax.
 - New `:record-macro` (`q`) and `:run-macro` (`@`) commands for keyboard macros.
+- New `ui -> hide-scrollbar` setting to hide the scrollbar independently of the
+  `user-stylesheet` setting.
 
 Changed
 ~~~~~~~
@@ -143,6 +145,8 @@ Changed
 - Various functionality now works when javascript is disabled with QtWebKit
 - Various commands/settings taking `left`/`right`/`previous` arguments now take
   `prev`/`next`/`last-used` to remove ambiguity.
+- The `ui -> user-stylesheet` setting now only takes filenames, not CSS snippets
+- `ui -> window-title-format` now has a new `{backend} ` replacement
 
 Deprecated
 ~~~~~~~~~~
@@ -168,6 +172,8 @@ Removed
   thus removed.
 - All `--qt-*` arguments got replaced by `--qt-arg` and `--qt-flag` and thus
   removed.
+- The `-c`/`--confdir`, `--datadir` and `--cachedir` arguments got removed, as
+  `--basedir` should be sufficient.
 
 Fixed
 ~~~~~

doc/help/settings.asciidoc

@@ -44,7 +44,8 @@
 |<<ui-confirm-quit,confirm-quit>>|Whether to confirm quitting the application.
 |<<ui-zoom-text-only,zoom-text-only>>|Whether the zoom factor on a frame applies only to the text or to all content.
 |<<ui-frame-flattening,frame-flattening>>|Whether to  expand each subframe to its contents.
-|<<ui-user-stylesheet,user-stylesheet>>|User stylesheet to use (absolute filename, filename relative to the config directory or CSS string). Will expand environment variables.
+|<<ui-user-stylesheet,user-stylesheet>>|User stylesheet to use (absolute filename or filename relative to the config directory). Will expand environment variables.
+|<<ui-hide-scrollbar,hide-scrollbar>>|Hide the main scrollbar.
 |<<ui-css-media-type,css-media-type>>|Set the CSS media type.
 |<<ui-smooth-scrolling,smooth-scrolling>>|Whether to enable smooth scrolling for webpages.
 |<<ui-remove-finished-downloads,remove-finished-downloads>>|Number of milliseconds to wait before removing finished downloads. Will not be removed if value is -1.
@@ -618,11 +619,20 @@ This setting is only available with the QtWebKit backend.
 
 [[ui-user-stylesheet]]
 === user-stylesheet
-User stylesheet to use (absolute filename, filename relative to the config directory or CSS string). Will expand environment variables.
+User stylesheet to use (absolute filename or filename relative to the config directory). Will expand environment variables.
 
-Default: +pass:[html &gt; ::-webkit-scrollbar { width: 0px; height: 0px; }]+
+Default: empty
 
-This setting is only available with the QtWebKit backend.
+[[ui-hide-scrollbar]]
+=== hide-scrollbar
+Hide the main scrollbar.
+
+Valid values:
+
+ * +true+
+ * +false+
+
+Default: +pass:[true]+
 
 [[ui-css-media-type]]
 === css-media-type
@@ -677,6 +687,7 @@ The format to use for the window title. The following placeholders are defined:
 * `{id}`: The internal window ID of this window.
 * `{scroll_pos}`: The page scroll position.
 * `{host}`: The host of the current web page.
+* `{backend}`: Either 'webkit' or 'webengine'
 
 Default: +pass:[{perc}{title}{title_sep}qutebrowser]+
 
@@ -756,8 +767,6 @@ User agent to send. Empty to send the default.
 
 Default: empty
 
-This setting is only available with the QtWebKit backend.
-
 [[network-proxy]]
 === proxy
 The proxy to use.
@@ -1195,6 +1204,7 @@ The format to use for the tab title. The following placeholders are defined:
 * `{id}`: The internal tab ID of this tab.
 * `{scroll_pos}`: The page scroll position.
 * `{host}`: The host of the current web page.
+* `{backend}`: Either 'webkit' or 'webengine'
 
 Default: +pass:[{index}: {title}]+
 

doc/qutebrowser.1.asciidoc

@@ -38,17 +38,8 @@ show it.
 *-h*, *--help*::
     show this help message and exit
 
-*-c* 'CONFDIR', *--confdir* 'CONFDIR'::
-    Set config directory (empty for no config storage).
-
-*--datadir* 'DATADIR'::
-    Set data directory (empty for no data storage).
-
-*--cachedir* 'CACHEDIR'::
-    Set cache directory (empty for no cache storage).
-
 *--basedir* 'BASEDIR'::
-    Base directory for all storage. Other --*dir arguments are ignored if this is given.
+    Base directory for all storage.
 
 *-V*, *--version*::
     Show version and quit.
@@ -111,8 +102,8 @@ show it.
 *--no-err-windows*::
     Don't show any error windows (used for tests/smoke.py).
 
-*--qt-arg* 'QT_ARG'::
+*--qt-arg* 'NAME' 'VALUE'::
-    Pass an argument with a value to Qt.
+    Pass an argument with a value to Qt. For example, you can do `--qt-arg geometry 650x555+200+300` to set the window geometry.
 
 *--qt-flag* 'QT_FLAG'::
     Pass an argument to Qt as flag.

misc/requirements/requirements-codecov.txt

@@ -2,4 +2,4 @@
 
 codecov==2.0.5
 coverage==4.2
-requests==2.11.1
+requests==2.12.1

misc/requirements/requirements-cxfreeze.txt

@@ -1,3 +1,3 @@
 # This file is automatically generated by scripts/dev/recompile_requirements.py
 
-cx-Freeze==4.3.4
+cx-Freeze==4.3.4  # rq.filter: < 5.0.0

misc/requirements/requirements-cxfreeze.txt-raw

@@ -1 +1,5 @@
-cx_Freeze
+cx-Freeze < 5.0.0
+
+# We'll probably switch to PyInstaller soon, and 5.x doesn't install without a
+# compiler?
+#@ filter: cx-Freeze < 5.0.0

misc/requirements/requirements-flake8.txt

@@ -15,7 +15,7 @@ flake8-tuple==0.2.12
 mccabe==0.5.2
 packaging==16.8
 pep8-naming==0.4.1
-pycodestyle==2.1.0
+pycodestyle==2.2.0
 pydocstyle==1.1.1
 pyflakes==1.3.0
 pyparsing==2.1.10

misc/requirements/requirements-flake8.txt-raw

@@ -15,7 +15,7 @@ pydocstyle
 pyflakes
 
 # Pinned to 2.0.0 otherwise
-pycodestyle==2.1.0
+pycodestyle==2.2.0
 
 # Waiting until flake8-putty updated
 #@ filter: flake8 < 3.0.0

misc/requirements/requirements-pylint-master.txt

@@ -7,6 +7,6 @@ lazy-object-proxy==1.2.2
 mccabe==0.5.2
 -e git+https://github.com/PyCQA/pylint.git#egg=pylint
 ./scripts/dev/pylint_checkers
-requests==2.11.1
+requests==2.12.1
 six==1.10.0
 wrapt==1.10.8

misc/requirements/requirements-pylint.txt

@@ -7,7 +7,7 @@ lazy-object-proxy==1.2.2
 mccabe==0.5.2
 pylint==1.6.4
 ./scripts/dev/pylint_checkers
-requests==2.11.1
+requests==2.12.1
 six==1.10.0
 uritemplate==3.0.0
 uritemplate.py==3.0.2

misc/requirements/requirements-tests.txt

@@ -20,7 +20,7 @@ pytest==3.0.4
 pytest-bdd==2.18.1
 pytest-catchlog==1.2.2
 pytest-cov==2.4.0
-pytest-faulthandler==1.3.0
+pytest-faulthandler==1.3.1
 pytest-instafail==0.3.0
 pytest-mock==1.4.0
 pytest-qt==2.1.0

misc/requirements/requirements-tox.txt

@@ -2,5 +2,5 @@
 
 pluggy==0.4.0
 py==1.4.31
-tox==2.4.1
+tox==2.5.0
-virtualenv==15.0.3
+virtualenv==15.1.0

pytest.ini

@@ -17,7 +17,6 @@ markers =
     qtwebengine_todo: Features still missing with QtWebEngine
     qtwebengine_skip: Tests not applicable with QtWebEngine
     qtwebkit_skip: Tests not applicable with QtWebKit
-    qtwebengine_createWindow: Tests using createWindow with QtWebEngine (QTBUG-54419)
     qtwebengine_flaky: Tests which are flaky (and currently skipped) with QtWebEngine
     qtwebengine_osx_xfail: Tests which fail on OS X with QtWebEngine
     js_prompt: Tests needing to display a javascript prompt

qutebrowser/app.py

@@ -320,8 +320,8 @@ def _open_quickstart(args):
     Args:
         args: The argparse namespace.
     """
-    if args.datadir is not None or args.basedir is not None:
+    if args.basedir is not None:
-        # With --datadir or --basedir given, don't open quickstart.
+        # With --basedir given, don't open quickstart.
         return
     state_config = objreg.get('state-config')
     try:

qutebrowser/browser/adblock.py

@@ -29,7 +29,7 @@ import fnmatch
 from qutebrowser.browser import downloads
 from qutebrowser.config import config
 from qutebrowser.utils import objreg, standarddir, log, message
-from qutebrowser.commands import cmdutils, cmdexc
+from qutebrowser.commands import cmdutils
 
 
 def guess_zip_filename(zf):
@@ -113,17 +113,11 @@ class HostBlocker:
         self._done_count = 0
 
         data_dir = standarddir.data()
-        if data_dir is None:
+        self._local_hosts_file = os.path.join(data_dir, 'blocked-hosts')
-            self._local_hosts_file = None
-        else:
-            self._local_hosts_file = os.path.join(data_dir, 'blocked-hosts')
         self.on_config_changed()
 
         config_dir = standarddir.config()
-        if config_dir is None:
+        self._config_hosts_file = os.path.join(config_dir, 'blocked-hosts')
-            self._config_hosts_file = None
-        else:
-            self._config_hosts_file = os.path.join(config_dir, 'blocked-hosts')
 
         objreg.get('config').changed.connect(self.on_config_changed)
 
@@ -146,7 +140,7 @@ class HostBlocker:
         Return:
             True if a read was attempted, False otherwise
         """
-        if filename is None or not os.path.exists(filename):
+        if not os.path.exists(filename):
             return False
 
         try:
@@ -162,9 +156,6 @@ class HostBlocker:
         """Read hosts from the existing blocked-hosts file."""
         self._blocked_hosts = set()
 
-        if self._local_hosts_file is None:
-            return
-
         self._read_hosts_file(self._config_hosts_file,
                               self._config_blocked_hosts)
 
@@ -186,8 +177,6 @@ class HostBlocker:
         """
         self._read_hosts_file(self._config_hosts_file,
                               self._config_blocked_hosts)
-        if self._local_hosts_file is None:
-            raise cmdexc.CommandError("No data storage is configured!")
         self._blocked_hosts = set()
         self._done_count = 0
         urls = config.get('content', 'host-block-lists')
@@ -275,7 +264,7 @@ class HostBlocker:
     def on_config_changed(self):
         """Update files when the config changed."""
         urls = config.get('content', 'host-block-lists')
-        if urls is None and self._local_hosts_file is not None:
+        if urls is None:
             try:
                 os.remove(self._local_hosts_file)
             except FileNotFoundError:

qutebrowser/browser/browsertab.py

@@ -84,6 +84,7 @@ class TabData:
         inspector: The QWebInspector used for this webview.
         viewing_source: Set if we're currently showing a source view.
         override_target: Override for open_target for fake clicks (like hints).
+                         Only used for QtWebKit.
         pinned: Flag to pin the tab
     """
 

qutebrowser/browser/commands.py

@@ -1539,7 +1539,7 @@ class CommandDispatcher:
             message.error("Focused element is not editable!")
             return
 
-        text = elem.text(use_js=True)
+        text = elem.value()
         ed = editor.ExternalEditor(self._tabbed_browser)
         ed.editing_finished.connect(functools.partial(
             self.on_editing_finished, elem))
@@ -1566,7 +1566,7 @@ class CommandDispatcher:
             text: The new text to insert.
         """
         try:
-            elem.set_text(text, use_js=True)
+            elem.set_value(text)
         except webelem.Error as e:
             raise cmdexc.CommandError(str(e))
 
@@ -1625,7 +1625,7 @@ class CommandDispatcher:
         def single_cb(elem):
             """Click a single element."""
             if elem is None:
-                message.error("No element found!")
+                message.error("No element found with id {}!".format(value))
                 return
             try:
                 elem.click(target)

qutebrowser/browser/history.py

@@ -88,12 +88,8 @@ class Entry:
         if not url.isValid():
             raise ValueError("Invalid URL: {}".format(url.errorString()))
 
-        if atime.startswith('\0'):
+        # https://github.com/The-Compiler/qutebrowser/issues/670
-            log.init.debug(
+        atime = atime.lstrip('\0')
-                "Removing NUL bytes from entry {!r} - see "
-                "https://github.com/The-Compiler/qutebrowser/issues/"
-                "670".format(data))
-            atime = atime.lstrip('\0')
 
         if '-' in atime:
             atime, flags = atime.split('-')
@@ -129,7 +125,6 @@ class WebHistory(QObject):
 
     Attributes:
         history_dict: An OrderedDict of URLs read from the on-disk history.
-        _hist_dir: The directory to store the history in
         _lineparser: The AppendLineParser used to save the history.
         _new_history: A list of Entry items of the current session.
         _saved_count: How many HistoryEntries have been written to disk.
@@ -157,7 +152,6 @@ class WebHistory(QObject):
         super().__init__(parent)
         self._initial_read_started = False
         self._initial_read_done = False
-        self._hist_dir = hist_dir
         self._lineparser = lineparser.AppendLineParser(hist_dir, hist_name,
                                                        parent=self)
         self.history_dict = collections.OrderedDict()
@@ -183,12 +177,6 @@ class WebHistory(QObject):
             return
         self._initial_read_started = True
 
-        if self._hist_dir is None:
-            self._initial_read_done = True
-            self.async_read_done.emit()
-            assert not self._temp_history
-            return
-
         with self._lineparser.open():
             for line in self._lineparser:
                 yield

qutebrowser/browser/shared.py

@@ -146,9 +146,13 @@ def ignore_certificate_errors(url, errors, abort_on):
         """.strip())
         msg = err_template.render(url=url, errors=errors)
 
-        return message.ask(title="Certificate errors - continue?", text=msg,
+        ignore = message.ask(title="Certificate errors - continue?", text=msg,
-                           mode=usertypes.PromptMode.yesno, default=False,
+                             mode=usertypes.PromptMode.yesno, default=False,
-                           abort_on=abort_on)
+                             abort_on=abort_on)
+        if ignore is None:
+            # prompt aborted
+            ignore = False
+        return ignore
     elif ssl_strict is False:
         log.webview.debug("ssl-strict is False, only warning about errors")
         for err in errors:
@@ -222,3 +226,19 @@ def get_tab(win_id, target):
     tabbed_browser = objreg.get('tabbed-browser', scope='window',
                                 window=win_id)
     return tabbed_browser.tabopen(url=None, background=bg_tab)
+
+
+def get_user_stylesheet():
+    """Get the combined user-stylesheet."""
+    filename = config.get('ui', 'user-stylesheet')
+
+    if filename is None:
+        css = ''
+    else:
+        with open(filename, 'r', encoding='utf-8') as f:
+            css = f.read()
+
+    if config.get('ui', 'hide-scrollbar'):
+        css += '\nhtml > ::-webkit-scrollbar { width: 0px; height: 0px; }'
+
+    return css

qutebrowser/browser/urlmarks.py

@@ -73,8 +73,7 @@ class UrlMarkManager(QObject):
 
     Attributes:
         marks: An OrderedDict of all quickmarks/bookmarks.
-        _lineparser: The LineParser used for the marks, or None
+        _lineparser: The LineParser used for the marks
-                     (when qutebrowser is started with -c '').
 
     Signals:
         changed: Emitted when anything changed.
@@ -91,10 +90,6 @@ class UrlMarkManager(QObject):
         super().__init__(parent)
 
         self.marks = collections.OrderedDict()
-        self._lineparser = None
-
-        if standarddir.config() is None:
-            return
 
         self._init_lineparser()
         for line in self._lineparser:
@@ -115,10 +110,8 @@ class UrlMarkManager(QObject):
 
     def save(self):
         """Save the marks to disk."""
-        if self._lineparser is not None:
+        self._lineparser.data = [' '.join(tpl) for tpl in self.marks.items()]
-            self._lineparser.data = [' '.join(tpl)
+        self._lineparser.save()
-                                     for tpl in self.marks.items()]
-            self._lineparser.save()
 
     def delete(self, key):
         """Delete a quickmark/bookmark.

qutebrowser/browser/webelem.py

@@ -87,7 +87,7 @@ class AbstractWebElement(collections.abc.MutableMapping):
         raise NotImplementedError
 
     def __str__(self):
-        return self.text()
+        raise NotImplementedError
 
     def __getitem__(self, key):
         raise NotImplementedError
@@ -138,24 +138,12 @@ class AbstractWebElement(collections.abc.MutableMapping):
         """Get the full HTML representation of this element."""
         raise NotImplementedError
 
-    def text(self, *, use_js=False):
+    def value(self):
-        """Get the plain text content for this element.
+        """Get the value attribute for this element."""
-
-        Args:
-            use_js: Whether to use javascript if the element isn't
-                    content-editable.
-        """
-        # FIXME:qtwebengine what to do about use_js with WebEngine?
         raise NotImplementedError
 
-    def set_text(self, text, *, use_js=False):
+    def set_value(self, value):
-        """Set the given plain text.
+        """Set the element value."""
-
-        Args:
-            use_js: Whether to use javascript if the element isn't
-                    content-editable.
-        """
-        # FIXME:qtwebengine what to do about use_js with WebEngine?
         raise NotImplementedError
 
     def insert_text(self, text):
@@ -338,6 +326,8 @@ class AbstractWebElement(collections.abc.MutableMapping):
         """Simulate a click on the element."""
         # FIXME:qtwebengine do we need this?
         # self._widget.setFocus()
+
+        # For QtWebKit
         self._tab.data.override_target = click_target
 
         pos = self._mouse_pos()
@@ -345,20 +335,24 @@ class AbstractWebElement(collections.abc.MutableMapping):
         log.webelem.debug("Sending fake click to {!r} at position {} with "
                           "target {}".format(self, pos, click_target))
 
-        if click_target in [usertypes.ClickTarget.tab,
+        modifiers = {
-                            usertypes.ClickTarget.tab_bg,
+            usertypes.ClickTarget.normal: Qt.NoModifier,
-                            usertypes.ClickTarget.window]:
+            usertypes.ClickTarget.window: Qt.AltModifier | Qt.ShiftModifier,
-            modifiers = Qt.ControlModifier
+            usertypes.ClickTarget.tab: Qt.ControlModifier,
+            usertypes.ClickTarget.tab_bg: Qt.ControlModifier,
+        }
+        if config.get('tabs', 'background-tabs'):
+            modifiers[usertypes.ClickTarget.tab] |= Qt.ShiftModifier
         else:
-            modifiers = Qt.NoModifier
+            modifiers[usertypes.ClickTarget.tab_bg] |= Qt.ShiftModifier
 
         events = [
             QMouseEvent(QEvent.MouseMove, pos, Qt.NoButton, Qt.NoButton,
                         Qt.NoModifier),
             QMouseEvent(QEvent.MouseButtonPress, pos, Qt.LeftButton,
-                        Qt.LeftButton, modifiers),
+                        Qt.LeftButton, modifiers[click_target]),
             QMouseEvent(QEvent.MouseButtonRelease, pos, Qt.LeftButton,
-                        Qt.NoButton, modifiers),
+                        Qt.NoButton, modifiers[click_target]),
         ]
 
         for evt in events:

qutebrowser/browser/webengine/interceptor.py

@@ -23,6 +23,7 @@
 from PyQt5.QtWebEngineCore import QWebEngineUrlRequestInterceptor
 # pylint: enable=no-name-in-module,import-error,useless-suppression
 
+from qutebrowser.config import config
 from qutebrowser.browser import shared
 from qutebrowser.utils import utils, log
 
@@ -64,3 +65,7 @@ class RequestInterceptor(QWebEngineUrlRequestInterceptor):
 
         for header, value in shared.custom_headers():
             info.setHttpHeader(header, value)
+
+        user_agent = config.get('network', 'user-agent')
+        if user_agent is not None:
+            info.setHttpHeader(b'User-Agent', user_agent.encode('ascii'))

qutebrowser/browser/webengine/webengineelem.py

@@ -37,6 +37,9 @@ class WebEngineElement(webelem.AbstractWebElement):
         self._id = js_dict['id']
         self._js_dict = js_dict
 
+    def __str__(self):
+        return self._js_dict.get('text', '')
+
     def __eq__(self, other):
         if not isinstance(other, WebEngineElement):
             return NotImplemented
@@ -87,27 +90,11 @@ class WebEngineElement(webelem.AbstractWebElement):
         """Get the full HTML representation of this element."""
         return self._js_dict['outer_xml']
 
-    def text(self, *, use_js=False):
+    def value(self):
-        """Get the plain text content for this element.
+        return self._js_dict['value']
 
-        Args:
+    def set_value(self, value):
-            use_js: Whether to use javascript if the element isn't
+        js_code = javascript.assemble('webelem', 'set_value', self._id, value)
-                    content-editable.
-        """
-        if use_js:
-            # FIXME:qtwebengine what to do about use_js with WebEngine?
-            log.stub('with use_js=True')
-        return self._js_dict.get('text', '')
-
-    def set_text(self, text, *, use_js=False):
-        """Set the given plain text.
-
-        Args:
-            use_js: Whether to use javascript if the element isn't
-                    content-editable.
-        """
-        # FIXME:qtwebengine what to do about use_js with WebEngine?
-        js_code = javascript.assemble('webelem', 'set_text', self._id, text)
         self._tab.run_js_async(js_code)
 
     def insert_text(self, text):

qutebrowser/browser/webengine/webenginesettings.py

@@ -27,11 +27,13 @@ Module attributes:
 import os
 
 # pylint: disable=no-name-in-module,import-error,useless-suppression
-from PyQt5.QtWebEngineWidgets import QWebEngineSettings
+from PyQt5.QtWebEngineWidgets import (QWebEngineSettings, QWebEngineProfile,
+                                      QWebEngineScript)
 # pylint: enable=no-name-in-module,import-error,useless-suppression
 
+from qutebrowser.browser import shared
 from qutebrowser.config import websettings, config
-from qutebrowser.utils import objreg, utils
+from qutebrowser.utils import objreg, utils, standarddir, javascript
 
 
 class Attribute(websettings.Attribute):
@@ -63,19 +65,59 @@ class StaticSetter(websettings.StaticSetter):
     GLOBAL_SETTINGS = QWebEngineSettings.globalSettings
 
 
+def _init_stylesheet(profile):
+    """Initialize custom stylesheets.
+
+    Mostly inspired by QupZilla:
+    https://github.com/QupZilla/qupzilla/blob/v2.0/src/lib/app/mainapplication.cpp#L1063-L1101
+    https://github.com/QupZilla/qupzilla/blob/v2.0/src/lib/tools/scripts.cpp#L119-L132
+
+    FIXME:qtwebengine Use QWebEngineStyleSheet once that's available
+    https://codereview.qt-project.org/#/c/148671/
+    """
+    old_script = profile.scripts().findScript('_qute_stylesheet')
+    if not old_script.isNull():
+        profile.scripts().remove(old_script)
+
+    css = shared.get_user_stylesheet()
+    source = """
+        (function() {{
+            var css = document.createElement('style');
+            css.setAttribute('type', 'text/css');
+            css.appendChild(document.createTextNode('{}'));
+            document.getElementsByTagName('head')[0].appendChild(css);
+        }})()
+    """.format(javascript.string_escape(css))
+
+    script = QWebEngineScript()
+    script.setName('_qute_stylesheet')
+    script.setInjectionPoint(QWebEngineScript.DocumentReady)
+    script.setWorldId(QWebEngineScript.ApplicationWorld)
+    script.setRunsOnSubFrames(True)
+    script.setSourceCode(source)
+    profile.scripts().insert(script)
+
+
 def update_settings(section, option):
     """Update global settings when qwebsettings changed."""
     websettings.update_mappings(MAPPINGS, section, option)
+    profile = QWebEngineProfile.defaultProfile()
+    if section == 'ui' and option in ['hide-scrollbar', 'user-stylesheet']:
+        _init_stylesheet(profile)
 
 
 def init():
     """Initialize the global QWebSettings."""
-    # FIXME:qtwebengine set paths in profile
-
     if config.get('general', 'developer-extras'):
         # FIXME:qtwebengine Make sure we call globalSettings *after* this...
         os.environ['QTWEBENGINE_REMOTE_DEBUGGING'] = str(utils.random_port())
 
+    profile = QWebEngineProfile.defaultProfile()
+    profile.setCachePath(os.path.join(standarddir.cache(), 'webengine'))
+    profile.setPersistentStoragePath(
+        os.path.join(standarddir.data(), 'webengine'))
+    _init_stylesheet(profile)
+
     websettings.init_mappings(MAPPINGS)
     objreg.get('config').changed.connect(update_settings)
 
@@ -98,18 +140,12 @@ def shutdown():
 # - PictographFont
 #
 # TODO settings on profile:
-# - cachePath
-# - httpAcceptLanguage
 # - httpCacheMaximumSize
-# - httpUserAgent
 # - persistentCookiesPolicy
 # - offTheRecord
-# - persistentStoragePath
 #
 # TODO settings elsewhere:
 # - proxy
-# - custom headers
-# - ssl-strict
 
 MAPPINGS = {
     'content': {

qutebrowser/browser/webengine/webenginetab.py

@@ -469,7 +469,6 @@ class WebEngineTab(browsertab.AbstractTab):
         self._set_widget(widget)
         self._connect_signals()
         self.backend = usertypes.Backend.QtWebEngine
-        # init js stuff
         self._init_js()
         self._child_event_filter = None
         self.needs_qtbug54419_workaround = False

qutebrowser/browser/webengine/webview.py

@@ -74,18 +74,19 @@ class WebEngineView(QWebEngineView):
         """
         debug_type = debug.qenum_key(QWebEnginePage, wintype)
         background_tabs = config.get('tabs', 'background-tabs')
-        override_target = self._tabdata.override_target
 
         log.webview.debug("createWindow with type {}, background_tabs "
-                          "{}, override_target {}".format(
+                          "{}".format(debug_type, background_tabs))
-                              debug_type, background_tabs, override_target))
 
-        if override_target is not None:
+        try:
-            target = override_target
+            background_tab_wintype = QWebEnginePage.WebBrowserBackgroundTab
-            self._tabdata.override_target = None
+        except AttributeError:
-        elif wintype == QWebEnginePage.WebBrowserWindow:
+            # This is unavailable with an older PyQt, but we still might get
-            log.webview.debug("createWindow with WebBrowserWindow - when does "
+            # this with a newer Qt...
-                              "this happen?!")
+            background_tab_wintype = 0x0003
+
+        if wintype == QWebEnginePage.WebBrowserWindow:
+            # Shift-Alt-Click
             target = usertypes.ClickTarget.window
         elif wintype == QWebEnginePage.WebDialog:
             log.webview.warning("{} requested, but we don't support "
@@ -93,12 +94,12 @@ class WebEngineView(QWebEngineView):
             target = usertypes.ClickTarget.tab
         elif wintype == QWebEnginePage.WebBrowserTab:
             # Middle-click / Ctrl-Click with Shift
+            # FIXME:qtwebengine this also affects target=_blank links...
             if background_tabs:
                 target = usertypes.ClickTarget.tab
             else:
                 target = usertypes.ClickTarget.tab_bg
-        elif (hasattr(QWebEnginePage, 'WebBrowserBackgroundTab') and
+        elif wintype == background_tab_wintype:
-              wintype == QWebEnginePage.WebBrowserBackgroundTab):
             # Middle-click / Ctrl-Click
             if background_tabs:
                 target = usertypes.ClickTarget.tab_bg

qutebrowser/browser/webkit/cache.py

@@ -32,23 +32,14 @@ class DiskCache(QNetworkDiskCache):
 
     """Disk cache which sets correct cache dir and size.
 
-    If the cache is deactivated via the command line argument --cachedir="",
-    both attributes _cache_dir and _http_cache_dir are set to None.
-
     Attributes:
         _activated: Whether the cache should be used.
-        _cache_dir: The base directory for cache files (standarddir.cache()) or
+        _cache_dir: The base directory for cache files (standarddir.cache())
-                    None.
-        _http_cache_dir: the HTTP subfolder in _cache_dir or None.
     """
 
     def __init__(self, cache_dir, parent=None):
         super().__init__(parent)
         self._cache_dir = cache_dir
-        if cache_dir is None:
-            self._http_cache_dir = None
-        else:
-            self._http_cache_dir = os.path.join(cache_dir, 'http')
         self._maybe_activate()
         objreg.get('config').changed.connect(self.on_config_changed)
 
@@ -59,12 +50,11 @@ class DiskCache(QNetworkDiskCache):
 
     def _maybe_activate(self):
         """Activate/deactivate the cache based on the config."""
-        if (config.get('general', 'private-browsing') or
+        if config.get('general', 'private-browsing'):
-                self._cache_dir is None):
             self._activated = False
         else:
             self._activated = True
-            self.setCacheDirectory(self._http_cache_dir)
+            self.setCacheDirectory(os.path.join(self._cache_dir, 'http'))
             self.setMaximumCacheSize(config.get('storage', 'cache-size'))
 
     @pyqtSlot(str, str)

qutebrowser/browser/webkit/webkitelem.py

@@ -46,6 +46,10 @@ class WebKitElement(webelem.AbstractWebElement):
             raise IsNullError('{} is a null element!'.format(elem))
         self._elem = elem
 
+    def __str__(self):
+        self._check_vanished()
+        return self._elem.toPlainText()
+
     def __eq__(self, other):
         if not isinstance(other, WebKitElement):
             return NotImplemented
@@ -116,22 +120,19 @@ class WebKitElement(webelem.AbstractWebElement):
         self._check_vanished()
         return self._elem.toOuterXml()
 
-    def text(self, *, use_js=False):
+    def value(self):
         self._check_vanished()
-        if self.is_content_editable() or not use_js:
+        return self._elem.evaluateJavaScript('this.value')
-            return self._elem.toPlainText()
-        else:
-            return self._elem.evaluateJavaScript('this.value')
 
-    def set_text(self, text, *, use_js=False):
+    def set_value(self, value):
         self._check_vanished()
-        if self.is_content_editable() or not use_js:
+        if self.is_content_editable():
             log.webelem.debug("Filling {!r} via set_text.".format(self))
-            self._elem.setPlainText(text)
+            self._elem.setPlainText(value)
         else:
             log.webelem.debug("Filling {!r} via javascript.".format(self))
-            text = javascript.string_escape(text)
+            value = javascript.string_escape(value)
-            self._elem.evaluateJavaScript("this.value='{}'".format(text))
+            self._elem.evaluateJavaScript("this.value='{}'".format(value))
 
     def insert_text(self, text):
         self._check_vanished()

qutebrowser/browser/webkit/webkitsettings.py

@@ -29,7 +29,8 @@ import os.path
 from PyQt5.QtWebKit import QWebSettings
 
 from qutebrowser.config import config, websettings
-from qutebrowser.utils import standarddir, objreg
+from qutebrowser.utils import standarddir, objreg, urlutils
+from qutebrowser.browser import shared
 
 
 class Attribute(websettings.Attribute):
@@ -80,14 +81,24 @@ class CookiePolicy(websettings.Base):
             self.MAPPING[value])
 
 
+def _set_user_stylesheet():
+    """Set the generated user-stylesheet."""
+    stylesheet = shared.get_user_stylesheet().encode('utf-8')
+    url = urlutils.data_url('text/css;charset=utf-8', stylesheet)
+    QWebSettings.globalSettings().setUserStyleSheetUrl(url)
+
+
 def update_settings(section, option):
     """Update global settings when qwebsettings changed."""
-    cache_path = standarddir.cache()
     if (section, option) == ('general', 'private-browsing'):
+        cache_path = standarddir.cache()
         if config.get('general', 'private-browsing') or cache_path is None:
             QWebSettings.setIconDatabasePath('')
         else:
             QWebSettings.setIconDatabasePath(cache_path)
+    elif section == 'ui' and option in ['hide-scrollbar', 'user-stylesheet']:
+        _set_user_stylesheet()
+
     websettings.update_mappings(MAPPINGS, section, option)
 
 
@@ -95,20 +106,20 @@ def init():
     """Initialize the global QWebSettings."""
     cache_path = standarddir.cache()
     data_path = standarddir.data()
-    if config.get('general', 'private-browsing') or cache_path is None:
+    if config.get('general', 'private-browsing'):
         QWebSettings.setIconDatabasePath('')
     else:
         QWebSettings.setIconDatabasePath(cache_path)
-    if cache_path is not None:
+
-        QWebSettings.setOfflineWebApplicationCachePath(
+    QWebSettings.setOfflineWebApplicationCachePath(
-            os.path.join(cache_path, 'application-cache'))
+        os.path.join(cache_path, 'application-cache'))
-    if data_path is not None:
+    QWebSettings.globalSettings().setLocalStoragePath(
-        QWebSettings.globalSettings().setLocalStoragePath(
+        os.path.join(data_path, 'local-storage'))
-            os.path.join(data_path, 'local-storage'))
+    QWebSettings.setOfflineStoragePath(
-        QWebSettings.setOfflineStoragePath(
+        os.path.join(data_path, 'offline-storage'))
-            os.path.join(data_path, 'offline-storage'))
 
     websettings.init_mappings(MAPPINGS)
+    _set_user_stylesheet()
     objreg.get('config').changed.connect(update_settings)
 
 
@@ -205,9 +216,7 @@ MAPPINGS = {
             Attribute(QWebSettings.ZoomTextOnly),
         'frame-flattening':
             Attribute(QWebSettings.FrameFlatteningEnabled),
-        'user-stylesheet':
+        # user-stylesheet is handled separately
-            Setter(getter=QWebSettings.userStyleSheetUrl,
-                   setter=QWebSettings.setUserStyleSheetUrl),
         'css-media-type':
             NullStringSetter(getter=QWebSettings.cssMediaType,
                              setter=QWebSettings.setCSSMediaType),

qutebrowser/commands/userscripts.py

@@ -408,15 +408,11 @@ def run_async(tab, cmd, *args, win_id, env, verbose=False):
     user_agent = config.get('network', 'user-agent')
     if user_agent is not None:
         env['QUTE_USER_AGENT'] = user_agent
-    config_dir = standarddir.config()
+
-    if config_dir is not None:
+    env['QUTE_CONFIG_DIR'] = standarddir.config()
-        env['QUTE_CONFIG_DIR'] = config_dir
+    env['QUTE_DATA_DIR'] = standarddir.data()
-    data_dir = standarddir.data()
+    env['QUTE_DOWNLOAD_DIR'] = downloads.download_dir()
-    if data_dir is not None:
+
-        env['QUTE_DATA_DIR'] = data_dir
-    download_dir = downloads.download_dir()
-    if download_dir is not None:
-        env['QUTE_DOWNLOAD_DIR'] = download_dir
     cmd_path = os.path.expanduser(cmd)
 
     # if cmd is not given as an absolute path, look it up

qutebrowser/config/config.py

@@ -160,19 +160,18 @@ def _init_main_config(parent=None):
         sys.exit(usertypes.Exit.err_config)
     else:
         objreg.register('config', config_obj)
-        if standarddir.config() is not None:
+        filename = os.path.join(standarddir.config(), 'qutebrowser.conf')
-            filename = os.path.join(standarddir.config(), 'qutebrowser.conf')
+        save_manager = objreg.get('save-manager')
-            save_manager = objreg.get('save-manager')
+        save_manager.add_saveable(
-            save_manager.add_saveable(
+            'config', config_obj.save, config_obj.changed,
-                'config', config_obj.save, config_obj.changed,
+            config_opt=('general', 'auto-save-config'), filename=filename)
-                config_opt=('general', 'auto-save-config'), filename=filename)
+        for sect in config_obj.sections.values():
-            for sect in config_obj.sections.values():
+            for opt in sect.values.values():
-                for opt in sect.values.values():
+                if opt.values['conf'] is None:
-                    if opt.values['conf'] is None:
+                    # Option added to built-in defaults but not in user's
-                        # Option added to built-in defaults but not in user's
+                    # config yet
-                        # config yet
+                    save_manager.save('config', explicit=True, force=True)
-                        save_manager.save('config', explicit=True, force=True)
+                    return
-                        return
 
 
 def _init_key_config(parent):
@@ -197,13 +196,12 @@ def _init_key_config(parent):
         sys.exit(usertypes.Exit.err_key_config)
     else:
         objreg.register('key-config', key_config)
-        if standarddir.config() is not None:
+        save_manager = objreg.get('save-manager')
-            save_manager = objreg.get('save-manager')
+        filename = os.path.join(standarddir.config(), 'keys.conf')
-            filename = os.path.join(standarddir.config(), 'keys.conf')
+        save_manager.add_saveable(
-            save_manager.add_saveable(
+            'key-config', key_config.save, key_config.config_dirty,
-                'key-config', key_config.save, key_config.config_dirty,
+            config_opt=('general', 'auto-save-config'), filename=filename,
-                config_opt=('general', 'auto-save-config'), filename=filename,
+            dirty=key_config.is_dirty)
-                dirty=key_config.is_dirty)
 
 
 def _init_misc():
@@ -237,10 +235,7 @@ def _init_misc():
     # This fixes one of the corruption issues here:
     # https://github.com/The-Compiler/qutebrowser/issues/515
 
-    if standarddir.config() is None:
+    path = os.path.join(standarddir.config(), 'qsettings')
-        path = os.devnull
-    else:
-        path = os.path.join(standarddir.config(), 'qsettings')
     for fmt in [QSettings.NativeFormat, QSettings.IniFormat]:
         QSettings.setPath(fmt, QSettings.UserScope, path)
 
@@ -442,6 +437,11 @@ class ConfigManager(QObject):
         ('fonts', 'hints'): _transform_hint_font,
         ('completion', 'show'):
             _get_value_transformer({'false': 'never', 'true': 'always'}),
+        ('ui', 'user-stylesheet'):
+            _get_value_transformer({
+                'html > ::-webkit-scrollbar { width: 0px; height: 0px; }': '',
+                '::-webkit-scrollbar { width: 0px; height: 0px; }': '',
+            }),
     }
 
     changed = pyqtSignal(str, str)
@@ -659,15 +659,11 @@ class ConfigManager(QObject):
     def read(self, configdir, fname, relaxed=False):
         """Read the config from the given directory/file."""
         self._fname = fname
-        if configdir is None:
+        self._configdir = configdir
-            self._configdir = None
+        parser = ini.ReadConfigParser(configdir, fname)
-            self._initialized = True
+        self._from_cp(parser, relaxed)
-        else:
+        self._initialized = True
-            self._configdir = configdir
+        self._validate_all()
-            parser = ini.ReadConfigParser(configdir, fname)
-            self._from_cp(parser, relaxed)
-            self._initialized = True
-            self._validate_all()
 
     def items(self, sectname, raw=True):
         """Get a list of (optname, value) tuples for a section.
@@ -884,8 +880,6 @@ class ConfigManager(QObject):
 
     def save(self):
         """Save the config file."""
-        if self._configdir is None:
-            return
         configfile = os.path.join(self._configdir, self._fname)
         log.destroy.debug("Saving config to {}".format(configfile))
         with qtutils.savefile_open(configfile) as f:

qutebrowser/config/configdata.py

@@ -323,13 +323,13 @@ def data(readonly=False):
              "page."),
 
             ('user-stylesheet',
-             SettingValue(typ.UserStyleSheet(none_ok=True),
+             SettingValue(typ.File(none_ok=True), ''),
-                          'html > ::-webkit-scrollbar { width: 0px; '
+             "User stylesheet to use (absolute filename or filename relative "
-                          'height: 0px; }',
+             "to the config directory). Will expand environment variables."),
-                          backends=[usertypes.Backend.QtWebKit]),
+
-             "User stylesheet to use (absolute filename, filename relative to "
+            ('hide-scrollbar',
-             "the config directory or CSS string). Will expand environment "
+             SettingValue(typ.Bool(), 'true'),
-             "variables."),
+             "Hide the main scrollbar."),
 
             ('css-media-type',
              SettingValue(typ.String(none_ok=True), '',
@@ -356,7 +356,8 @@ def data(readonly=False):
             ('window-title-format',
              SettingValue(typ.FormatString(fields=['perc', 'perc_raw', 'title',
                                                    'title_sep', 'id',
-                                                   'scroll_pos', 'host']),
+                                                   'scroll_pos', 'host',
+                                                   'backend']),
                           '{perc}{title}{title_sep}qutebrowser'),
              "The format to use for the window title. The following "
              "placeholders are defined:\n\n"
@@ -367,7 +368,8 @@ def data(readonly=False):
              "otherwise.\n"
              "* `{id}`: The internal window ID of this window.\n"
              "* `{scroll_pos}`: The page scroll position.\n"
-             "* `{host}`: The host of the current web page."),
+             "* `{host}`: The host of the current web page.\n"
+             "* `{backend}`: Either 'webkit' or 'webengine'"),
 
             ('modal-js-dialog',
              SettingValue(typ.Bool(), 'false'),
@@ -413,8 +415,7 @@ def data(readonly=False):
              "Send the Referer header"),
 
             ('user-agent',
-             SettingValue(typ.UserAgent(none_ok=True), '',
+             SettingValue(typ.UserAgent(none_ok=True), ''),
-                          backends=[usertypes.Backend.QtWebKit]),
              "User agent to send. Empty to send the default."),
 
             ('proxy',
@@ -681,7 +682,8 @@ def data(readonly=False):
              "* `{index}`: The index of this tab.\n"
              "* `{id}`: The internal tab ID of this tab.\n"
              "* `{scroll_pos}`: The page scroll position.\n"
-             "* `{host}`: The host of the current web page."),
+             "* `{host}`: The host of the current web page.\n"
+             "* `{backend}`: Either 'webkit' or 'webengine'"),
 
             ('title-format-pinned',
              SettingValue(typ.FormatString(

qutebrowser/config/configtypes.py

@@ -22,7 +22,6 @@
 import re
 import json
 import shlex
-import base64
 import codecs
 import os.path
 import itertools
@@ -859,9 +858,7 @@ class File(BaseType):
         value = os.path.expanduser(value)
         value = os.path.expandvars(value)
         if not os.path.isabs(value):
-            cfgdir = standarddir.config()
+            value = os.path.join(standarddir.config(), value)
-            assert cfgdir is not None
-            value = os.path.join(cfgdir, value)
         return value
 
     def validate(self, value):
@@ -872,12 +869,7 @@ class File(BaseType):
         value = os.path.expandvars(value)
         try:
             if not os.path.isabs(value):
-                cfgdir = standarddir.config()
+                value = os.path.join(standarddir.config(), value)
-                if cfgdir is None:
-                    raise configexc.ValidationError(
-                        value, "must be an absolute path when not using a "
-                        "config directory!")
-                value = os.path.join(cfgdir, value)
                 not_isfile_message = ("must be a valid path relative to the "
                                       "config directory!")
             else:
@@ -1172,48 +1164,6 @@ class Encoding(BaseType):
             raise configexc.ValidationError(value, "is not a valid encoding!")
 
 
-class UserStyleSheet(File):
-
-    """QWebSettings UserStyleSheet."""
-
-    def transform(self, value):
-        if not value:
-            return None
-
-        if standarddir.config() is None:
-            # We can't call super().transform() here as this counts on the
-            # validation previously ensuring that we don't have a relative path
-            # when starting with -c "".
-            path = None
-        else:
-            path = super().transform(value)
-
-        if path is not None and os.path.exists(path):
-            return QUrl.fromLocalFile(path)
-        else:
-            data = base64.b64encode(value.encode('utf-8')).decode('ascii')
-            return QUrl("data:text/css;charset=utf-8;base64,{}".format(data))
-
-    def validate(self, value):
-        self._basic_validation(value)
-        if not value:
-            return
-        value = os.path.expandvars(value)
-        value = os.path.expanduser(value)
-        try:
-            super().validate(value)
-        except configexc.ValidationError:
-            try:
-                if not os.path.isabs(value):
-                    # probably a CSS, so we don't handle it as filename.
-                    # FIXME We just try if it is encodable, maybe we should
-                    # validate CSS?
-                    # https://github.com/The-Compiler/qutebrowser/issues/115
-                    value.encode('utf-8')
-            except UnicodeEncodeError as e:
-                raise configexc.ValidationError(value, str(e))
-
-
 class AutoSearch(BaseType):
 
     """Whether to start a search when something else than a URL is entered."""
@@ -1476,6 +1426,11 @@ class UserAgent(BaseType):
 
     def validate(self, value):
         self._basic_validation(value)
+        try:
+            value.encode('ascii')
+        except UnicodeEncodeError as e:
+            msg = "User-Agent contains non-ascii characters: {}".format(e)
+            raise configexc.ValidationError(value, msg)
 
     # To update the following list of user agents, run the script 'ua_fetch.py'
     # Vim-protip: Place your cursor below this comment and run

qutebrowser/config/parsers/ini.py

@@ -47,15 +47,12 @@ class ReadConfigParser(configparser.ConfigParser):
         self.optionxform = lambda opt: opt  # be case-insensitive
         self._configdir = configdir
         self._fname = fname
-        if self._configdir is None:
-            self._configfile = None
-            return
         self._configfile = os.path.join(self._configdir, fname)
+
         if not os.path.isfile(self._configfile):
             return
         log.init.debug("Reading config from {}".format(self._configfile))
-        if self._configfile is not None:
+        self.read(self._configfile, encoding='utf-8')
-            self.read(self._configfile, encoding='utf-8')
 
     def __repr__(self):
         return utils.get_repr(self, constructor=True,

qutebrowser/config/parsers/keyconf.py

@@ -89,11 +89,9 @@ class KeyConfigParser(QObject):
         self._cur_command = None
         # Mapping of section name(s) to key binding -> command dicts.
         self.keybindings = collections.OrderedDict()
-        if configdir is None:
+        self._configfile = os.path.join(configdir, fname)
-            self._configfile = None
+
-        else:
+        if not os.path.exists(self._configfile):
-            self._configfile = os.path.join(configdir, fname)
-        if self._configfile is None or not os.path.exists(self._configfile):
             self._load_default()
         else:
             self._read(relaxed)
@@ -143,8 +141,6 @@ class KeyConfigParser(QObject):
 
     def save(self):
         """Save the key config file."""
-        if self._configfile is None:
-            return
         log.destroy.debug("Saving key config to {}".format(self._configfile))
         with qtutils.savefile_open(self._configfile, encoding='utf-8') as f:
             data = str(self)

qutebrowser/javascript/webelem.js

@@ -34,6 +34,7 @@ window._qutebrowser.webelem = (function() {
         var out = {
             "id": id,
             "text": elem.text,
+            "value": elem.value,
             "tag_name": elem.tagName,
             "outer_xml": elem.outerHTML,
             "class_name": elem.className,
@@ -129,8 +130,8 @@ window._qutebrowser.webelem = (function() {
         return serialize_elem(elem);
     };
 
-    funcs.set_text = function(id, text) {
+    funcs.set_value = function(id, value) {
-        elements[id].value = text;
+        elements[id].value = value;
     };
 
     funcs.insert_text = function(id, text) {

qutebrowser/mainwindow/mainwindow.py

@@ -232,8 +232,8 @@ class MainWindow(QWidget):
             width = self.width() - 2 * padding
             left = padding
         else:
-            width = size_hint.width()
+            width = min(size_hint.width(), self.width() - 2 * padding)
-            left = (self.width() - size_hint.width()) / 2 if centered else 0
+            left = (self.width() - width) / 2 if centered else 0
 
         height_padding = 20
         status_position = config.get('ui', 'status-position')

qutebrowser/mainwindow/tabwidget.py

@@ -149,6 +149,7 @@ class TabWidget(QTabWidget):
         fields['title'] = page_title
         fields['title_sep'] = ' - ' if page_title else ''
         fields['perc_raw'] = tab.progress()
+        fields['backend'] = objreg.get('args').backend
 
         if tab.load_status() == usertypes.LoadStatus.loading:
             fields['perc'] = '[{}%] '.format(tab.progress())

qutebrowser/misc/crashsignal.py

@@ -72,10 +72,7 @@ class CrashHandler(QObject):
 
     def handle_segfault(self):
         """Handle a segfault from a previous run."""
-        data_dir = standarddir.data()
+        logname = os.path.join(standarddir.data(), 'crash.log')
-        if data_dir is None:
-            return
-        logname = os.path.join(data_dir, 'crash.log')
         try:
             # First check if an old logfile exists.
             if os.path.exists(logname):
@@ -131,10 +128,7 @@ class CrashHandler(QObject):
     def _init_crashlogfile(self):
         """Start a new logfile and redirect faulthandler to it."""
         assert not self._args.no_err_windows
-        data_dir = standarddir.data()
+        logname = os.path.join(standarddir.data(), 'crash.log')
-        if data_dir is None:
-            return
-        logname = os.path.join(data_dir, 'crash.log')
         try:
             self._crash_log_file = open(logname, 'w', encoding='ascii')
         except OSError:

qutebrowser/misc/lineparser.py

@@ -57,10 +57,7 @@ class BaseLineParser(QObject):
         """
         super().__init__(parent)
         self._configdir = configdir
-        if self._configdir is None:
+        self._configfile = os.path.join(self._configdir, fname)
-            self._configfile = None
-        else:
-            self._configfile = os.path.join(self._configdir, fname)
         self._fname = fname
         self._binary = binary
         self._opened = False
@@ -76,8 +73,6 @@ class BaseLineParser(QObject):
         Return:
             True if the file should be saved, False otherwise.
         """
-        if self._configdir is None:
-            return False
         if not os.path.exists(self._configdir):
             os.makedirs(self._configdir, 0o755)
         return True
@@ -222,7 +217,7 @@ class LineParser(BaseLineParser):
             binary: Whether to open the file in binary mode.
         """
         super().__init__(configdir, fname, binary=binary, parent=parent)
-        if configdir is None or not os.path.isfile(self._configfile):
+        if not os.path.isfile(self._configfile):
             self.data = []
         else:
             log.init.debug("Reading {}".format(self._configfile))

qutebrowser/misc/sessions.py

@@ -47,15 +47,11 @@ def init(parent=None):
     Args:
         parent: The parent to use for the SessionManager.
     """
-    data_dir = standarddir.data()
+    base_path = os.path.join(standarddir.data(), 'sessions')
-    if data_dir is None:
+    try:
-        base_path = None
+        os.mkdir(base_path)
-    else:
+    except FileExistsError:
-        base_path = os.path.join(standarddir.data(), 'sessions')
+        pass
-        try:
-            os.mkdir(base_path)
-        except FileExistsError:
-            pass
 
     session_manager = SessionManager(base_path, parent)
     objreg.register('session-manager', session_manager)
@@ -137,11 +133,6 @@ class SessionManager(QObject):
         if os.path.isabs(path) and ((not check_exists) or
                                     os.path.exists(path)):
             return path
-        elif self._base_path is None:
-            if check_exists:
-                raise SessionNotFoundError(name)
-            else:
-                return None
         else:
             path = os.path.join(self._base_path, name + '.yml')
             if check_exists and not os.path.exists(path):
@@ -282,8 +273,6 @@ class SessionManager(QObject):
         """
         name = self._get_session_name(name)
         path = self._get_session_path(name)
-        if path is None:
-            raise SessionError("No data storage configured.")
 
         log.sessions.debug("Saving session {} to {}...".format(name, path))
         if last_window:
@@ -400,8 +389,6 @@ class SessionManager(QObject):
     def list_sessions(self):
         """Get a list of all session names."""
         sessions = []
-        if self._base_path is None:
-            return sessions
         for filename in os.listdir(self._base_path):
             base, ext = os.path.splitext(filename)
             if ext == '.yml':

qutebrowser/qutebrowser.py

@@ -47,14 +47,7 @@ def get_argparser():
     """Get the argparse parser."""
     parser = argparse.ArgumentParser(prog='qutebrowser',
                                      description=qutebrowser.__description__)
-    parser.add_argument('-c', '--confdir', help="Set config directory (empty "
+    parser.add_argument('--basedir', help="Base directory for all storage.")
-                        "for no config storage).")
-    parser.add_argument('--datadir', help="Set data directory (empty for "
-                        "no data storage).")
-    parser.add_argument('--cachedir', help="Set cache directory (empty for "
-                        "no cache storage).")
-    parser.add_argument('--basedir', help="Base directory for all storage. "
-                        "Other --*dir arguments are ignored if this is given.")
     parser.add_argument('-V', '--version', help="Show version and quit.",
                         action='store_true')
     parser.add_argument('-s', '--set', help="Set a temporary setting for "
@@ -112,8 +105,10 @@ def get_argparser():
                        "temporary basedir.")
     debug.add_argument('--no-err-windows', action='store_true', help="Don't "
                        "show any error windows (used for tests/smoke.py).")
-    debug.add_argument('--qt-arg', help="Pass an argument with a value to Qt.",
+    debug.add_argument('--qt-arg', help="Pass an argument with a value to Qt. "
-                       nargs=2)
+                       "For example, you can do "
+                       "`--qt-arg geometry 650x555+200+300` to set the window "
+                       "geometry.", nargs=2, metavar=('NAME', 'VALUE'))
     debug.add_argument('--qt-flag', help="Pass an argument to Qt as flag.",
                        nargs=1)
     parser.add_argument('command', nargs='*', help="Commands to execute on "
@@ -124,6 +119,11 @@ def get_argparser():
     return parser
 
 
+def directory(arg):
+    if not arg:
+        raise argparse.ArgumentTypeError("Invalid empty value")
+
+
 def logfilter_error(logfilter: str):
     """Validate logger names passed to --logfilter.
 

qutebrowser/utils/jinja.py

@@ -23,7 +23,6 @@ import os
 import os.path
 import traceback
 import mimetypes
-import base64
 
 import jinja2
 import jinja2.exceptions
@@ -82,8 +81,7 @@ def data_url(path):
     filename = utils.resource_filename(path)
     mimetype = mimetypes.guess_type(filename)
     assert mimetype is not None, path
-    b64 = base64.b64encode(data).decode('ascii')
+    return urlutils.data_url(mimetype[0], data).toString()
-    return 'data:{};charset=utf-8;base64,{}'.format(mimetype[0], b64)
 
 
 def render(template, **kwargs):

qutebrowser/utils/standarddir.py

@@ -21,6 +21,7 @@
 
 import os
 import sys
+import shutil
 import os.path
 
 from PyQt5.QtCore import QCoreApplication, QStandardPaths
@@ -43,7 +44,7 @@ def config():
             # WORKAROUND - see
             # https://bugreports.qt.io/browse/QTBUG-38872
             path = os.path.join(path, appname)
-    _maybe_create(path)
+    _create(path)
     return path
 
 
@@ -61,7 +62,7 @@ def data():
                 QStandardPaths.ConfigLocation)
             if data_path == config_path:
                 path = os.path.join(path, 'data')
-    _maybe_create(path)
+    _create(path)
     return path
 
 
@@ -82,7 +83,7 @@ def cache():
     overridden, path = _from_args(typ, _args)
     if not overridden:
         path = _writable_location(typ)
-    _maybe_create(path)
+    _create(path)
     return path
 
 
@@ -92,7 +93,7 @@ def download():
     overridden, path = _from_args(typ, _args)
     if not overridden:
         path = _writable_location(typ)
-    _maybe_create(path)
+    _create(path)
     return path
 
 
@@ -116,7 +117,7 @@ def runtime():
         # maximum length of 104 chars), so we don't add the username here...
         appname = QCoreApplication.instance().applicationName()
         path = os.path.join(path, appname)
-    _maybe_create(path)
+    _create(path)
     return path
 
 
@@ -145,11 +146,6 @@ def _from_args(typ, args):
             override: boolean, if the user did override the path
             path: The overridden path, or None to turn off storage.
     """
-    typ_to_argparse_arg = {
-        QStandardPaths.ConfigLocation: 'confdir',
-        QStandardPaths.DataLocation: 'datadir',
-        QStandardPaths.CacheLocation: 'cachedir',
-    }
     basedir_suffix = {
         QStandardPaths.ConfigLocation: 'config',
         QStandardPaths.DataLocation: 'data',
@@ -158,9 +154,6 @@ def _from_args(typ, args):
         QStandardPaths.RuntimeLocation: 'runtime',
     }
 
-    if args is None:
-        return (False, None)
-
     if getattr(args, 'basedir', None) is not None:
         basedir = args.basedir
 
@@ -169,22 +162,12 @@ def _from_args(typ, args):
         except KeyError:  # pragma: no cover
             return (False, None)
         return (True, os.path.abspath(os.path.join(basedir, suffix)))
-
-    try:
-        argname = typ_to_argparse_arg[typ]
-    except KeyError:
-        return (False, None)
-    arg_value = getattr(args, argname)
-    if arg_value is None:
-        return (False, None)
-    elif arg_value == '':
-        return (True, None)
     else:
-        return (True, arg_value)
+        return (False, None)
 
 
-def _maybe_create(path):
+def _create(path):
-    """Create the `path` directory if path is not None.
+    """Create the `path` directory.
 
     From the XDG basedir spec:
         If, when attempting to write a file, the destination directory is
@@ -192,11 +175,10 @@ def _maybe_create(path):
         0700. If the destination directory exists already the permissions
         should not be changed.
     """
-    if path is not None:
+    try:
-        try:
+        os.makedirs(path, 0o700)
-            os.makedirs(path, 0o700)
+    except FileExistsError:
-        except FileExistsError:
+        pass
-            pass
 
 
 def init(args):
@@ -207,6 +189,8 @@ def init(args):
         log.init.debug("Base directory: {}".format(args.basedir))
     _args = args
     _init_cachedir_tag()
+    if args is not None:
+        _move_webengine_data()
 
 
 def _init_cachedir_tag():
@@ -214,10 +198,7 @@ def _init_cachedir_tag():
 
     See http://www.brynosaurus.com/cachedir/spec.html
     """
-    cache_dir = cache()
+    cachedir_tag = os.path.join(cache(), 'CACHEDIR.TAG')
-    if cache_dir is None:
-        return
-    cachedir_tag = os.path.join(cache_dir, 'CACHEDIR.TAG')
     if not os.path.exists(cachedir_tag):
         try:
             with open(cachedir_tag, 'w', encoding='utf-8') as f:
@@ -229,3 +210,49 @@ def _init_cachedir_tag():
                         "cachedir/\n")
         except OSError:
             log.init.exception("Failed to create CACHEDIR.TAG")
+
+
+def _move_webengine_data():
+    """Move QtWebEngine data from an older location to the new one."""
+    # Do NOT use _writable_location here as that'd give us a wrong path
+    old_data_dir = QStandardPaths.writableLocation(QStandardPaths.DataLocation)
+    old_cache_dir = QStandardPaths.writableLocation(
+        QStandardPaths.CacheLocation)
+    new_data_dir = os.path.join(data(), 'webengine')
+    new_cache_dir = os.path.join(cache(), 'webengine')
+
+    if (not os.path.exists(os.path.join(old_data_dir, 'QtWebEngine')) and
+            not os.path.exists(os.path.join(old_cache_dir, 'QtWebEngine'))):
+        return
+
+    log.init.debug("Moving QtWebEngine data from {} to {}".format(
+        old_data_dir, new_data_dir))
+    log.init.debug("Moving QtWebEngine cache from {} to {}".format(
+        old_cache_dir, new_cache_dir))
+
+    if os.path.exists(new_data_dir):
+        log.init.warning("Failed to move old QtWebEngine data as {} already "
+                         "exists!".format(new_data_dir))
+        return
+    if os.path.exists(new_cache_dir):
+        log.init.warning("Failed to move old QtWebEngine cache as {} already "
+                         "exists!".format(new_cache_dir))
+        return
+
+    try:
+        shutil.move(os.path.join(old_data_dir, 'QtWebEngine', 'Default'),
+                    new_data_dir)
+        shutil.move(os.path.join(old_cache_dir, 'QtWebEngine', 'Default'),
+                    new_cache_dir)
+
+        # Remove e.g.
+        # ~/.local/share/qutebrowser/qutebrowser/QtWebEngine/Default
+        if old_data_dir.split(os.sep)[-2:] == ['qutebrowser', 'qutebrowser']:
+            log.init.debug("Removing {} / {}".format(
+                old_data_dir, old_cache_dir))
+            for old_dir in old_data_dir, old_cache_dir:
+                os.rmdir(os.path.join(old_dir, 'QtWebEngine'))
+                os.rmdir(old_dir)
+    except OSError as e:
+        log.init.exception("Failed to move old QtWebEngine data/cache: "
+                           "{}".format(e))

qutebrowser/utils/urlutils.py

@@ -20,6 +20,7 @@
 """Utils regarding URL handling."""
 
 import re
+import base64
 import os.path
 import ipaddress
 import posixpath
@@ -580,3 +581,11 @@ def file_url(path):
         path: The absolute path to the local file
     """
     return QUrl.fromLocalFile(path).toString(QUrl.FullyEncoded)
+
+
+def data_url(mimetype, data):
+    """Get a data: QUrl for the given data."""
+    b64 = base64.b64encode(data).decode('ascii')
+    url = QUrl('data:{};base64,{}'.format(mimetype, b64))
+    qtutils.ensure_valid(url)
+    return url

tests/end2end/conftest.py

@@ -132,14 +132,7 @@ if not getattr(sys, 'frozen', False):
 
 def pytest_collection_modifyitems(config, items):
     """Apply @qtwebengine_* markers; skip unittests with QUTE_BDD_WEBENGINE."""
-    vercheck = qtutils.version_check
-    qtbug_54419_fixed = ((vercheck('5.6.2') and not vercheck('5.7.0')) or
-                         qtutils.version_check('5.7.1') or
-                         os.environ.get('QUTE_QTBUG54419_PATCHED', ''))
-
     markers = [
-        ('qtwebengine_createWindow', 'Skipped because of QTBUG-54419',
-         pytest.mark.skipif, not qtbug_54419_fixed and config.webengine),
         ('qtwebengine_todo', 'QtWebEngine TODO', pytest.mark.xfail,
          config.webengine),
         ('qtwebengine_skip', 'Skipped with QtWebEngine', pytest.mark.skipif,

tests/end2end/features/conftest.py

@@ -121,6 +121,8 @@ def set_setting_given(quteproc, httpbin, sect, opt, value):
 
     This is available as "Given:" step so it can be used as "Background:".
     """
+    if value == '<empty>':
+        value = ''
     value = value.replace('(port)', str(httpbin.port))
     quteproc.set_setting(sect, opt, value)
 
@@ -211,6 +213,8 @@ def open_path(quteproc, path):
 @bdd.when(bdd.parsers.parse("I set {sect} -> {opt} to {value}"))
 def set_setting(quteproc, httpbin, sect, opt, value):
     """Set a qutebrowser setting."""
+    if value == '<empty>':
+        value = ''
     value = value.replace('(port)', str(httpbin.port))
     quteproc.set_setting(sect, opt, value)
 
@@ -479,7 +483,7 @@ def check_header(quteproc, header, value):
     content = quteproc.get_content()
     data = json.loads(content)
     print(data)
-    assert data['headers'][header] == value
+    assert utils.pattern_match(pattern=value, value=data['headers'][header])
 
 
 @bdd.then(bdd.parsers.parse('the page should contain the html "{text}"'))

tests/end2end/features/editor.feature

@@ -93,3 +93,15 @@ Feature: Opening external editors
         And I wait for "Read back: foobar" in the log
         And I run :click-element id qute-button
         Then the javascript message "text: foobar" should be logged
+
+    Scenario: Spawning an editor with existing text
+        When I set up a fake editor replacing "foo" by "bar"
+        And I open data/editor.html
+        And I run :click-element id qute-textarea
+        And I wait for "Clicked editable element!" in the log
+        And I run :insert-text foo
+        And I wait for "Inserting text into element *" in the log
+        And I run :open-editor
+        And I wait for "Read back: bar" in the log
+        And I run :click-element id qute-button
+        Then the javascript message "text: bar" should be logged

tests/end2end/features/hints.feature

@@ -22,7 +22,6 @@ Feature: Using hints
 
     ### Opening in current or new tab
 
-    @qtwebengine_createWindow
     Scenario: Following a hint and force to open in current tab.
         When I open data/hints/link_blank.html
         And I hint with args "links current" and follow a
@@ -30,7 +29,7 @@ Feature: Using hints
         Then the following tabs should be open:
             - data/hello.txt (active)
 
-    @qtwebengine_createWindow
+    @qtwebengine_skip: Opens in background
     Scenario: Following a hint and allow to open in new tab.
         When I open data/hints/link_blank.html
         And I hint with args "links normal" and follow a
@@ -39,7 +38,6 @@ Feature: Using hints
             - data/hints/link_blank.html
             - data/hello.txt (active)
 
-    @qtwebengine_createWindow
     Scenario: Following a hint to link with sub-element and force to open in current tab.
         When I open data/hints/link_span.html
         And I run :tab-close
@@ -186,13 +184,12 @@ Feature: Using hints
         And I hint wht args "links normal" and follow a
         Then "navigation request: url http://localhost:*/data/hello2.txt, type NavigationTypeLinkClicked, *" should be logged
 
-    @qtwebengine_createWindow
+    @qtwebengine_skip: Opens in new tab due to Chromium bug
     Scenario: Opening a link inside a specific iframe
         When I open data/hints/iframe_target.html
         And I hint with args "links normal" and follow a
         Then "navigation request: url http://localhost:*/data/hello.txt, type NavigationTypeLinkClicked, *" should be logged
 
-    @qtwebengine_createWindow
     Scenario: Opening a link with specific target frame in a new tab
         When I open data/hints/iframe_target.html
         And I run :tab-only

tests/end2end/features/history.feature

@@ -48,7 +48,6 @@ Feature: Page history
         Then the history file should contain:
             http://localhost:(port)/status/404 Error loading page: http://localhost:(port)/status/404
 
-    @qtwebengine_createWindow
     Scenario: History with invalid URL
         When I open data/javascript/window_open.html
         And I run :click-element id open-invalid

tests/end2end/features/javascript.feature

@@ -7,8 +7,6 @@ Feature: Javascript stuff
         And I open data/javascript/consolelog.html
         Then the javascript message "console.log works!" should be logged
 
-    # Causes segfaults...
-    @qtwebengine_createWindow
     Scenario: Opening/Closing a window via JS
         When I open data/javascript/window_open.html
         And I run :tab-only
@@ -18,8 +16,6 @@ Feature: Javascript stuff
         And I run :click-element id close-normal
         Then "Focus object changed: *" should be logged
 
-    # Causes segfaults...
-    @qtwebengine_createWindow
     Scenario: Opening/closing a modal window via JS
         When I open data/javascript/window_open.html
         And I run :tab-only
@@ -37,14 +33,14 @@ Feature: Javascript stuff
     Scenario: Closing a JS window twice (issue 906) - qtwebkit
         When I open about:blank
         And I run :tab-only
-        When I open data/javascript/window_open.html in a new tab
+        And I open data/javascript/window_open.html in a new tab
         And I run :click-element id open-normal
         And I wait for "Changing title for idx 2 to 'about:blank'" in the log
         And I run :tab-focus 2
         And I run :click-element id close-twice
         Then "Requested to close * which does not exist!" should be logged
 
-    @qtwebengine_createWindow @qtwebkit_skip
+    @qtwebkit_skip @flaky
     Scenario: Closing a JS window twice (issue 906) - qtwebengine
         When I open about:blank
         And I run :tab-only
@@ -56,7 +52,6 @@ Feature: Javascript stuff
         And I wait for "Focus object changed: *" in the log
         Then no crash should happen
 
-    @qtwebengine_createWindow
     Scenario: Opening window without user interaction with javascript-can-open-windows-automatically set to true
         When I open data/hello.txt
         And I set content -> javascript-can-open-windows-automatically to true
@@ -64,7 +59,6 @@ Feature: Javascript stuff
         And I run :jseval if (window.open('about:blank')) { console.log('window opened'); } else { console.log('error while opening window'); }
         Then the javascript message "window opened" should be logged
 
-    @qtwebengine_createWindow
     Scenario: Opening window without user interaction with javascript-can-open-windows-automatically set to false
         When I open data/hello.txt
         And I set content -> javascript-can-open-windows-automatically to false

tests/end2end/features/misc.feature

@@ -415,6 +415,16 @@ Feature: Various utility commands.
         And I open headers
         Then the header Accept-Language should be set to en,de
 
+    Scenario: Setting a custom user-agent header
+        When I set network -> user-agent to toaster
+        And I open headers
+        Then the header User-Agent should be set to toaster
+
+    Scenario: Setting the default user-agent header
+        When I set network -> user-agent to <empty>
+        And I open headers
+        Then the header User-Agent should be set to Mozilla/5.0 *
+
     ## :messages
 
     Scenario: Showing error messages
@@ -556,7 +566,7 @@ Feature: Various utility commands.
     Scenario: Clicking an element with unknown ID
         When I open data/click_element.html
         And I run :click-element id blah
-        Then the error "No element found!" should be shown
+        Then the error "No element found with id blah!" should be shown
 
     Scenario: Clicking an element by ID
         When I open data/click_element.html

tests/end2end/features/prompts.feature

@@ -197,6 +197,14 @@ Feature: Prompts
         And I run :prompt-accept no
         Then a SSL error page should be shown
 
+    Scenario: SSL error with ssl-strict = ask -> abort
+        When I clear SSL errors
+        And I set network -> ssl-strict to ask
+        And I load an SSL page
+        And I wait for a prompt
+        And I run :leave-mode
+        Then a SSL error page should be shown
+
     # Geolocation
 
     Scenario: Always rejecting geolocation

tests/end2end/features/utilcmds.feature

@@ -124,11 +124,10 @@ Feature: Miscellaneous utility commands exposed to the user.
         Then the page should not be scrolled
         And the error "prompt-accept: This command is only allowed in prompt/yesno mode, not normal." should be shown
 
-    @qtwebengine_createWindow
     Scenario: :repeat-command with mode-switching command
         When I open data/hints/link_blank.html
         And I run :tab-only
-        And I hint with args "all"
+        And I hint with args "all tab-fg"
         And I run :leave-mode
         And I run :repeat-command
         And I run :follow-hint a

tests/end2end/test_invocations.py

@@ -68,24 +68,6 @@ def temp_basedir_env(tmpdir, short_tmpdir):
     return env
 
 
-@pytest.mark.linux
-def test_no_config(request, temp_basedir_env, quteproc_new):
-    """Test starting with -c ""."""
-    args = ['-c', ''] + _base_args(request.config)
-    quteproc_new.start(args, env=temp_basedir_env)
-    quteproc_new.send_cmd(':quit')
-    quteproc_new.wait_for_quit()
-
-
-@pytest.mark.linux
-def test_no_cache(request, temp_basedir_env, quteproc_new):
-    """Test starting with --cachedir=""."""
-    args = ['--cachedir='] + _base_args(request.config)
-    quteproc_new.start(args, env=temp_basedir_env)
-    quteproc_new.send_cmd(':quit')
-    quteproc_new.wait_for_quit()
-
-
 @pytest.mark.linux
 def test_ascii_locale(request, httpbin, tmpdir, quteproc_new):
     """Test downloads with LC_ALL=C set.

tests/helpers/fixtures.py

@@ -155,10 +155,11 @@ def tab_registry(win_registry):
 
 
 @pytest.fixture
-def fake_web_tab(stubs, tab_registry, mode_manager, qapp):
+def fake_web_tab(stubs, tab_registry, mode_manager, qapp, fake_args):
     """Fixture providing the FakeWebTab *class*."""
     if PYQT_VERSION < 0x050600:
         pytest.skip('Causes segfaults, see #1638')
+    fake_args.backend = 'webengine'
     return stubs.FakeWebTab
 
 

tests/unit/browser/test_adblock.py

@@ -29,7 +29,6 @@ from PyQt5.QtCore import pyqtSignal, QUrl, QObject
 
 from qutebrowser.browser import adblock
 from qutebrowser.utils import objreg
-from qutebrowser.commands import cmdexc
 
 pytestmark = pytest.mark.usefixtures('qapp', 'config_tmpdir')
 
@@ -225,32 +224,6 @@ def generic_blocklists(directory):
     return [blocklist1, blocklist2, blocklist3, blocklist4, blocklist5]
 
 
-def test_without_datadir(config_stub, tmpdir, monkeypatch, win_registry):
-    """No directory for data configured so no hosts file exists.
-
-    Ensure CommandError is raised and no URL is blocked.
-    """
-    config_stub.data = {
-        'content': {
-            'host-block-lists': generic_blocklists(tmpdir),
-            'host-blocking-enabled': True,
-        }
-    }
-    monkeypatch.setattr('qutebrowser.utils.standarddir.data', lambda: None)
-    host_blocker = adblock.HostBlocker()
-
-    with pytest.raises(cmdexc.CommandError) as excinfo:
-        host_blocker.adblock_update()
-    assert str(excinfo.value) == "No data storage is configured!"
-
-    host_blocker.read_hosts()
-    for str_url in URLS_TO_CHECK:
-        assert not host_blocker.is_blocked(QUrl(str_url))
-
-    # To test on_config_changed
-    config_stub.set('content', 'host-block-lists', None)
-
-
 def test_disabled_blocking_update(basedir, config_stub, download_stub,
                                   data_tmpdir, tmpdir, win_registry, caplog):
     """Ensure no URL is blocked when host blocking is disabled."""

tests/unit/browser/webkit/test_cache.py

@@ -111,16 +111,6 @@ def test_cache_size_deactivated(config_stub, tmpdir):
     assert disk_cache.cacheSize() == 0
 
 
-def test_cache_no_cache_dir(config_stub):
-    """Confirm that the cache is deactivated when cache_dir is None."""
-    config_stub.data = {
-        'storage': {'cache-size': 1024},
-        'general': {'private-browsing': False},
-    }
-    disk_cache = cache.DiskCache(None)
-    assert disk_cache.cacheSize() == 0
-
-
 def test_cache_existing_metadata_file(config_stub, tmpdir):
     """Test querying existing meta data file from activated cache."""
     config_stub.data = {

tests/unit/browser/webkit/test_history.py

@@ -19,7 +19,6 @@
 
 """Tests for the global page history."""
 
-import base64
 import logging
 
 import pytest
@@ -28,7 +27,7 @@ from hypothesis import strategies
 from PyQt5.QtCore import QUrl
 
 from qutebrowser.browser import history
-from qutebrowser.utils import objreg
+from qutebrowser.utils import objreg, urlutils
 
 
 class FakeWebHistory:
@@ -65,13 +64,6 @@ def test_async_read_twice(monkeypatch, qtbot, tmpdir, caplog):
     assert caplog.records[0].msg == expected
 
 
-def test_async_read_no_datadir(qtbot, config_stub, fake_save_manager):
-    config_stub.data = {'general': {'private-browsing': False}}
-    hist = history.WebHistory(hist_dir=None, hist_name='history')
-    with qtbot.waitSignal(hist.async_read_done):
-        list(hist.async_read())
-
-
 @pytest.mark.parametrize('redirect', [True, False])
 def test_adding_item_during_async_read(qtbot, hist, redirect):
     """Check what happens when adding URL while reading the history."""
@@ -382,9 +374,8 @@ def hist_interface():
 
 
 def test_history_interface(qtbot, webview, hist_interface):
-    html = "<a href='about:blank'>foo</a>"
+    html = b"<a href='about:blank'>foo</a>"
-    data = base64.b64encode(html.encode('utf-8')).decode('ascii')
+    url = urlutils.data_url('text/html', html)
-    url = QUrl("data:text/html;charset=utf-8;base64,{}".format(data))
     with qtbot.waitSignal(webview.loadFinished):
         webview.load(url)
 

tests/unit/browser/webkit/test_webkitelem.py

@@ -258,8 +258,8 @@ class TestWebKitElement:
         lambda e: e.has_frame(),
         lambda e: e.geometry(),
         lambda e: e.style_property('visibility', strategy='computed'),
-        lambda e: e.text(),
+        lambda e: e.value(),
-        lambda e: e.set_text('foo'),
+        lambda e: e.set_value('foo'),
         lambda e: e.insert_text('foo'),
         lambda e: e.is_writable(),
         lambda e: e.is_content_editable(),
@@ -271,7 +271,7 @@ class TestWebKitElement:
         lambda e: e.rect_on_view(),
         lambda e: e._is_visible(None),
     ], ids=['str', 'getitem', 'setitem', 'delitem', 'contains', 'iter', 'len',
-            'frame', 'geometry', 'style_property', 'text', 'set_text',
+            'frame', 'geometry', 'style_property', 'value', 'set_value',
             'insert_text', 'is_writable', 'is_content_editable', 'is_editable',
             'is_text_input', 'remove_blank_target', 'outer_xml', 'tag_name',
             'rect_on_view', 'is_visible'])
@@ -411,28 +411,18 @@ class TestWebKitElement:
     def test_style_property(self, elem):
         assert elem.style_property('foo', strategy='computed') == 'bar'
 
-    @pytest.mark.parametrize('use_js, editable, expected', [
+    def test_value(self, elem):
-        (True, 'false', 'js'),
-        (True, 'true', 'nojs'),
-        (False, 'false', 'nojs'),
-        (False, 'true', 'nojs'),
-    ])
-    def test_text(self, use_js, editable, expected):
-        elem = get_webelem(attributes={'contenteditable': editable})
-        elem._elem.toPlainText.return_value = 'nojs'
         elem._elem.evaluateJavaScript.return_value = 'js'
-        assert elem.text(use_js=use_js) == expected
+        assert elem.value() == 'js'
 
-    @pytest.mark.parametrize('use_js, editable, text, uses_js, arg', [
+    @pytest.mark.parametrize('editable, value, uses_js, arg', [
-        (True, 'false', 'foo', True, "this.value='foo'"),
+        ('false', 'foo', True, "this.value='foo'"),
-        (True, 'false', "foo'bar", True, r"this.value='foo\'bar'"),
+        ('false', "foo'bar", True, r"this.value='foo\'bar'"),
-        (True, 'true', 'foo', False, 'foo'),
+        ('true', 'foo', False, 'foo'),
-        (False, 'false', 'foo', False, 'foo'),
-        (False, 'true', 'foo', False, 'foo'),
     ])
-    def test_set_text(self, use_js, editable, text, uses_js, arg):
+    def test_set_value(self, editable, value, uses_js, arg):
         elem = get_webelem(attributes={'contenteditable': editable})
-        elem.set_text(text, use_js=use_js)
+        elem.set_value(value)
         attr = 'evaluateJavaScript' if uses_js else 'setPlainText'
         called_mock = getattr(elem._elem, attr)
         called_mock.assert_called_with(arg)

tests/unit/config/test_config.py

@@ -23,9 +23,7 @@ import os.path
 import configparser
 import collections
 import shutil
-from unittest import mock
 
-from PyQt5.QtCore import QObject
 from PyQt5.QtGui import QColor
 import pytest
 
@@ -33,7 +31,6 @@ import qutebrowser
 from qutebrowser.config import config, configexc, configdata
 from qutebrowser.config.parsers import keyconf
 from qutebrowser.commands import runners
-from qutebrowser.utils import objreg, standarddir
 
 
 class TestConfigParser:
@@ -43,12 +40,12 @@ class TestConfigParser:
     Objects = collections.namedtuple('Objects', ['cp', 'cfg'])
 
     @pytest.fixture
-    def objects(self):
+    def objects(self, tmpdir):
         cp = configparser.ConfigParser(interpolation=None,
                                        comment_prefixes='#')
         cp.optionxform = lambda opt: opt  # be case-insensitive
         cfg = config.ConfigManager()
-        cfg.read(None, None)
+        cfg.read(str(tmpdir), 'qutebrowser.conf')
         return self.Objects(cp=cp, cfg=cfg)
 
     @pytest.mark.parametrize('config, section, option, value', [
@@ -247,7 +244,7 @@ class TestKeyConfigParser:
 
     """Test config.parsers.keyconf.KeyConfigParser."""
 
-    def test_cmd_binding(self, cmdline_test, config_stub):
+    def test_cmd_binding(self, cmdline_test, config_stub, tmpdir):
         """Test various command bindings.
 
         See https://github.com/The-Compiler/qutebrowser/issues/615
@@ -256,7 +253,7 @@ class TestKeyConfigParser:
             cmdline_test: A pytest fixture which provides testcases.
         """
         config_stub.data = {'aliases': []}
-        kcp = keyconf.KeyConfigParser(None, None)
+        kcp = keyconf.KeyConfigParser(str(tmpdir), 'keys.conf')
         kcp._cur_section = 'normal'
         if cmdline_test.valid:
             kcp._read_command(cmdline_test.cmd)
@@ -379,17 +376,17 @@ class TestDefaultConfig:
     """Test validating of the default config."""
 
     @pytest.mark.usefixtures('qapp')
-    def test_default_config(self):
+    def test_default_config(self, tmpdir):
         """Test validating of the default config."""
         conf = config.ConfigManager()
-        conf.read(None, None)
+        conf.read(str(tmpdir), 'qutebrowser.conf')
         conf._validate_all()
 
-    def test_default_key_config(self):
+    def test_default_key_config(self, tmpdir):
         """Test validating of the default key config."""
         # We import qutebrowser.app so the cmdutils.register decorators run.
         import qutebrowser.app  # pylint: disable=unused-variable
-        conf = keyconf.KeyConfigParser(None, None)
+        conf = keyconf.KeyConfigParser(str(tmpdir), 'keys.conf')
         runner = runners.CommandRunner(win_id=0)
         for sectname in configdata.KEY_DATA:
             for cmd in conf.get_bindings_for(sectname).values():
@@ -418,48 +415,3 @@ class TestDefaultConfig:
         shutil.copy(full_path, str(tmpdir / 'qutebrowser.conf'))
         conf = config.ConfigManager()
         conf.read(str(tmpdir), 'qutebrowser.conf')
-
-
-@pytest.mark.integration
-class TestConfigInit:
-
-    """Test initializing of the config."""
-
-    @pytest.fixture(autouse=True)
-    def patch(self, fake_args):
-        objreg.register('app', QObject())
-        objreg.register('save-manager', mock.MagicMock())
-        fake_args.relaxed_config = False
-        old_standarddir_args = standarddir._args
-        yield
-        objreg.delete('app')
-        objreg.delete('save-manager')
-        # registered by config.init()
-        objreg.delete('config')
-        objreg.delete('key-config')
-        objreg.delete('state-config')
-        standarddir._args = old_standarddir_args
-
-    @pytest.fixture
-    def env(self, tmpdir):
-        conf_path = (tmpdir / 'config').ensure(dir=1)
-        data_path = (tmpdir / 'data').ensure(dir=1)
-        cache_path = (tmpdir / 'cache').ensure(dir=1)
-        env = {
-            'XDG_CONFIG_HOME': str(conf_path),
-            'XDG_DATA_HOME': str(data_path),
-            'XDG_CACHE_HOME': str(cache_path),
-        }
-        return env
-
-    def test_config_none(self, monkeypatch, env, fake_args):
-        """Test initializing with config path set to None."""
-        fake_args.confdir = ''
-        fake_args.datadir = ''
-        fake_args.cachedir = ''
-        fake_args.basedir = None
-        for k, v in env.items():
-            monkeypatch.setenv(k, v)
-        standarddir.init(fake_args)
-        config.init()
-        assert not os.listdir(env['XDG_CONFIG_HOME'])

tests/unit/config/test_configtypes.py

@@ -22,7 +22,6 @@ import re
 import collections
 import itertools
 import os.path
-import base64
 import warnings
 
 import pytest
@@ -1211,15 +1210,9 @@ def unrequired_class(**kwargs):
 
 @pytest.mark.usefixtures('qapp')
 @pytest.mark.usefixtures('config_tmpdir')
-class TestFileAndUserStyleSheet:
+class TestFile:
 
-    """Test File/UserStyleSheet."""
+    @pytest.fixture(params=[configtypes.File, unrequired_class])
-
-    @pytest.fixture(params=[
-        configtypes.File,
-        configtypes.UserStyleSheet,
-        unrequired_class,
-    ])
     def klass(self, request):
         return request.param
 
@@ -1227,18 +1220,12 @@ class TestFileAndUserStyleSheet:
     def file_class(self):
         return configtypes.File
 
-    @pytest.fixture
-    def userstylesheet_class(self):
-        return configtypes.UserStyleSheet
-
     def _expected(self, klass, arg):
         """Get the expected value."""
         if not arg:
             return None
         elif klass is configtypes.File:
             return arg
-        elif klass is configtypes.UserStyleSheet:
-            return QUrl.fromLocalFile(arg)
         elif klass is unrequired_class:
             return arg
         else:
@@ -1262,11 +1249,6 @@ class TestFileAndUserStyleSheet:
         os_mock.path.isfile.return_value = False
         configtypes.File(required=False).validate('foobar')
 
-    def test_validate_does_not_exist_userstylesheet(self, os_mock):
-        """Test validate with a file which does not exist (UserStyleSheet)."""
-        os_mock.path.isfile.return_value = False
-        configtypes.UserStyleSheet().validate('foobar')
-
     def test_validate_exists_abs(self, klass, os_mock):
         """Test validate with a file which does exist."""
         os_mock.path.isfile.return_value = True
@@ -1284,21 +1266,10 @@ class TestFileAndUserStyleSheet:
         os_mock.path.join.assert_called_once_with(
             '/home/foo/.config/', 'foobar')
 
-    def test_validate_rel_config_none_file(self, os_mock, monkeypatch):
-        """Test with a relative path and standarddir.config returning None."""
-        monkeypatch.setattr(
-            'qutebrowser.config.configtypes.standarddir.config', lambda: None)
-        os_mock.path.isabs.return_value = False
-        with pytest.raises(configexc.ValidationError):
-            configtypes.File().validate('foobar')
-
     @pytest.mark.parametrize('configtype, value, raises', [
         (configtypes.File(), 'foobar', True),
-        (configtypes.UserStyleSheet(), 'foobar', False),
-        (configtypes.UserStyleSheet(), '\ud800', True),
         (configtypes.File(required=False), 'foobar', False),
-    ], ids=['file-foobar', 'userstylesheet-foobar', 'userstylesheet-unicode',
+    ], ids=['file-foobar', 'file-optional-foobar'])
-            'file-optional-foobar'])
     def test_validate_rel_inexistent(self, os_mock, monkeypatch, configtype,
                                      value, raises):
         """Test with a relative path and standarddir.config returning None."""
@@ -1347,7 +1318,6 @@ class TestFileAndUserStyleSheet:
 
     def test_transform_relative(self, klass, os_mock, monkeypatch):
         """Test transform() with relative dir and an available configdir."""
-        os_mock.path.exists.return_value = True  # for TestUserStyleSheet
         os_mock.path.isabs.return_value = False
         monkeypatch.setattr(
             'qutebrowser.config.configtypes.standarddir.config',
@@ -1355,18 +1325,6 @@ class TestFileAndUserStyleSheet:
         expected = self._expected(klass, '/configdir/foo')
         assert klass().transform('foo') == expected
 
-    @pytest.mark.parametrize('no_config', [False, True])
-    def test_transform_userstylesheet_base64(self, monkeypatch, no_config):
-        """Test transform with a data string."""
-        if no_config:
-            monkeypatch.setattr(
-                'qutebrowser.config.configtypes.standarddir.config',
-                lambda: None)
-
-        b64 = base64.b64encode(b"test").decode('ascii')
-        url = QUrl("data:text/css;charset=utf-8;base64,{}".format(b64))
-        assert configtypes.UserStyleSheet().transform("test") == url
-
 
 class TestDirectory:
 
@@ -1990,10 +1948,11 @@ class TestUserAgent:
     def test_validate_valid(self, klass, val):
         klass(none_ok=True).validate(val)
 
-    def test_validate_invalid(self, klass):
+    @pytest.mark.parametrize('val', ['', 'überbrowser'])
+    def test_validate_invalid(self, klass, val):
         """Test validate with empty string and none_ok = False."""
         with pytest.raises(configexc.ValidationError):
-            klass().validate('')
+            klass().validate(val)
 
     def test_transform(self, klass):
         assert klass().transform('foobar') == 'foobar'

tests/unit/config/test_configtypes_hypothesis.py

@@ -50,8 +50,7 @@ def gen_classes():
 @hypothesis.given(strategies.text())
 @hypothesis.example('\x00')
 def test_configtypes_hypothesis(klass, s):
-    if (klass in [configtypes.File, configtypes.UserStyleSheet] and
+    if (klass == configtypes.File and sys.platform == 'linux' and
-            sys.platform == 'linux' and
             not os.environ.get('DISPLAY', '')):
         pytest.skip("No DISPLAY available")
 

tests/unit/misc/test_lineparser.py

@@ -52,15 +52,6 @@ class TestBaseLineParser:
         lineparser._prepare_save()
         os_mock.makedirs.assert_called_with(self.CONFDIR, 0o755)
 
-    def test_prepare_save_no_config(self, mocker):
-        """Test if _prepare_save doesn't create a None config dir."""
-        os_mock = mocker.patch('qutebrowser.misc.lineparser.os')
-        os_mock.path.exists.return_value = True
-
-        lineparser = lineparsermod.BaseLineParser(None, self.FILENAME)
-        assert not lineparser._prepare_save()
-        assert not os_mock.makedirs.called
-
     def test_double_open(self, mocker, lineparser):
         """Test if _open refuses reentry."""
         mocker.patch('builtins.open', mock.mock_open())
@@ -158,15 +149,6 @@ class TestAppendLineParser:
         lineparser.save()
         assert (tmpdir / 'file').read() == self._get_expected(new_data)
 
-    def test_save_without_configdir(self, tmpdir, lineparser):
-        """Test save() failing because no configdir was set."""
-        new_data = ['new data 1', 'new data 2']
-        lineparser.new_data = new_data
-        lineparser._configdir = None
-        assert not lineparser.save()
-        # make sure new data is still there
-        assert lineparser.new_data == new_data
-
     def test_clear(self, tmpdir, lineparser):
         """Check if calling clear() empties both pending and persisted data."""
         lineparser.new_data = ['one', 'two']
@@ -179,14 +161,6 @@ class TestAppendLineParser:
         assert not lineparser.new_data
         assert (tmpdir / 'file').read() == ""
 
-    def test_clear_without_configdir(self, tmpdir, lineparser):
-        """Test clear() failing because no configdir was set."""
-        new_data = ['new data 1', 'new data 2']
-        lineparser.new_data = new_data
-        lineparser._configdir = None
-        assert not lineparser.clear()
-        assert lineparser.new_data == new_data
-
     def test_iter_without_open(self, lineparser):
         """Test __iter__ without having called open()."""
         with pytest.raises(ValueError):

tests/unit/misc/test_sessions.py

@@ -40,9 +40,9 @@ webengine_refactoring_xfail = pytest.mark.xfail(
 
 
 @pytest.fixture
-def sess_man():
+def sess_man(tmpdir):
-    """Fixture providing a SessionManager with no session dir."""
+    """Fixture providing a SessionManager."""
-    return sessions.SessionManager(base_path=None)
+    return sessions.SessionManager(base_path=str(tmpdir))
 
 
 class TestInit:
@@ -52,13 +52,6 @@ class TestInit:
         yield
         objreg.delete('session-manager')
 
-    def test_no_standarddir(self, monkeypatch):
-        monkeypatch.setattr('qutebrowser.misc.sessions.standarddir.data',
-                            lambda: None)
-        sessions.init()
-        manager = objreg.get('session-manager')
-        assert manager._base_path is None
-
     @pytest.mark.parametrize('create_dir', [True, False])
     def test_with_standarddir(self, tmpdir, monkeypatch, create_dir):
         monkeypatch.setattr('qutebrowser.misc.sessions.standarddir.data',
@@ -110,16 +103,6 @@ class TestExists:
 
         assert not man.exists(name)
 
-    @pytest.mark.parametrize('absolute', [True, False])
-    def test_no_datadir(self, sess_man, tmpdir, absolute):
-        abs_session = tmpdir / 'foo.yml'
-        abs_session.ensure()
-
-        if absolute:
-            assert sess_man.exists(str(abs_session))
-        else:
-            assert not sess_man.exists('foo')
-
 
 @webengine_refactoring_xfail
 class TestSaveTab:
@@ -247,11 +230,6 @@ class TestSave:
         objreg.delete('main-window', scope='window', window=0)
         objreg.delete('tabbed-browser', scope='window', window=0)
 
-    def test_no_config_storage(self, sess_man):
-        with pytest.raises(sessions.SessionError) as excinfo:
-            sess_man.save('foo')
-        assert str(excinfo.value) == "No data storage configured."
-
     def test_update_completion_signal(self, sess_man, tmpdir, qtbot):
         session_path = tmpdir / 'foo.yml'
         with qtbot.waitSignal(sess_man.update_completion):
@@ -415,9 +393,6 @@ def test_delete_update_completion_signal(sess_man, qtbot, tmpdir):
 
 class TestListSessions:
 
-    def test_no_base_path(self, sess_man):
-        assert not sess_man.list_sessions()
-
     def test_no_sessions(self, tmpdir):
         sess_man = sessions.SessionManager(str(tmpdir))
         assert not sess_man.list_sessions()

tests/unit/utils/test_jinja.py

@@ -102,7 +102,7 @@ def test_data_url():
     print(data)
     url = QUrl(data)
     assert url.isValid()
-    assert data == 'data:text/plain;charset=utf-8;base64,Zm9v'  # 'foo'
+    assert data == 'data:text/plain;base64,Zm9v'  # 'foo'
 
 
 def test_not_found():

tests/unit/utils/test_standarddir.py

@@ -110,6 +110,7 @@ class TestStandardDir:
         (standarddir.data, 'XDG_DATA_HOME'),
         (standarddir.config, 'XDG_CONFIG_HOME'),
         (standarddir.cache, 'XDG_CACHE_HOME'),
+        (standarddir.runtime, 'XDG_RUNTIME_DIR'),
     ])
     @pytest.mark.linux
     def test_linux_explicit(self, monkeypatch, tmpdir, func, varname):
@@ -163,57 +164,7 @@ DirArgTest = collections.namedtuple('DirArgTest', 'arg, expected')
 @pytest.mark.usefixtures('reset_standarddir')
 class TestArguments:
 
-    """Tests with confdir/cachedir/datadir arguments."""
+    """Tests the --basedir argument."""
-
-    @pytest.fixture(params=[DirArgTest('', None), DirArgTest('foo', 'foo')])
-    def testcase(self, request, tmpdir):
-        """Fixture providing testcases."""
-        if request.param.expected is None:
-            return request.param
-        else:
-            # prepend tmpdir to both
-            arg = str(tmpdir / request.param.arg)
-            return DirArgTest(arg, arg)
-
-    def test_confdir(self, testcase):
-        """Test --confdir."""
-        args = types.SimpleNamespace(confdir=testcase.arg, cachedir=None,
-                                     datadir=None, basedir=None)
-        standarddir.init(args)
-        assert standarddir.config() == testcase.expected
-
-    def test_cachedir(self, testcase):
-        """Test --cachedir."""
-        args = types.SimpleNamespace(confdir=None, cachedir=testcase.arg,
-                                     datadir=None, basedir=None)
-        standarddir.init(args)
-        assert standarddir.cache() == testcase.expected
-
-    def test_datadir(self, testcase):
-        """Test --datadir."""
-        args = types.SimpleNamespace(confdir=None, cachedir=None,
-                                     datadir=testcase.arg, basedir=None)
-        standarddir.init(args)
-        assert standarddir.data() == testcase.expected
-
-    def test_confdir_none(self, mocker):
-        """Test --confdir with None given."""
-        # patch makedirs to a noop so we don't really create a directory
-        mocker.patch('qutebrowser.utils.standarddir.os.makedirs')
-        args = types.SimpleNamespace(confdir=None, cachedir=None, datadir=None,
-                                     basedir=None)
-        standarddir.init(args)
-        assert standarddir.config().split(os.sep)[-1] == 'qute_test'
-
-    def test_runtimedir(self, tmpdir, monkeypatch):
-        """Test runtime dir (which has no args)."""
-        monkeypatch.setattr(
-            'qutebrowser.utils.standarddir.QStandardPaths.writableLocation',
-            lambda _typ: str(tmpdir))
-        args = types.SimpleNamespace(confdir=None, cachedir=None, datadir=None,
-                                     basedir=None)
-        standarddir.init(args)
-        assert standarddir.runtime() == str(tmpdir / 'qute_test')
 
     @pytest.mark.parametrize('typ', ['config', 'data', 'cache', 'download',
                                      pytest.mark.linux('runtime')])
@@ -239,13 +190,6 @@ class TestInitCacheDirTag:
 
     """Tests for _init_cachedir_tag."""
 
-    def test_no_cache_dir(self, mocker, monkeypatch):
-        """Smoke test with cache() returning None."""
-        monkeypatch.setattr('qutebrowser.utils.standarddir.cache',
-                            lambda: None)
-        mocker.patch('builtins.open', side_effect=AssertionError)
-        standarddir._init_cachedir_tag()
-
     def test_existent_cache_dir_tag(self, tmpdir, mocker, monkeypatch):
         """Test with an existent CACHEDIR.TAG."""
         monkeypatch.setattr('qutebrowser.utils.standarddir.cache',
@@ -356,3 +300,102 @@ class TestSystemData:
         standarddir.init(fake_args)
         monkeypatch.setattr('sys.platform', "potato")
         assert standarddir.system_data() == standarddir.data()
+
+
+class TestMoveWebEngineData:
+
+    """Test moving QtWebEngine data from an old location."""
+
+    @pytest.fixture(autouse=True)
+    def patch_standardpaths(self, tmpdir, monkeypatch):
+        locations = {
+            QStandardPaths.DataLocation: str(tmpdir / 'data'),
+            QStandardPaths.CacheLocation: str(tmpdir / 'cache'),
+        }
+        monkeypatch.setattr(standarddir.QStandardPaths, 'writableLocation',
+                            locations.get)
+        monkeypatch.setattr(standarddir, 'data',
+                            lambda: str(tmpdir / 'new_data'))
+        monkeypatch.setattr(standarddir, 'cache',
+                            lambda: str(tmpdir / 'new_cache'))
+
+    @pytest.fixture
+    def files(self, tmpdir):
+        files = collections.namedtuple('Files', ['old_data', 'new_data',
+                                                 'old_cache', 'new_cache'])
+        return files(
+            old_data=tmpdir / 'data' / 'QtWebEngine' / 'Default' / 'datafile',
+            new_data=tmpdir / 'new_data' / 'webengine' / 'datafile',
+            old_cache=(tmpdir / 'cache' / 'QtWebEngine' / 'Default' /
+                       'cachefile'),
+            new_cache=(tmpdir / 'new_cache' / 'webengine' / 'cachefile'),
+        )
+
+    def test_no_webengine_dir(self, caplog):
+        """Nothing should happen without any QtWebEngine directory."""
+        standarddir._move_webengine_data()
+        assert not any(rec.message.startswith('Moving QtWebEngine')
+                       for rec in caplog.records)
+
+    def test_moving_data(self, files):
+        files.old_data.ensure()
+        files.old_cache.ensure()
+
+        standarddir._move_webengine_data()
+
+        assert not files.old_data.exists()
+        assert not files.old_cache.exists()
+        assert files.new_data.exists()
+        assert files.new_cache.exists()
+
+    @pytest.mark.parametrize('what', ['data', 'cache'])
+    def test_already_existing(self, files, caplog, what):
+        files.old_data.ensure()
+        files.old_cache.ensure()
+
+        if what == 'data':
+            files.new_data.ensure()
+        else:
+            files.new_cache.ensure()
+
+        with caplog.at_level(logging.WARNING):
+            standarddir._move_webengine_data()
+
+        record = caplog.records[-1]
+        expected = "Failed to move old QtWebEngine {}".format(what)
+        assert record.message.startswith(expected)
+
+    def test_deleting_empty_dirs(self, monkeypatch, tmpdir):
+        """When we have a qutebrowser/qutebrowser subfolder, clean it up."""
+        old_data = tmpdir / 'data' / 'qutebrowser' / 'qutebrowser'
+        old_cache = tmpdir / 'cache' / 'qutebrowser' / 'qutebrowser'
+        locations = {
+            QStandardPaths.DataLocation: str(old_data),
+            QStandardPaths.CacheLocation: str(old_cache),
+        }
+        monkeypatch.setattr(standarddir.QStandardPaths, 'writableLocation',
+                            locations.get)
+
+        old_data_file = old_data / 'QtWebEngine' / 'Default' / 'datafile'
+        old_cache_file = old_cache / 'QtWebEngine' / 'Default' / 'cachefile'
+        old_data_file.ensure()
+        old_cache_file.ensure()
+
+        standarddir._move_webengine_data()
+
+        assert not (tmpdir / 'data' / 'qutebrowser' / 'qutebrowser').exists()
+        assert not (tmpdir / 'cache' / 'qutebrowser' / 'qutebrowser').exists()
+
+    def test_deleting_error(self, files, monkeypatch, mocker, caplog):
+        """When there was an error it should be logged."""
+        mock = mocker.Mock(side_effect=OSError('error'))
+        monkeypatch.setattr(standarddir.shutil, 'move', mock)
+        files.old_data.ensure()
+        files.old_cache.ensure()
+
+        with caplog.at_level(logging.ERROR):
+            standarddir._move_webengine_data()
+
+        record = caplog.records[-1]
+        expected = "Failed to move old QtWebEngine data/cache: error"
+        assert record.message == expected

tests/unit/utils/test_urlutils.py

@@ -731,3 +731,8 @@ class TestIncDecNumber:
 
 def test_file_url():
     assert urlutils.file_url('/foo/bar') == 'file:///foo/bar'
+
+
+def test_data_url():
+    url = urlutils.data_url('text/plain', b'foo')
+    assert url == QUrl('data:text/plain;base64,Zm9v')