Always interpret the first word in the command string as the command to
offer completions for, even if that word looks like a flag.
Fixes#3460, where the command string `:-w open` would attempt to offer
completions for `open` but crash because the parsing was thrown off.
By moving the flag-stripping logic to _after_ we determine the command,
`:-w open` interprets `:-w` as the command. Since that is not a valid
command, we won't offer any completions.
Added spaces to comments, use python naming conventions, remove
_load_progress_fake as unnecessary, rename _on_load_*_fake to
_on_load_*_workaround (for less confusion), and use qtutils rather than
QT_VERSION_STR
This uses the much more reliable `loadProgress(100)` in place of
`loadFinished(true)` for WebEngine, with `loadProgressFake` and
`loadFinishedFake` used instead of the 'official' variants.
Update the description to mention the number of columns and change the
default to ["white", "white", "white"] to make it more obvious that
multiple colors can be specified. This also satisfies the config test
that expects the default value for ListOrValue types to be a list.
One other test had to be tweaked to use a config option that is still
just a QtColor rather than a ListOrValue.
While it is possible to provide just two colors, it is "undefined
behavior". It will use the first color as the third color, but that is
an artifact of the implementation and therefore not documented (though
also not an error, as it is harmless).
When QtWebEngine shuts down, it calls pending callbacks, which means we access
an invalid 'settings' object when that happens. The stack would look something
like this:
0 QtWebEngineCore::WebEngineSettings::setAttribute(QtWebEngineCore::WebEngineSettings::Attribute, bool)
12 QtWebEngineCore::CallbackDirectory::invokeEmptyInternal<QVariant const&>(QtWebEnginePrivate::QWebEngineCallbackPrivateBase<QVariant const&>*)
14 QtWebEngineCore::CallbackDirectory::~CallbackDirectory()
19 QWebEnginePage::~QWebEnginePage()
If we instead get the settings from the view freshly, we get a RuntimeError from
PyQt telling us that it's dead. Not sure why it doesn't know about settings
being dead...
With that, we'd get a RuntimeError, which we can simply ignore as it doesn't
matter anyways if the tab is gone.
Fixes#3399
This means something like this:
:hint ;; later 20 follow-hint a ;; later 20 tab-close
Won't crash anymore, as the webelem.Error will be shown in the statusbar.
This gives us slightly different values it seems, but I think they are more
correct (and don't need the "+ 5" above).
Taking the width directly (not the sizeHint width) returned values like 100 or
so when the scrollbar wasn't shown yet, so that seems wrong.
The situation where there's no scroll bar at the point the column widths are
calculated is a very rare one. What happens more often is that the scroll bar
disappears due to filtering, in which case we didn't recalculate the column
widths anyways.
Furthermore, we can even go wrong with this calculation, when we calculate the
column widths while the entire completion is invisible - then the scroll bar
won't be visible either, and we won't subtract the space needed for it.
Let's not try to optimize for this uncommon case, and just always subtract the
scrollbar, even if it's not there initially.
Fixes#3359Closes#3389
Now colors.completion.fg may be set to a list to specify a different
color for each completion column. For example:
:set colors.completion.fg [black,blue,white] will use black text for the
first column, blue for the second, and white for the third.
Setting to a single value still works and behaves as before. The default
is unchanged from 'white'.
Resolves#1794.
command-accept --rapid will run the command without clearing the prompt,
allowing "rapid fire" commands. For example, one could open completion
for `open -t` and open several tabs in a row.
The default binding is ctrl+enter.
Resolves#588.
I mistakenly checked the length of wheres instead of words. This fixes
that check, renames 'wheres' to 'where_clause' to be clear
that it is a string and not an array, and adds a test.
Previously this simply crashed if there was ever malformed utf-8 in the
stderr or stdout streams, perhaps as a result of an incorrectly spawned
command. See e.g. #3222
re.match features an implicit left anchor, which can be surprising.
re.fullmatch features implicit anchors on both sides, but is aptly named
and unsurprising.
re.search has no such implicit anchors, which ought to be the default
even if a single anchor is needed.
Perviously, 'foo bar' would match 'foo/bar' but not 'bar/foo'. Now it
will match both, using a query with a WHERE clause like:
WHERE ((url || title) like '%foo%' AND (url || title) like '%bar%')
This does not seem to change the performance benchmark. However, it does
create a new query for every character added rather than re-running the
same query with different parameters. We could re-use queries if we
maintained a list like self._queries=[1_arg_query, 2_arg_query, ...].
However, it isn't clear that such a complexity would be necessary.
Resolves#1651.
This change makes it so that stderr and stdout is unconditionally read
from for a completed process, and sent to qute://spawn-output. This
allows the user to see the results of the previous process, even if they
had forgotten to use --output.
This fixes the following problems found in a review:
1. Manual modification of the asciidoc has been undone.
2. --output-to-tab has been renamed to the less verbose --output.
3. spawn_output has been changed to spawn-output in the url.
4. Erroneous newline in imports has been removed.
5. output in guiprocess.py has been marked private.
6. If there is no output for either stderr or stdout, say so.
7. Missing space in a text line was added.
8. Redundant initialising of an empty string removed.
This implements the orphaned editor fix for WebKit. Webkit wasn't
crashing before, but this causes webkit to show the same warning
webengine does if the editor is orhpaned (rather than silently
continuing). This allows the same BDD test to pass for both webkit and
webengine.
Instead of rewiring signals on tab.shutting_down, have the webelem check
if its parent tab is deleted, and throw a specific exception.
This is only necessary in WebEngine, Webkit does not crash when the
editor is orphaned.
I tried to write a test for is_deleted, but could not get it to pass:
```
def test_is_deleted(qtbot, view, config_stub, tab_registry, mode_manager):
tab_w = Tab(win_id=0, mode_manager=mode_manager)
qtbot.add_widget(tab_w)
tab_w._set_widget(view)
assert not tab_w.is_deleted()
sip.delete(view)
#assert tab_w.is_deleted()
```
The qtbot post-test cleanup would error due to the deleted view.
This reverts commit e72e8b8556.
Now that the SQL category works in isolation, it is possible to hide
quickmarks/bookmarks when those categories are empty.
Fixes#960
While QSortFilterProxyModel emits layoutChanged when changing the
pattern, QSqlQueryModel emits modelReset. As only layoutChanged was
connected, a HistoryCategory would only work in a model that also had at
least one ListCategory.
The simplest solution is to have the parent model emit the signal
directly. This also emits a single signal on a pattern change rather
that one for each child model.
Resolves#3016.
The signal we were using to inject greasemonkey scripts registered to
run at document-start (javaScriptWindowObjectCleared) was unreliable to
non-existant. The initialLayoutCompleted signal is a bit of an odd duck
too I suppose. Anyway, we don't anticipate any scripts would break from
being injected when the page is finished loaded that wouldn't already
have been flaky due to the complexities of the modern web. If there is
an issue hopefully someone raises an issue and we can look into it.
We weren't actually picking up the @noframes greasemonkey directive
because of this. I haven't tested this very extensively but it seems to
work for making the property value optional.
Use the `QWebPage.frameCreated` signal to get notifications of subframes
and connect the javascript injection triggering signals on those frames
too.
I had to add a `url = url() or requestedUrl()` bit in there because the
inject_userjs method was getting called to early or something when
frame.url() wasn't set or was set to the previous page so we were
passing the wrong url to greasemonkey.scripts_for().
I ran into a bizarre (I maybe it is completely obvious and I just don't
see it) issue where the signals attached to the main frame that were
connected to a partial function with the main frame as an argument were
not getting emitted, or at least those partial functions were not being
called. I worked around it by using None to mean defaulting to the main
frame in a couple of places.
Thanks to @sandrosc. A few breaking changes fixed (default method to
GM_xhr not working, GM_listvalues not cleaning up output, GM_setvalue
param checking logic wrong) and a few hygenic changes made.
Add qute version to GM_info object in GM wrapper.
Support using the greasemonkey @namespace metadata for its intended
purpose of avoiding name collisions.
Get a nice utf8 encoded string from a QUrl more better.
QTWebEngine 5.8 added support for parsing greasemonkey metadata blocks
and scripts added to the QWebEngineScriptCollection of a page or its
profile and then deciding what urls to run those scripts on and at what
point in the load process to run them. For earlier versions we must do
that work ourselves. But with the additional handicap of the less rich
qtwebengine api.
We have acceptNavigationRequest, loadStarted, loadProgress,
loadFinished, urlChanged to choose from regarding points at which to
register scripts for the current page.
Adding scripts on acceptNavigation loadStarted and loadFinished causes
scripts to run too early or too late (eg on the pages being navigated
from/to) and not run on the desired page at the time they are inserted.
We could maybe do some more sophisticated stuff with loadProgress but it
didn't have any better behaviour in the brief testing I gave it.
Registering scripts on the urlChanged event seems to work fine. Even if
it seems like there could be problems with the signal firing too often,
due to not necessarily being tied to the page load progress, that
doesn't seem to have an effect in practice. The event is fired when, for
example, the url fragment changes and even if we add a new script to the
collection (or remove an existing one) it doesn't have an effect on what
is running on the page.
I suspect all of those timing issues is due to the signals being
forwarded fairly directly from the underlying chomium/blink code but the
webengine script stuff only being pushed back to the implementation on
certain events.
Anyway, using urlChanged seems to work fine due to some quirk(s) of the
implementation. That might change with later development but this
codepath is only ever going to be used for version 5.7.
There are other potential optimizations like not removing and then
re-adding scripts for the current page. But they probably wouldn't do
anything anyway, or at least anything that you would expect.
This change requires urls specified in @include, @exclude and @matches
directives in metadata blocks to be in the same form that
QUrl.toEncoded() returns. That is a punycoded domain and percent encoded
path and query. This seems to be what Tampermonkey on chrome expects to.
Also changes the scripts_for() function to take a QUrl arg so the caller
doesn't need to worry about encodings.
This regex was broken since the original PR and subsequent code seemed to be
working around it. Before re.split was returning [everything up to
/UserScript, everything else], now it returns [before UserScript, metadata,
after /UserScript], which is good.
Also I added the check for the UserScript line starting at column 0 as per
spec.
These argument type restrictions are mentioned on the greasespot pages for
these value storage functions. We could call JSON.dumps() instead but better
to push that onto the caller so we don't have to try handle deserialization.
Also removes the check for localstorage because everyone has supported that
for years.
WebEngine only. Previously we were just removing every script from the
main world. But some other scripts might got here in the future so new
we are overriding the name field to add a GM- prefix so hopefully we
only remove greasemonkey scripts before adding new ones.
For qtwebengine 5.8+ only. This is because as of 5.8 some greasemonkey
script support is in upstream. That is, qtwebenginescript(collection)
parses the greasemonkey metadata block and uses @include/match/exclude
to decide what sites to inject a script onto and @run-at to decide when
to inject it, which saves us the trouble. Notes on doing this in <5.8
are below.
Scripts are currently injected into the main "world", that is the same
world as the javascript from the page. This is good because it means
userscripts can modify more stuff on the page but it would be nice if we
could have more isolation without sacrificing functionality. I'm still
looking into why my more feature-full scripts are not having any effect
on the page while running in a separate world.
Userscripts are added to both the default and private profile because I
that if people have scripts installed they want them to run in private mode
too.
We are grabbing the scripts from the greasemonkey module, as opposed to
reading them directly from disk, because the module adds some GM_* functions
that the scripts may expect, and because that is used for webkit anyway.
I have code to support qtwebengine <5.8 but didn't because I am not
happy with the timing of some of the signals that we are provided
regarding page load state, and the actual load state. While the
difference between document-end and document-idle isn't so bad,
injecting document-start scripts when a urlChanged event is emitted
results in the script being injected into the environment for the page
being navigated away from. Anyway, if anyone wants this for earlier
webengines I can oblige them.
Supposed to be after all the assets have finished loading and in page js
has run. Not that we can garuntee that last bit. If a script misbehaves
because a precondition isn't yet met I suggest adding a defer method to
the script that adds a timer until the precondition is met.
Also changed the map/filter calls to use list comprehensions to keep
pylint happy. Even if it does look uglier.
WebKit backend only for now. Loads all .js files from a directory,
specified in the greasemonkey-directory key in the storage section,
defaulting to data/greasemonkey, and wraps them in a minimal environment
providing some GM_* functions. Makes those scripts available via the
"greasemonkey" registered object in objreg and injects scripts at appropriate
times in a page load base on @run-at directives.
I can't reproduce this, but someone on KDE reported always getting a crash (as
msg.splitlines()[0] gives an IndexError) when trying to select a file with
Qt 5.9.3.
If an editor is open on a form in a tab and that tab is closed, rewire
the callback to print a warning. Previously, the callback would access a
deleted C++ object and cause a crash.
Resolves#2758.
Updated requirements and adjusted the configuration in `.flake8`; other
files have been modified where the lack of per-file auto-ignore caused
problems, where putty's `# flake8: disable=` syntax could be replaced
with a simpler `noqa`, or where pylint directives already suppressed the
same error.
Show an error message if the user edits the command such that it is
missing a start character (:, /, or ?). Previously, this would cause the
browser to crash.
Resolves#3326.
While 64-bit values are allowed with QtWebKit/QNetworkDiskCache, QtWebEngine
only allows 32-bit values here. With the updated sip's strict overflow checking,
that means we get an exception when setting a too big value.
:edit-command opens the current command line in an editor, and updates
the command line if the editor exits successfully. If --run is passed,
the command is executed when the editor exits sucessfully.
Resolves#2453.
If there are no quickmarks/bookmarks, hide the entire category in url
completion. Note that this only hides the category if
quickmarks/bookmarks is empty to begin with. An empty category is still
shown if the completion pattern filters out all items in that category.
See #960.
This changes the undo stack from a list of UndoEntry objects to a list
of lists of UndoEntry objects, so groups of tabs can be added. Only
:tab-only does that, but it's exposed by TabbedBrowser.close_tab as a
keyword argument.
With subprocess.run, we wait until the subprocess has completed, which means the
parent process will hang on :restart.
Since we *don't* want to wait for the subprocess here, using subprocess.Popen
seems the right thing to do.
This was introduced in bb54a954fe / #3203
Probably doesn't affect #3210 and #3220.
By default, ctrl+c will yank the selection from the completion menu onto
the clipboard, and ctrl+shift+c will yank it onto the primary selection.
Unfortunately ctrl+y was already taken by rl-yank (which,
counter-intuitively to vim users, will paste the last deleted text).
When doing :unbind with a default keybinding the first time, it gets inserted
into bindings.commands with None as value.
When then doing :unbind a second time, instead of just leaving that None value
as-is, we removed it again (because it got treated as a custom binding).
Fixes#3162
This reverts commit acfb3aa26f.
The related code doesn't really belong in webenginesettings.py after all, and
for some reason I don't understand right now this breaks tests in
javascript.feature because window._qutebrowser is undefined when running them.
* Keep descriptions concise.
* Prefer starting descriptions with a noun.
* Don't explain that placeholders are placeholders/get replaced.
* Introduce placeholder list by the following line:
'The following placeholders are defined:'
* List placeholders in a markdown-style list:
'* `{<placeholder>}`: <description>.'
* for selections, use appropriate determiner/pro-adverb/pronoun:
* 'When to [...].'
* 'How to [...].'
* 'Which [...] to [...].'
* 'What [...] to [...].'
* available placeholders are defined for tabs.title.format
* tabs.title.format_pinned refers to that definition
* window.title.format repeats the identical(!) definition
* this replaces the repetion by another reference
I previously removed the sorting logic from SortFilter thinking it was
unnecessary if we construct the model with a sorted list. However, this
only worked when no pattern was set, and the items are misordered as
soon as a pattern is input.
This patch reintroduces alpha-sorting, which can be disabled by passing
sort=False to the ListCategory constructor. The session completion test
had to be tweaked as it simulated the incorrect assumption that the
session list is not alpha-ordered; sessions come out of the
session-manager pre-sorted so we may as well use alpha-sorting in the
session completion model.
Resolves#3156.
This now works correctly in XML documents. The stylesheet is applied at
document creation to reduce flickering, and is updated if the
user_stylesheets setting is changed after page load.
The command :edit-url --private (or :edit-url -p) will spawn a new
private window with the url input from the editor.
I had to add 'Given I have a fresh instance' to the feature file to
ensure tests were not interfering.
Resolves#3185.
Instead of setting `_ignore_change` to `True` before calling
`_change_completed_part` we just stop `_cmd` from emitting
`update_completion`.
This has the nice side-effect that when writing a complete command
`_ignore_change` was set to `True` regardless, and thus hitting space
would not update the completion view.
Now, hitting space will (as always) schedule a completion update that
now will not be incorrectly ignored
The radius for the keyhint dialog box should be configurable vi via
c.keyhint.radius. The default was set to 6px, which is the previous
hardcoded value.
Added placeholders are:
* `{file}` has the same function as `{}`
* `{line}` is the 1-based line number
* `{column}` is the 1-based column number
* `{line0}` like `{line}`, but 0-based
* `{column0}` like `{column}` but 0-based
With the previous commit, we also checked that PyQt was >= 5.7.1, but we want to
support PyQt 5.7.0. Instead, we now check the individual components by hand.
Also, the previous check accidentally allowed PyQt >= 5.2.0 instead of 5.7.0.
With an older PyQt built against a newer Qt, we still don't have its features
available.
This also drops support for exact=True with compiled=True as the semantics for
that are unclear, and it's not used.
This also renames 'strict' to 'compiled' to be more descriptive.
It also fixes a crash when starting qutebrowser with an older compiled Qt
version which was introduced recently (calling setSpellCheckEnabled).
QtWebEngine emits scrollPositionChanged a lot during smooth scrolling, and
there's no reason we need to update percentages when they didn't *actually*
change.
This reduces the updates with a single spacebar press from 6-7 to 2-3 on my
machine, which might not be enough though.
See #2233
The keyhintwidget was not showing up when a keychain was prefixed with a
count. For example, 'g' would show a keyhint but '5g' would not. Now
keyhints are shown even when a count is given.
Resolves#3045.
Turns out --force is just in the way for most people, and at least for default
bindings it's easy to reset them.
Also, it makes :config-source fail when config.py contains keybindings.
Closes#3049
Ideally, we'd update all existing tables to add the new constraints, but sqlite
doesn't offer an easy way to do so: https://www.sqlite.org/lang_altertable.html
Since that migration really isn't worth the effort, we only set the constraint
for new tables...
Before, we allowed :set to take multiple values, which often lead to confusing
error messages when a user forgot to quote the value.
Now, we instead have a dedicated :config-cycle command for that.
See #1840, #2794
This renames SqlException to SqlError (to be more consistent with how Python
names exceptions), and adds an utility function which logs a few more useful
details about errors.
See #3004
Previously, indicator_padding was not taken into account, causing
problems when using a indicator_padding too small
Also removed icon padding to width calculation (seemed to be overestimating)
We removed various caches in b5eac744b5 but the
completion delegate stylesheet gets rendered a lot, causing things to slow down.
The rendering takes around 1ms, but it gets done ~10k times with a simple
profiling run, so that adds up quickly.
We don't use a functools.lru_cache here as the stylesheet template never
changes.
Thanks a lot to gilbertw1 for tracking this down!
See #2812 - there's probably more possible, but this should fix the performance
regression some people saw with the new config.
Previously, a successful import of the text history into sqlite would
move 'history' to 'history.bak'. If history.bak already existed, this
would overwrite it on unix and fail on windows.
With this patch, the most recently imported history is appended to
history.bak to avoid data loss.
Resolves#3005.
A few other options I considered:
1. os.replace:
- fast, simple, no error on Windows
- potential data loss
2. numbered backups (.bak.1, .bak.2, ...):
- fast, no data loss, but more complex
3. append each line to the backup as it is read:
- more efficient than current patch (no need to read history twice)
- potentially duplicate data if backup fails
This is done so config.py can import other python files in the config
directory. For example, config.py can 'import theme' which would load
a theme.py.
The previous path is restored at the end of this function, to avoid
tainting qutebrowser's path
Windows: The instructions are outdated and not really relevant anymore with the
standalone packages;
pip: Let's recommend tox/virtualenv by just linking to the install docs.
Closes#2998
There are just way too many gotchas related to valid modes, aliases, and
circular dependencies when validating aliases/bindings in the config.
Let's just remove this and let invalid commands fail late, when they're actually
used.
I considered introducing another list of deleted options (or a "deleted: True"
in configdata.yml), similar to what we had with the old config.
However, let's take the easier route and just delete everything we don't know
from configdata.yml. If someone edits it by hand, it's their fault :P
See #2772, #2847
The parsing bind() did manually is now available through CommandParser.
Resolves#2952.
This also adds a unit test for the case when there is no current
binding, as I broke that while working on this and there was no test to
catch it :)
Importing qtutils for version_check needs pkg_resources, so we need to check
that's available earlier.
Also, import pkg_resources also shows warnings on older setuptools versions
because of invalid escapes, so we need to import it with warnings suppressed.
Includes a test for persistence of intermediate mutations in a
configuration file (i.e. more than one update) and a switch of the
_mutable attribute in configurations to a dictionary of (old, new)
values rather than (name, old, new). get_obj() now checks for an
existing mutable value and returns a reference to that value, only
making an initial copy; this preserves changes between update_mutables()
Versions before v0.9.0 (which didn't even support hinting with QtWebEngine!)
used to write QtWebEngine data to:
~/.local/share/qutebrowser/qutebrowser/QtWebEngine/Default
~/.cache/qutebrowser/qutebrowser/QtWebEngine/Default
In v0.9.0 this was changed to:
~/.local/share/qutebrowser/webengine
~/.cache/qutebrowser/webengine
Now we don't try to migrate data from the old location anymore.
We were only rendering .html files before, so the old _guess_autoescape function
had the effect of always autoescaping .render() (from a file) but never
autoescaping .from_string(). However, most places using .from_string() actually
render (Qt-)HTML via jinja, so they should escape stuff!
Now, we always autoescape, except when the caller uses the
jinja.environment.no_autoescape() context manager, which places rendering
stylesheets now do.
This impacted:
- Confirm quit texts (no HTML here)
- config.py loading errors
(where this was found because of an error containing - a <keybinding>)
- Certificate error prompts
(should be fine from what I can tell, as the only user-controllable output is
the hostname, which cannot contain HTML)
This also changes qute://help to show the documentation generation error if a
help page wasn't found. This way, people who pull from git but not re-generate
the documentation hopefully get the idea.
Those now look at the history again.
Looking at the behavior in different applications:
- vim: History
- spacemacs: Completion if open, else history
- luakit: Completion if open, else history
- dwb: Always completion (has no history?)
- vimb: Nothing if completion open, else history
- vimperator: Always history
So this is consistent with at least some of them - the much more important
factor is that <Tab> is probably intuitively easy to discover if up/down doesn't
do what's expected, but <ctrl-p>/<ctrl-n> are not.
We accidentally did show the command as a list in to_str(). However, after
correcting that to use shlex.escape, we got ugly qutebrowser command lines
when tabbing to the default value, because of how shlex handles double-escaping:
>>> print(shlex.quote("gvim -f '{}'"))
'gvim -f '"'"'{}'"'"''
While in this case, outputting "gvim -f '{}'" would be much more appropriate, it
doesn't look like we can teach shlex.quote to do that.
Instead, we now only accept a list as input for ShellCommand, at the price that
the user needs to do
:set editor.command '["gvim", "-f", "{}"]'
instead of
:set editor.command 'gvim -f {}'
Fixes#2962.
Should default to -1, not 1000 as the new history completion is better
able to handle large numbers of entries. I believe this was acidentally
reset to 1000 while fixing a merge conflict.
Also re-run src2asciidoc.
In python3.4, there is a circular dependency between the config module
and configmodel.bind. This is resolved by dependency injection. The
config/keyconfig instances are embedded in a struct passed to every
completion function, so the functions no longer depend on the modules.
This will also enable completion functions to access other previously
inaccessible info, such as the window id.
See #2814.
When upgrading from an old table that used different url formatting, two
entries might map to the same key, so we'll need to replace the previous
entry to avoid a primary key conflict.
Incrementing _USER_VERSION in the source will cause the
HistoryCompletion table to regenerate when users update.
This is currently necessary to support some recent formatting fixes, but
could be incremented again in the future for other changes.
When in miscmodels, the config module was unable to find the function.
It appears to be some sort of circular import issue:
```
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/app.py", line 44, in <module>
from qutebrowser.completion.models import miscmodels
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/miscmodels.py", line 24, in <module>
from qutebrowser.completion.models import completionmodel, listcategory, util
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/util.py", line 24, in <module>
from qutebrowser.config import config
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 223, in <module>
class ConfigCommands:
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 314, in ConfigCommands
@cmdutils.argument('command', completion=miscmodels.bind)
AttributeError: module 'qutebrowser.completion.models.miscmodels' has no attribute 'bind'
```
As configmodel imports util (and thereby config as well) it is unclear
to me why moving bind() to configmodel actually fixes this, but it does.
If the HistoryCompletion table is removed, regenerate it from the
History table. This allows users to manually edit History, then remove
HistoryCompletion to prompt regeneration.
See #2903.
Encode urls that are inserted into the history, but do not encode urls
for completion (other than removing passwords).
Also ensure that urls read from the history text file are formatted
consistenly with those added while browsing.
Fixes#2903.
Get qutebrowser to the point where it can at least start
- Declare _messages earlier in MessageView.__init__ so it is set before
the config trigger tries to access it.
- Remove unused configmodel completion functions
- Move bind completion to configmodel to avoid a circular import with
the config module
- Fix some config accesses (forgot to use .val)
- Fix old Completion.CompletionKind references
Instead, expect the data to be given in the desired order. Completion
functions should sort their data _if_ they want it sorted in the
completion. This has a few implications:
- {book,quick}marks appear in the same order they do in the text file.
This means users can rearrange their mark files for custom sorting.
Fixes#2354
- Sessions are sorted as they appear in the session manager
- Tabs are sorted numerically, not alphabetically (Fixes#2883)
Note that prefix-based filter sorting is still performed, so items
starting with the filter pattern come first.
The added/removed signals for the urlmark managers are no longer used as
the completion models are generated on-the-fly. The changed signal is
still needed so the save-manager knows when to trigger a write to disk.
Also removes session_manager.update_completion, which is no longer
needed for the same reason as above.
keyconf.changed cannot be removed, as it is still wired up to
basekeyparser.
Resolves#2874.
Fixes#2868, where pressing <shift-tab> then <ctrl-d> in history
completion (with > 256 items) would cause later items to disappear (and
cause a crash if you try to delete again).
Cause:
Scrolling to the bottom would fetch an additional 256 items (in addition
to the 256 that are fetched at first). Deleting causes the query to
re-run, but it only fetches the initial 256 items, so the current index
is now invalid.
Fix:
After deleting from the history category, call fetchMore until it has
enough rows populated that the current index is valid.
When tabbing to the last index of history completion, call expandAll
which will call fetchMore to retrieve more query results, if available.
Calling fetchMore directly will not update the view, and for some
reason self.expand(idx.parent()) and
self.expand(self.model().index(idx.row(), 0)) did not work, so I'm using
expandAll.
Fixes#2841.
When a key is bound to a command line that includes one or more
arguments to a command, bind completion should show the whole command
for the "Current" category, and use only the command name to look up the
description.
Fixes#2859, where a crash was caused by looking up the description by
the full command text rather than just the name.
Sometimes, we get another error with "Renderer process was killed" and the data:
URL for the error page. This is probably because the renderer process wasn't
restarted yet. This hopefully helps.
Override removeRows instead of removeRow.
> removeRow is not virtual in C++, so if this gets called by Qt
> internally for some reason, it wouldn't use the overloaded version -
> so I think it'd be better to implement removeRows and then use
> removeRow without overloading that
- The-Compiler
It doesn't make sense to have an active selection while you are
filtering by entering text. You should be in one of two states:
1. Tabbing through completions (valid selection)
2. Entering a filter pattern (invalid selection)
Fixes#2843, where a crash would occur after the following:
1. tab to an item other than the first
2. <backspace>
3. re-type last character
4. <ctrl-d>
This would try to delete an out of range index.
Even though no item was deleted, it was manipulating the completion
model because beginRemoveRows was called before the exception was
raised.
This fixes that problem by moving the removal logic (and delete_func
check) into the parent model, so it can check whether deletion is
possible before calling beginRemoveRows.
Fixes#2839.
We get no last_atime limit at all otherwise:
qutebrowser.misc.sql.SqlException: Failed to prepare query "SELECT url, title,
strftime('%Y-%m-%d', last_atime, 'unixepoch', 'localtime') FROM
CompletionHistory WHERE (url LIKE :pat escape '\' or title LIKE :pat escape '\')
AND last_atime >= ORDER BY last_atime DESC": "near "ORDER": syntax error Unable
to execute statement"
We need to tell sqlite to convert the timestamps to localtime during
formatting, otherwise it formats them as though you are in UTC.
Also fix up a few uses of mktime.
For some reason, calling search.clear() while no search is displayed causes the
backends to un-focus inputs, and with QtWebKit, even hinting can't focus them
again after that.
For performance, re-introduce web-history-max-items.
As the history query has now become a very specific multi-part query and
history completion was the only consumer of SqlCategory, SqlCategory is
now replaced by a HistoryCategory class.
The javascript history page was requesting the new start_time in ms, but
the python code was expecting seconds. This is fixed by removing all the
millisecond translations in the python code and only translating to
milliseconds in the javascript code that formats dates.
- Fix flake8
- history.clear should also clear completion table
- call _resize_columns in set_model, not set_pattern
- add more unit-testing for the history completion table
- Ignore invalid variable name in flake8 (pylint already checks this and
we don't want to have to double-ignore)
- Fix and test completion bug with `:set asdf `
- Remove unused import
- Use `assert not func.called` instead of `func.assert_not_called` for
backwards compatibility
- Fix comment and empty line check in _parse_entry
- connect layoutAboutToBeChanged signal
- assert sort_order is None if sort_by is None
- modify sql init failure message to ask about Qt sqlite support.
Check for interval being positive instead of checking for it to be
non-zero. So if somehow some unexpected thing happend and made
message-timeout negative, the bug doesn't cascade.
- remove outdated comment
- fix sql init error message
- clean up history text import code
- fix test_history file path in coverage check
- use real web history, not stub, for completion model tests
- use qtmodeltester in sql/list_category tests
- test url encoding in history tests
- fix test_clear by using a callable mock
- remove test_debug_dump_history_oserror as the check is now the same as
for the file not existing
- rename nonempty to data in test_completionmodel
- add more delete_cur_item tests
- test empty option/value completion
If creating the sql database fails, show an error dialog assuming sqlite
is not installed.
This removes the isDriverAvailable check as it was true even with sqlite
uninstalled.
sql.version now inits itself if sql is not already initialized and
prints 'UNAVAILABLE (<error message>)' if init fails. This is to avoid
cascading errors, where one error would create a crash dialog, which
calls sql.version, which would create another error.
This is also needed to make the docs environment work on Travis - as otherwise,
doc generation wasn't deterministic because of changing dict key order.
In Dict.to_str() and List.to_str() we use json.dump to get a value. However,
JSON includes surrogate escapes in the dumped values, which breaks round trips.
>>> yaml.load(json.dumps({'\U00010000': True}))
{'\ud800\udc00': True}
>>> yaml.load(json.dumps({'\U00010000': True}, ensure_ascii=False))
yaml.reader.ReaderError: unacceptable character #x10000: special characters are not allowed
See:
https://stackoverflow.com/a/38552626/2085149https://news.ycombinator.com/item?id=12798032
Now that the StyleSheetObserver is a child of the object it observes, it should
get cleaned up properly when the object is deleted.
This means this is hopefully not needed anymore, even on Qt 5.2.
This gets called a lot, and caused some :bind calls to take ~3s.
Stats after starting with a bit of :bind:
CacheInfo(hits=25917, misses=139, maxsize=256, currsize=139)
Now the "object" kind of value (like in YAML) is stored internally, and that's
the canonical value. The methods changed their meaning slightly, see the
docstring in configtypes.py for details.
- Fix outdated comments
- Use mock specs when possible
- More precise error message check in test_import_txt_invalid.
- Fix copyright message
- Tweak missing pyqt error message
- Dead code: remove group_by and where from sqlcategory.
With the new separate completion table, these are no longer used.
- Move test_history out of webkit/. History is no longer purely webkit
related, it could be webengine.
Show a graphical error box with install instructions if PyQt.QtSql is
not found, rather than failing with CLI errors. Also show an error box
if the sqlite driver is not available.
This ensures we actually know when an AttributeError happens.
It also changes most external code to use the correct environment, rather than
simply creating a jinja2.Template, which wouldn't use the more tightened
environment.
When binding a key, the first row will be the current binding if the key
is already bound. This should make it easier for users to tell when they
are binding a key that is already bound, and what it is bound to.
There were two issues here:
- The comparison was backwards, causing scroller.at_bottom() to always return
true.
- When zoomed in, jsret['px']['y'] can be a float, which means we can be
slightly off when checking the difference - math.ceil() fixes that.
With the new completion API, we no longer need a custom filterAcceptsRow
function. This was necessary to handle the tree structure of the model,
but now we use a separate QSortFilterProxyModel for each category, so
the data it filters is flat. We can simplify the code by using the
builtin setFilterRegExp.
This changes the behavior a little, as now all list categories filter on
all columns. This should be beneficial if anything. For example, help
topics are now filtered on description in addition to name.
This also seems to slightly speed up filtering, according to the url
model benchmark.
Before:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.2806 1.3817 1.3195 0.0390 1.3068 0.0487 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
After:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.1183 1.1508 1.1281 0.0132 1.1241 0.0142 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
Taking the completion widget as an argument was overly complex.
The process now looks like:
1. CompletionView gets deletion request
2. CompletionView passes selected index to CompletionModel
3. CompletionModel passes the row data to the owning category
4. The category runs its custom completion function.
This also fixes a bug. With the switch to the hybrid (list/sql)
completion model, the view was no longer updating when items were
deleted. This fixes that by ensuring the correct signals are emitted.
The SQL model must be refreshed by running the query. We could try using
a SqlTableModel so we can call removeRows instead.
The test for deleting a url fails because qmodeltester claims the length
of the query model is still 3.
For QSqlQueryModel, the argument should always be an invalid index:
http://doc.qt.io/qt-5/qsqlquerymodel.html#canFetchMore
For a QStandardItemModel, it doesn't matter. Either way, passing the
top-level parent index was wrong.
_insert_query gets called with a query and dict of values such as:
{'val': 1, 'lucky': False, 'name': 'one'}
Via bindValues(), we only assign a placeholder in the query string to a value,
so we get a query with bindings like:
INSERT INTO Foo values(:lucky,:val,:name)
{':name': 'one', ':val': 1, ':lucky': False}
So what we're executing is something like:
INSERT INTO Foo values(false,1,"one")
However, if the column order in the database doesn't happen to be the order
we're passing the values in, we get the wrong values in the wrong columns.
Instead, we now do:
INSERT INTO Foo (lucky, val, name) values(false,1,"one")
Which inserts the values in the order we intended.
With Python 3.6, this just happened to work before because we always passed the
keyword arguments in the table column order, and in 3.6 dicts
(and thus **kwargs) happen to be ordered:
https://mail.python.org/pipermail/python-dev/2016-September/146327.html
On Travis CI we are sometimes seeing:
```
CompletionView.selection_changed[str].emit():
argument 1 has unexpected type 'int'
```
Cast the data to a string before emitting it just to be safe.
This allows replace to be a named parameter and allows consolidating
some duplicate code between various insert methods.
This also fixes some tests that broke because batch insert was broken.
No longer needed with sql backend. Query results build their own
namedtuple from the returned columns, and inserting new entries is just
done with named parameters.
Fix the issue where pressing `o<esc>o` would show a url completion
dialog where attempting to <Tab> select items would do nothing but show
a Qt warning.
The fix is to ensure we set _last_completion_func to None whenever we
clear completion (there was a case I missed).
It also ensures we always delete the old model and adds a safety to
prevent deleting an in-use model is set_model is called with the current
model.
This was changed during code review but was causing Qt errors while
TAB-completing in the selection view:
08:42:34 WARNING qt Unknown module:none:0 Can't select indexes from different model or with different parents
before
------
sqlite> SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
Run Time: real 0.072 user 0.063334 sys 0.010000
sqlite> explain query plan SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
0|0|0|SCAN TABLE History
0|0|0|USE TEMP B-TREE FOR ORDER BY
sqlite> explain query plan select url, title, strftime('%Y-%m-%d', last_atime, 'unixepoch') from CompletionHistory where (url like "%qute%" or title like "%qute%") order by last_atime desc;
0|0|0|SCAN TABLE CompletionHistory
0|0|0|USE TEMP B-TREE FOR ORDER BY
after
-----
sqlite> SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
Run Time: real 0.000 user 0.000000 sys 0.000000
sqlite> explain query plan SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
0|0|0|SEARCH TABLE History USING INDEX AtimeIndex (atime>? AND atime<?)
sqlite> explain query plan select url, title, strftime('%Y-%m-%d', last_atime, 'unixepoch') from CompletionHistory where (url like "%qute%" or title like "%qute%") order by last_atime desc;
0|0|0|SCAN TABLE CompletionHistory USING INDEX CompletionAtimeIndex
When loading heise.de, for some crazy reason QtWebKit calls historyContains
about 16'000 times.
With this cache (which we simply clear when *any* page has been loaded, as then
the links which have been visited can change), that's down to 250 or so...
- Show an error message when import fails, not a generic crash dialog
- Raise CommandError when debug-dump-history fails
- Check that the path exists for debug-dump-history
This is called often, hopefully a prepared query will speed it up.
This also modifies Query.run to return self for easier chaining, so you
can use `query.run.value()` instead of `query.run` ; query.value()`.
For real this time. A mistake on the last commit like this meant models
were still spuriously instantiated.
Now that the completion model is reused, the layoutChanged signal needs
to be forwarded through, otherwise the view will not update.
This seemed to have a significant performance impact. Removing it means
that instead of just seeing the most recent atime for a given url, you
will see multiple entries.
If the completion model would stay the same, just keep it and update the
filter pattern rather than instantiating a new model each time the
pattern changes.
Instead set this on inidividual categories, as that is where it actually
gets used. This makes it easier for SqlCompletionCategory to reuse a
prepared query (as it gets the filter field names in its constructor).
Trying to read from the sql database from another process was flaky.
This adds a debug-dump-history command which is used by the history BDD
tests to validate the history contents.
It outputs history in the old pre-SQL text format, so it might be
useful for those who want to manipulate their history as text.
Returning "next" was no longer possible as the SQL query does not fetch
more items than necessary. This is solved by using a start time, a
limit, and an offset. The offset is needed to prevent fetching duplicate
items if multiple entries have the same timestamp.
Two of the history tests that relied on qute://history were changed to
rely on qute://history/data instead to make them less failure-prone.
The history completion query is extended to pick only the most recent item for
a given url.
The tests in test_models now check for ordering of elements.
The old implementation was looping through the whole history list, which for
SQL was selecting every row in the database. The history benchmark was taking
~2s. If this is rewritten as a specialized SQL query, the benchmark takes
~10ms, an order of magnitude faster than the original non-SQL implementation.
Vulture exposed the following dead code:
- AppendLineParse was only used for reading the history text file, which is now
a sql database (and the import code for the old text file is simpler and does
not need a complex line parser)
- async_read_done is no longer used as importing the history text file is
synchronous (and should only happen once)
- config._init_key_config is unused as it was moved to keyconf.init
Calling sql.init() in version.version() would replace the existing sql
connection and cause a crash when accessed by opening qute://version.
Now version relies on sql already being initted, and app.py inits sql early if
the --version arg is given.
Turns out historyContains was getting called for the webkit backend multiple
times when the browser starts. This was calling `url in history`, which was
enumerating the entire history as `__contains__` was not defined.
Instead of skipping bad history lines during the import to sql, fail hard. We
don't want to delete the user's old history file if we couldn't parse all of
the lines.
Now that sql is only used for history (not quickmarks/bookmarks) a number of
functions are no longer needed. In addition, primary key support was removed as
we actually need to support multiple entries for the same url with different
access times. The completion model will have to handle this by selecting
something like (url, title, max(atime)).
This also fixes up a number of tests that were broken with the last few
sql-related commits.
If qutebrowser detects a history text file when it starts
(~/.local/share/qutebrowser/history by default on Linux), it will import this
file into the new sqlite database, then delete it.
The read is done as a coroutine as it can take some time.
Instead of reading sqlite history from a file and storing it in an in-memory
database, just directly use an on-disk database. This resolves#755, where
history entries don't pop in to the completion menu immediately as they are
still being read asynchronously for a few seconds after the browser starts.
Instead of add_list and add_sqltable, the completion model now supports
add_category, and callees either pass in a SqlCategory or ListCategory. This
makes unit testing much easier.
This also folds CompletionFilterModel into the ListCategory class.
The RFC on moving from plaintext to SQL storage (#2340) showed that many would
be upset if bookmarks and quickmarks were no longer stored in plaintext.
This commit uses list-based completion for quickmarks and bookmarks. Now the
history storage can be moved from plaintext to an on-disk SQL database while
leaving bookmarks and quickmarks as-is.
Now all completion models are of a single type called CompletionModel.
This model combines one or more categories. A category can either be a
ListCategory or a SqlCategory.
This simplifies the API, and will allow the use of models that combine simple
list-based and sql sources. This is important for two reasons:
- Adding searchengines to url completion
- Using an on-disk sqlite database for history, while keeping bookmarks and
quickmars as text files.
This was a performance optimization that shouldn't be needed with the new SQL
history backend. This also removes support for the LIMIT feature from SqlTable
as it only existed to support web-history-max-items.
Respond to the low-hanging code review fruit:
- Clean up some comments
- Remove an acidentally added duplicate init_autosave
- Combine two test_history tests
- Move test_init cleanup into a fixture to ensure it gets called.
- Name the _ argument of bind(_) to _key
- Ensure index is valid for first_item/last_item
- Move SqlException to top of module
- Rename test_index to test_getitem
- Return QItemFlags.None instead of None
- Fix copyright dates (its 2017 now!)
- Use * to force some args to be keyword-only
- Make some returns explicit
- Add sql to LOGGER_NAMES
- Add a comment to explain the sql escape statement
newest_slice is no longer needed after the completion refactor. Now that
history is based on the SQL backend, LIMIT is used instead.
StatusBar._option is not used, though I'm not sure why vulture only caught it
now.
The new function-based completion API introduced a circular import:
config -> keyconf -> miscmodels -> config.
config only depended on keyconf so it could initialize it as part of
config.init. This can be resolved by moving this to keyconf.init and
initializing keyconf as part of app.init.
Change the logging to report the completion function name and have the end2end
tests check for this.
Remove the tests for realtime completion, as it was decided this is not an
important feature and the code is much simpler without it.
This just forwards canFetchMore and fetchMore to the underlying tables.
It seems to be returning True and fetching in some cases (with a large
history), so I guess it is useful?
Instead of returning a regular tuple and trying to remember which index maps to
which field, return named tuples that allow accessing the fields by name.
Allow categories to specify a WHERE clause that applies in addition to the
pattern filter. This allows the url completion model to filter out redirect
entries.
This also fixed the usage of ESCAPE so it applies to all the LIKE statements.
A SQL completion category can now provide a customized column expression for
the select statement. This enables the url model to format timestamps, as well
as rearrange the name and url in the quickmark section.
This allows setting the query as a QSqlQuery instead of a string, which allows:
- Escaping quotes
- Using LIMIT (needed for history-max-items)
- Using ORDER BY (needed for sorting history)
- SELECTing columns (needed for quickmark completion)
- Creating a custom select (needed for history timestamp formatting)
