The parsing bind() did manually is now available through CommandParser.
Resolves#2952.
This also adds a unit test for the case when there is no current
binding, as I broke that while working on this and there was no test to
catch it :)
We were only rendering .html files before, so the old _guess_autoescape function
had the effect of always autoescaping .render() (from a file) but never
autoescaping .from_string(). However, most places using .from_string() actually
render (Qt-)HTML via jinja, so they should escape stuff!
Now, we always autoescape, except when the caller uses the
jinja.environment.no_autoescape() context manager, which places rendering
stylesheets now do.
This impacted:
- Confirm quit texts (no HTML here)
- config.py loading errors
(where this was found because of an error containing - a <keybinding>)
- Certificate error prompts
(should be fine from what I can tell, as the only user-controllable output is
the hostname, which cannot contain HTML)
In python3.4, there is a circular dependency between the config module
and configmodel.bind. This is resolved by dependency injection. The
config/keyconfig instances are embedded in a struct passed to every
completion function, so the functions no longer depend on the modules.
This will also enable completion functions to access other previously
inaccessible info, such as the window id.
See #2814.
When in miscmodels, the config module was unable to find the function.
It appears to be some sort of circular import issue:
```
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/app.py", line 44, in <module>
from qutebrowser.completion.models import miscmodels
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/miscmodels.py", line 24, in <module>
from qutebrowser.completion.models import completionmodel, listcategory, util
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/util.py", line 24, in <module>
from qutebrowser.config import config
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 223, in <module>
class ConfigCommands:
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 314, in ConfigCommands
@cmdutils.argument('command', completion=miscmodels.bind)
AttributeError: module 'qutebrowser.completion.models.miscmodels' has no attribute 'bind'
```
As configmodel imports util (and thereby config as well) it is unclear
to me why moving bind() to configmodel actually fixes this, but it does.
Get qutebrowser to the point where it can at least start
- Declare _messages earlier in MessageView.__init__ so it is set before
the config trigger tries to access it.
- Remove unused configmodel completion functions
- Move bind completion to configmodel to avoid a circular import with
the config module
- Fix some config accesses (forgot to use .val)
- Fix old Completion.CompletionKind references
Instead, expect the data to be given in the desired order. Completion
functions should sort their data _if_ they want it sorted in the
completion. This has a few implications:
- {book,quick}marks appear in the same order they do in the text file.
This means users can rearrange their mark files for custom sorting.
Fixes#2354
- Sessions are sorted as they appear in the session manager
- Tabs are sorted numerically, not alphabetically (Fixes#2883)
Note that prefix-based filter sorting is still performed, so items
starting with the filter pattern come first.
Fixes#2868, where pressing <shift-tab> then <ctrl-d> in history
completion (with > 256 items) would cause later items to disappear (and
cause a crash if you try to delete again).
Cause:
Scrolling to the bottom would fetch an additional 256 items (in addition
to the 256 that are fetched at first). Deleting causes the query to
re-run, but it only fetches the initial 256 items, so the current index
is now invalid.
Fix:
After deleting from the history category, call fetchMore until it has
enough rows populated that the current index is valid.
When tabbing to the last index of history completion, call expandAll
which will call fetchMore to retrieve more query results, if available.
Calling fetchMore directly will not update the view, and for some
reason self.expand(idx.parent()) and
self.expand(self.model().index(idx.row(), 0)) did not work, so I'm using
expandAll.
Fixes#2841.
When a key is bound to a command line that includes one or more
arguments to a command, bind completion should show the whole command
for the "Current" category, and use only the command name to look up the
description.
Fixes#2859, where a crash was caused by looking up the description by
the full command text rather than just the name.
Override removeRows instead of removeRow.
> removeRow is not virtual in C++, so if this gets called by Qt
> internally for some reason, it wouldn't use the overloaded version -
> so I think it'd be better to implement removeRows and then use
> removeRow without overloading that
- The-Compiler
It doesn't make sense to have an active selection while you are
filtering by entering text. You should be in one of two states:
1. Tabbing through completions (valid selection)
2. Entering a filter pattern (invalid selection)
Fixes#2843, where a crash would occur after the following:
1. tab to an item other than the first
2. <backspace>
3. re-type last character
4. <ctrl-d>
This would try to delete an out of range index.
Even though no item was deleted, it was manipulating the completion
model because beginRemoveRows was called before the exception was
raised.
This fixes that problem by moving the removal logic (and delete_func
check) into the parent model, so it can check whether deletion is
possible before calling beginRemoveRows.
Fixes#2839.
We get no last_atime limit at all otherwise:
qutebrowser.misc.sql.SqlException: Failed to prepare query "SELECT url, title,
strftime('%Y-%m-%d', last_atime, 'unixepoch', 'localtime') FROM
CompletionHistory WHERE (url LIKE :pat escape '\' or title LIKE :pat escape '\')
AND last_atime >= ORDER BY last_atime DESC": "near "ORDER": syntax error Unable
to execute statement"
We need to tell sqlite to convert the timestamps to localtime during
formatting, otherwise it formats them as though you are in UTC.
Also fix up a few uses of mktime.
For performance, re-introduce web-history-max-items.
As the history query has now become a very specific multi-part query and
history completion was the only consumer of SqlCategory, SqlCategory is
now replaced by a HistoryCategory class.
- Fix flake8
- history.clear should also clear completion table
- call _resize_columns in set_model, not set_pattern
- add more unit-testing for the history completion table
- Ignore invalid variable name in flake8 (pylint already checks this and
we don't want to have to double-ignore)
- Fix and test completion bug with `:set asdf `
- Remove unused import
- Use `assert not func.called` instead of `func.assert_not_called` for
backwards compatibility
- Fix comment and empty line check in _parse_entry
- connect layoutAboutToBeChanged signal
- assert sort_order is None if sort_by is None
- modify sql init failure message to ask about Qt sqlite support.
- Fix outdated comments
- Use mock specs when possible
- More precise error message check in test_import_txt_invalid.
- Fix copyright message
- Tweak missing pyqt error message
- Dead code: remove group_by and where from sqlcategory.
With the new separate completion table, these are no longer used.
- Move test_history out of webkit/. History is no longer purely webkit
related, it could be webengine.
When binding a key, the first row will be the current binding if the key
is already bound. This should make it easier for users to tell when they
are binding a key that is already bound, and what it is bound to.
With the new completion API, we no longer need a custom filterAcceptsRow
function. This was necessary to handle the tree structure of the model,
but now we use a separate QSortFilterProxyModel for each category, so
the data it filters is flat. We can simplify the code by using the
builtin setFilterRegExp.
This changes the behavior a little, as now all list categories filter on
all columns. This should be beneficial if anything. For example, help
topics are now filtered on description in addition to name.
This also seems to slightly speed up filtering, according to the url
model benchmark.
Before:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.2806 1.3817 1.3195 0.0390 1.3068 0.0487 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
After:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.1183 1.1508 1.1281 0.0132 1.1241 0.0142 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
Taking the completion widget as an argument was overly complex.
The process now looks like:
1. CompletionView gets deletion request
2. CompletionView passes selected index to CompletionModel
3. CompletionModel passes the row data to the owning category
4. The category runs its custom completion function.
This also fixes a bug. With the switch to the hybrid (list/sql)
completion model, the view was no longer updating when items were
deleted. This fixes that by ensuring the correct signals are emitted.
The SQL model must be refreshed by running the query. We could try using
a SqlTableModel so we can call removeRows instead.
The test for deleting a url fails because qmodeltester claims the length
of the query model is still 3.
For QSqlQueryModel, the argument should always be an invalid index:
http://doc.qt.io/qt-5/qsqlquerymodel.html#canFetchMore
For a QStandardItemModel, it doesn't matter. Either way, passing the
top-level parent index was wrong.
On Travis CI we are sometimes seeing:
```
CompletionView.selection_changed[str].emit():
argument 1 has unexpected type 'int'
```
Cast the data to a string before emitting it just to be safe.
Fix the issue where pressing `o<esc>o` would show a url completion
dialog where attempting to <Tab> select items would do nothing but show
a Qt warning.
The fix is to ensure we set _last_completion_func to None whenever we
clear completion (there was a case I missed).
It also ensures we always delete the old model and adds a safety to
prevent deleting an in-use model is set_model is called with the current
model.
This was changed during code review but was causing Qt errors while
TAB-completing in the selection view:
08:42:34 WARNING qt Unknown module:none:0 Can't select indexes from different model or with different parents
For real this time. A mistake on the last commit like this meant models
were still spuriously instantiated.
Now that the completion model is reused, the layoutChanged signal needs
to be forwarded through, otherwise the view will not update.
This seemed to have a significant performance impact. Removing it means
that instead of just seeing the most recent atime for a given url, you
will see multiple entries.
If the completion model would stay the same, just keep it and update the
filter pattern rather than instantiating a new model each time the
pattern changes.
Instead set this on inidividual categories, as that is where it actually
gets used. This makes it easier for SqlCompletionCategory to reuse a
prepared query (as it gets the filter field names in its constructor).
The history completion query is extended to pick only the most recent item for
a given url.
The tests in test_models now check for ordering of elements.
Instead of add_list and add_sqltable, the completion model now supports
add_category, and callees either pass in a SqlCategory or ListCategory. This
makes unit testing much easier.
This also folds CompletionFilterModel into the ListCategory class.
The RFC on moving from plaintext to SQL storage (#2340) showed that many would
be upset if bookmarks and quickmarks were no longer stored in plaintext.
This commit uses list-based completion for quickmarks and bookmarks. Now the
history storage can be moved from plaintext to an on-disk SQL database while
leaving bookmarks and quickmarks as-is.
Now all completion models are of a single type called CompletionModel.
This model combines one or more categories. A category can either be a
ListCategory or a SqlCategory.
This simplifies the API, and will allow the use of models that combine simple
list-based and sql sources. This is important for two reasons:
- Adding searchengines to url completion
- Using an on-disk sqlite database for history, while keeping bookmarks and
quickmars as text files.
This was a performance optimization that shouldn't be needed with the new SQL
history backend. This also removes support for the LIMIT feature from SqlTable
as it only existed to support web-history-max-items.
Respond to the low-hanging code review fruit:
- Clean up some comments
- Remove an acidentally added duplicate init_autosave
- Combine two test_history tests
- Move test_init cleanup into a fixture to ensure it gets called.
- Name the _ argument of bind(_) to _key
- Ensure index is valid for first_item/last_item
- Move SqlException to top of module
- Rename test_index to test_getitem
- Return QItemFlags.None instead of None
- Fix copyright dates (its 2017 now!)
- Use * to force some args to be keyword-only
- Make some returns explicit
- Add sql to LOGGER_NAMES
- Add a comment to explain the sql escape statement
Change the logging to report the completion function name and have the end2end
tests check for this.
Remove the tests for realtime completion, as it was decided this is not an
important feature and the code is much simpler without it.
This just forwards canFetchMore and fetchMore to the underlying tables.
It seems to be returning True and fetching in some cases (with a large
history), so I guess it is useful?
Allow categories to specify a WHERE clause that applies in addition to the
pattern filter. This allows the url completion model to filter out redirect
entries.
This also fixed the usage of ESCAPE so it applies to all the LIKE statements.
A SQL completion category can now provide a customized column expression for
the select statement. This enables the url model to format timestamps, as well
as rearrange the name and url in the quickmark section.
This allows setting the query as a QSqlQuery instead of a string, which allows:
- Escaping quotes
- Using LIMIT (needed for history-max-items)
- Using ORDER BY (needed for sorting history)
- SELECTing columns (needed for quickmark completion)
- Creating a custom select (needed for history timestamp formatting)
For URL completion, time-based sorting is handled by the SQL model.
All the other models use simple alphabetical sorting. This allowed cleaning up
some logic in the sortfilter, removing DUMB_SORT, and removing the
completion.Role.sort.
This also removes the userdata completion field as it was only used in url
completion and is no longer necessary with the SQL model.
The new completion API no longer needs either of these. Instead of
referencing an enum member, cmdutils.argument.completion now points to
a function that returnsthe desired completion model.
This vastly simplifies the addition of new completion types. Previously
it was necessary to define the new model as well as editing usertypes
and completion.models.instances. Now it is only necessary to define a
single function under completion.models.
This is the next step of Completion Model/View Revamping (#74).
First step of Completion Model/View revamping (#74). Rewrite the
completion models as functions that each return an instance of a
CompletionModel class.
Caching is removed from all models except the UrlModel. Models other
than the UrlModel can be generated very quickly so caching just adds
needless complexity and can lead to incorrect results if one forgets to
wire up a signal.
This makes those UI elements look the same on different platforms/OS styles,
with the small drawback of overriding the context menu style.
This most likely fixes#80 (though I couldn't reproduce that on Windows 10).
I discovered this because of #2287 but it doesn't actually change anything
there. When we don't have a third column, subtract the scrollbar width from the
second one.
We already had some duplicated logic for completion/keyhint/messageview,
and plan to add prompt overlays too now - so here we refactor related
code to have a list of overlays instead, which are all
resized/positioned by the mainwindow when needed.
This also changes the size management, which gets moved into the
sizeHint of the respective overlay widgets.
Simplify the CompletionWidget/Completer interface by changing
on_selection_changed to pass the newly selected text rather than the
index of the newly selected item.
This moves the logic from Completer to CompletionWidget but simplifies
the interaction between the two and makes testing easier.
When the commandline reads ':open |', quick-completing the only offered
completion will set the commandline to ':open some_url |'. Since `open`
has `maxsplit=0`, everything after ':open' is (correctly) treated as
one argument. This means completion is opened again with 'some url '
as the pattern (note trailing whitespace), which makes the comletion
menu 'flicker' and stay open even though it was 'supposed' to quick
compelte.
This is fixed by ignoring the next completion request if we just
completed something after maxsplit (because we don't expect any more
completions after the last split).
Resolves#1519.
After the refactoring, _split is only called by _partition so just make
it part of the same method. This also removes the use of
_empty_item_index, as it can be figured out on the fly.
Remove the class variables _cursor_part and _empty_item_index. Instead,
split up the commandline around the cursor whenever that information is
needed. Using locals instead of class variables makes the logic easier
to follow and ends up requiring much less code.
Remove the dependency on the class variables _empty_item_index
and _cursor_part to make the code easier to follow. If
_update_completion is refactored in a similar way these variables can
be removed.
Turns out re.escape also escapes spaces, so we'd need to replace '(\\ )'
groups after escaping. At this point it's easier to just combine spaces
before escaping the pattern.
Fixes#1934.
Supersedes #1935.
Implement `completion-item-focus next-category` and
`completion-item-focus prev-category` to jump through completions by
category rather than by item.
Resolves#1567.
