The parsing bind() did manually is now available through CommandParser.
Resolves#2952.
This also adds a unit test for the case when there is no current
binding, as I broke that while working on this and there was no test to
catch it :)
We use fake_runtime_dir which simply patches XDG_RUNTIME_DIR for this test.
Since we patch QApplication.applicationName() during the tests, but standarddir
doesn't use that anymore, we get a different name.
Versions before v0.9.0 (which didn't even support hinting with QtWebEngine!)
used to write QtWebEngine data to:
~/.local/share/qutebrowser/qutebrowser/QtWebEngine/Default
~/.cache/qutebrowser/qutebrowser/QtWebEngine/Default
In v0.9.0 this was changed to:
~/.local/share/qutebrowser/webengine
~/.cache/qutebrowser/webengine
Now we don't try to migrate data from the old location anymore.
We were only rendering .html files before, so the old _guess_autoescape function
had the effect of always autoescaping .render() (from a file) but never
autoescaping .from_string(). However, most places using .from_string() actually
render (Qt-)HTML via jinja, so they should escape stuff!
Now, we always autoescape, except when the caller uses the
jinja.environment.no_autoescape() context manager, which places rendering
stylesheets now do.
This impacted:
- Confirm quit texts (no HTML here)
- config.py loading errors
(where this was found because of an error containing - a <keybinding>)
- Certificate error prompts
(should be fine from what I can tell, as the only user-controllable output is
the hostname, which cannot contain HTML)
"Downloading with SSL errors" from downloads.feature still fails, but like this,
at least all other tests pass without the need to modify LD_LIBRARY_PATH.
We accidentally did show the command as a list in to_str(). However, after
correcting that to use shlex.escape, we got ugly qutebrowser command lines
when tabbing to the default value, because of how shlex handles double-escaping:
>>> print(shlex.quote("gvim -f '{}'"))
'gvim -f '"'"'{}'"'"''
While in this case, outputting "gvim -f '{}'" would be much more appropriate, it
doesn't look like we can teach shlex.quote to do that.
Instead, we now only accept a list as input for ShellCommand, at the price that
the user needs to do
:set editor.command '["gvim", "-f", "{}"]'
instead of
:set editor.command 'gvim -f {}'
Fixes#2962.
While the test worked again with eb4691adfc, it
broke again immediately because of 40ee89bddc.
With that fix in, the lower-case monospace in the set value was immediately
replaced by the full list of fonts again. With an upper-case Monospace, this
won't happen.
Fixes#2825, for real this time.
In python3.4, there is a circular dependency between the config module
and configmodel.bind. This is resolved by dependency injection. The
config/keyconfig instances are embedded in a struct passed to every
completion function, so the functions no longer depend on the modules.
This will also enable completion functions to access other previously
inaccessible info, such as the window id.
See #2814.
When upgrading from an old table that used different url formatting, two
entries might map to the same key, so we'll need to replace the previous
entry to avoid a primary key conflict.
Incrementing _USER_VERSION in the source will cause the
HistoryCompletion table to regenerate when users update.
This is currently necessary to support some recent formatting fixes, but
could be incremented again in the future for other changes.
When in miscmodels, the config module was unable to find the function.
It appears to be some sort of circular import issue:
```
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/app.py", line 44, in <module>
from qutebrowser.completion.models import miscmodels
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/miscmodels.py", line 24, in <module>
from qutebrowser.completion.models import completionmodel, listcategory, util
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/util.py", line 24, in <module>
from qutebrowser.config import config
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 223, in <module>
class ConfigCommands:
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 314, in ConfigCommands
@cmdutils.argument('command', completion=miscmodels.bind)
AttributeError: module 'qutebrowser.completion.models.miscmodels' has no attribute 'bind'
```
As configmodel imports util (and thereby config as well) it is unclear
to me why moving bind() to configmodel actually fixes this, but it does.
If the HistoryCompletion table is removed, regenerate it from the
History table. This allows users to manually edit History, then remove
HistoryCompletion to prompt regeneration.
See #2903.
Encode urls that are inserted into the history, but do not encode urls
for completion (other than removing passwords).
Also ensure that urls read from the history text file are formatted
consistenly with those added while browsing.
Fixes#2903.
Get qutebrowser to the point where it can at least start
- Declare _messages earlier in MessageView.__init__ so it is set before
the config trigger tries to access it.
- Remove unused configmodel completion functions
- Move bind completion to configmodel to avoid a circular import with
the config module
- Fix some config accesses (forgot to use .val)
- Fix old Completion.CompletionKind references
Instead, expect the data to be given in the desired order. Completion
functions should sort their data _if_ they want it sorted in the
completion. This has a few implications:
- {book,quick}marks appear in the same order they do in the text file.
This means users can rearrange their mark files for custom sorting.
Fixes#2354
- Sessions are sorted as they appear in the session manager
- Tabs are sorted numerically, not alphabetically (Fixes#2883)
Note that prefix-based filter sorting is still performed, so items
starting with the filter pattern come first.
The added/removed signals for the urlmark managers are no longer used as
the completion models are generated on-the-fly. The changed signal is
still needed so the save-manager knows when to trigger a write to disk.
Also removes session_manager.update_completion, which is no longer
needed for the same reason as above.
keyconf.changed cannot be removed, as it is still wired up to
basekeyparser.
Resolves#2874.
Fixes#2868, where pressing <shift-tab> then <ctrl-d> in history
completion (with > 256 items) would cause later items to disappear (and
cause a crash if you try to delete again).
Cause:
Scrolling to the bottom would fetch an additional 256 items (in addition
to the 256 that are fetched at first). Deleting causes the query to
re-run, but it only fetches the initial 256 items, so the current index
is now invalid.
Fix:
After deleting from the history category, call fetchMore until it has
enough rows populated that the current index is valid.
When tabbing to the last index of history completion, call expandAll
which will call fetchMore to retrieve more query results, if available.
Calling fetchMore directly will not update the view, and for some
reason self.expand(idx.parent()) and
self.expand(self.model().index(idx.row(), 0)) did not work, so I'm using
expandAll.
Fixes#2841.
When a key is bound to a command line that includes one or more
arguments to a command, bind completion should show the whole command
for the "Current" category, and use only the command name to look up the
description.
Fixes#2859, where a crash was caused by looking up the description by
the full command text rather than just the name.
Override removeRows instead of removeRow.
> removeRow is not virtual in C++, so if this gets called by Qt
> internally for some reason, it wouldn't use the overloaded version -
> so I think it'd be better to implement removeRows and then use
> removeRow without overloading that
- The-Compiler
It doesn't make sense to have an active selection while you are
filtering by entering text. You should be in one of two states:
1. Tabbing through completions (valid selection)
2. Entering a filter pattern (invalid selection)
Fixes#2843, where a crash would occur after the following:
1. tab to an item other than the first
2. <backspace>
3. re-type last character
4. <ctrl-d>
This would try to delete an out of range index.
Even though no item was deleted, it was manipulating the completion
model because beginRemoveRows was called before the exception was
raised.
This fixes that problem by moving the removal logic (and delete_func
check) into the parent model, so it can check whether deletion is
possible before calling beginRemoveRows.
Fixes#2839.
We get no last_atime limit at all otherwise:
qutebrowser.misc.sql.SqlException: Failed to prepare query "SELECT url, title,
strftime('%Y-%m-%d', last_atime, 'unixepoch', 'localtime') FROM
CompletionHistory WHERE (url LIKE :pat escape '\' or title LIKE :pat escape '\')
AND last_atime >= ORDER BY last_atime DESC": "near "ORDER": syntax error Unable
to execute statement"
We need to tell sqlite to convert the timestamps to localtime during
formatting, otherwise it formats them as though you are in UTC.
Also fix up a few uses of mktime.
For performance, re-introduce web-history-max-items.
As the history query has now become a very specific multi-part query and
history completion was the only consumer of SqlCategory, SqlCategory is
now replaced by a HistoryCategory class.
The javascript history page was requesting the new start_time in ms, but
the python code was expecting seconds. This is fixed by removing all the
millisecond translations in the python code and only translating to
milliseconds in the javascript code that formats dates.
- Fix flake8
- history.clear should also clear completion table
- call _resize_columns in set_model, not set_pattern
- add more unit-testing for the history completion table
- Ignore invalid variable name in flake8 (pylint already checks this and
we don't want to have to double-ignore)
- Fix and test completion bug with `:set asdf `
- Remove unused import
- Use `assert not func.called` instead of `func.assert_not_called` for
backwards compatibility
- Fix comment and empty line check in _parse_entry
- connect layoutAboutToBeChanged signal
- assert sort_order is None if sort_by is None
- modify sql init failure message to ask about Qt sqlite support.
- use mocker.Mock instead of mock.Mock to avoid an extra import
- attach model to validator sooner so it can validate changes in the
model during the test
- remove outdated comment
- fix sql init error message
- clean up history text import code
- fix test_history file path in coverage check
- use real web history, not stub, for completion model tests
- use qtmodeltester in sql/list_category tests
- test url encoding in history tests
- fix test_clear by using a callable mock
- remove test_debug_dump_history_oserror as the check is now the same as
for the file not existing
- rename nonempty to data in test_completionmodel
- add more delete_cur_item tests
- test empty option/value completion
This is also needed to make the docs environment work on Travis - as otherwise,
doc generation wasn't deterministic because of changing dict key order.
In Dict.to_str() and List.to_str() we use json.dump to get a value. However,
JSON includes surrogate escapes in the dumped values, which breaks round trips.
>>> yaml.load(json.dumps({'\U00010000': True}))
{'\ud800\udc00': True}
>>> yaml.load(json.dumps({'\U00010000': True}, ensure_ascii=False))
yaml.reader.ReaderError: unacceptable character #x10000: special characters are not allowed
See:
https://stackoverflow.com/a/38552626/2085149https://news.ycombinator.com/item?id=12798032
This required some changes on how URLs are handled during those tests. Before,
we simply could return a path and (since we had a patched QNAM), nobody
complained.
Now this actually needs to be a valid URL, so we use
https://www.example.com/path everywhere instead.
Now the "object" kind of value (like in YAML) is stored internally, and that's
the canonical value. The methods changed their meaning slightly, see the
docstring in configtypes.py for details.
- Fix outdated comments
- Use mock specs when possible
- More precise error message check in test_import_txt_invalid.
- Fix copyright message
- Tweak missing pyqt error message
- Dead code: remove group_by and where from sqlcategory.
With the new separate completion table, these are no longer used.
- Move test_history out of webkit/. History is no longer purely webkit
related, it could be webengine.
This ensures we actually know when an AttributeError happens.
It also changes most external code to use the correct environment, rather than
simply creating a jinja2.Template, which wouldn't use the more tightened
environment.
When binding a key, the first row will be the current binding if the key
is already bound. This should make it easier for users to tell when they
are binding a key that is already bound, and what it is bound to.
There were two issues here:
- The comparison was backwards, causing scroller.at_bottom() to always return
true.
- When zoomed in, jsret['px']['y'] can be a float, which means we can be
slightly off when checking the difference - math.ceil() fixes that.
Taking the completion widget as an argument was overly complex.
The process now looks like:
1. CompletionView gets deletion request
2. CompletionView passes selected index to CompletionModel
3. CompletionModel passes the row data to the owning category
4. The category runs its custom completion function.
This also fixes a bug. With the switch to the hybrid (list/sql)
completion model, the view was no longer updating when items were
deleted. This fixes that by ensuring the correct signals are emitted.
The SQL model must be refreshed by running the query. We could try using
a SqlTableModel so we can call removeRows instead.
The test for deleting a url fails because qmodeltester claims the length
of the query model is still 3.
This allows replace to be a named parameter and allows consolidating
some duplicate code between various insert methods.
This also fixes some tests that broke because batch insert was broken.
No longer needed with sql backend. Query results build their own
namedtuple from the returned columns, and inserting new entries is just
done with named parameters.
This is called often, hopefully a prepared query will speed it up.
This also modifies Query.run to return self for easier chaining, so you
can use `query.run.value()` instead of `query.run` ; query.value()`.
For real this time. A mistake on the last commit like this meant models
were still spuriously instantiated.
Now that the completion model is reused, the layoutChanged signal needs
to be forwarded through, otherwise the view will not update.
This seemed to have a significant performance impact. Removing it means
that instead of just seeing the most recent atime for a given url, you
will see multiple entries.
If the completion model would stay the same, just keep it and update the
filter pattern rather than instantiating a new model each time the
pattern changes.
Instead set this on inidividual categories, as that is where it actually
gets used. This makes it easier for SqlCompletionCategory to reuse a
prepared query (as it gets the filter field names in its constructor).
Trying to read from the sql database from another process was flaky.
This adds a debug-dump-history command which is used by the history BDD
tests to validate the history contents.
It outputs history in the old pre-SQL text format, so it might be
useful for those who want to manipulate their history as text.
Returning "next" was no longer possible as the SQL query does not fetch
more items than necessary. This is solved by using a start time, a
limit, and an offset. The offset is needed to prevent fetching duplicate
items if multiple entries have the same timestamp.
Two of the history tests that relied on qute://history were changed to
rely on qute://history/data instead to make them less failure-prone.
The history completion query is extended to pick only the most recent item for
a given url.
The tests in test_models now check for ordering of elements.
The old implementation was looping through the whole history list, which for
SQL was selecting every row in the database. The history benchmark was taking
~2s. If this is rewritten as a specialized SQL query, the benchmark takes
~10ms, an order of magnitude faster than the original non-SQL implementation.
Vulture exposed the following dead code:
- AppendLineParse was only used for reading the history text file, which is now
a sql database (and the import code for the old text file is simpler and does
not need a complex line parser)
- async_read_done is no longer used as importing the history text file is
synchronous (and should only happen once)
- config._init_key_config is unused as it was moved to keyconf.init
Calling sql.init() in version.version() would replace the existing sql
connection and cause a crash when accessed by opening qute://version.
Now version relies on sql already being initted, and app.py inits sql early if
the --version arg is given.
Two history end2end tests are failing because sqlite is not flushing to disk in
time to be read by the test process. My understanding is that sqlite should
take an exclusive lock while writing, so it is difficult to understand why this
is happening. This can be fixed by adding a delay, but that seems flaky.
I'm fixing it by checking qute://history instead of reading the database file.
See:
https://github.com/qutebrowser/qutebrowser/pull/2295#issuecomment-292786138
and the following discussion.
Turns out historyContains was getting called for the webkit backend multiple
times when the browser starts. This was calling `url in history`, which was
enumerating the entire history as `__contains__` was not defined.
Instead of skipping bad history lines during the import to sql, fail hard. We
don't want to delete the user's old history file if we couldn't parse all of
the lines.
