Should default to -1, not 1000 as the new history completion is better
able to handle large numbers of entries. I believe this was acidentally
reset to 1000 while fixing a merge conflict.
Also re-run src2asciidoc.
In python3.4, there is a circular dependency between the config module
and configmodel.bind. This is resolved by dependency injection. The
config/keyconfig instances are embedded in a struct passed to every
completion function, so the functions no longer depend on the modules.
This will also enable completion functions to access other previously
inaccessible info, such as the window id.
See #2814.
When upgrading from an old table that used different url formatting, two
entries might map to the same key, so we'll need to replace the previous
entry to avoid a primary key conflict.
Incrementing _USER_VERSION in the source will cause the
HistoryCompletion table to regenerate when users update.
This is currently necessary to support some recent formatting fixes, but
could be incremented again in the future for other changes.
When in miscmodels, the config module was unable to find the function.
It appears to be some sort of circular import issue:
```
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/app.py", line 44, in <module>
from qutebrowser.completion.models import miscmodels
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/miscmodels.py", line 24, in <module>
from qutebrowser.completion.models import completionmodel, listcategory, util
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/completion/models/util.py", line 24, in <module>
from qutebrowser.config import config
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 223, in <module>
class ConfigCommands:
File "/home/rcorre/projects/contrib/qutebrowser/qutebrowser/config/config.py", line 314, in ConfigCommands
@cmdutils.argument('command', completion=miscmodels.bind)
AttributeError: module 'qutebrowser.completion.models.miscmodels' has no attribute 'bind'
```
As configmodel imports util (and thereby config as well) it is unclear
to me why moving bind() to configmodel actually fixes this, but it does.
If the HistoryCompletion table is removed, regenerate it from the
History table. This allows users to manually edit History, then remove
HistoryCompletion to prompt regeneration.
See #2903.
Encode urls that are inserted into the history, but do not encode urls
for completion (other than removing passwords).
Also ensure that urls read from the history text file are formatted
consistenly with those added while browsing.
Fixes#2903.
Get qutebrowser to the point where it can at least start
- Declare _messages earlier in MessageView.__init__ so it is set before
the config trigger tries to access it.
- Remove unused configmodel completion functions
- Move bind completion to configmodel to avoid a circular import with
the config module
- Fix some config accesses (forgot to use .val)
- Fix old Completion.CompletionKind references
Instead, expect the data to be given in the desired order. Completion
functions should sort their data _if_ they want it sorted in the
completion. This has a few implications:
- {book,quick}marks appear in the same order they do in the text file.
This means users can rearrange their mark files for custom sorting.
Fixes#2354
- Sessions are sorted as they appear in the session manager
- Tabs are sorted numerically, not alphabetically (Fixes#2883)
Note that prefix-based filter sorting is still performed, so items
starting with the filter pattern come first.
The added/removed signals for the urlmark managers are no longer used as
the completion models are generated on-the-fly. The changed signal is
still needed so the save-manager knows when to trigger a write to disk.
Also removes session_manager.update_completion, which is no longer
needed for the same reason as above.
keyconf.changed cannot be removed, as it is still wired up to
basekeyparser.
Resolves#2874.
Fixes#2868, where pressing <shift-tab> then <ctrl-d> in history
completion (with > 256 items) would cause later items to disappear (and
cause a crash if you try to delete again).
Cause:
Scrolling to the bottom would fetch an additional 256 items (in addition
to the 256 that are fetched at first). Deleting causes the query to
re-run, but it only fetches the initial 256 items, so the current index
is now invalid.
Fix:
After deleting from the history category, call fetchMore until it has
enough rows populated that the current index is valid.
When tabbing to the last index of history completion, call expandAll
which will call fetchMore to retrieve more query results, if available.
Calling fetchMore directly will not update the view, and for some
reason self.expand(idx.parent()) and
self.expand(self.model().index(idx.row(), 0)) did not work, so I'm using
expandAll.
Fixes#2841.
When a key is bound to a command line that includes one or more
arguments to a command, bind completion should show the whole command
for the "Current" category, and use only the command name to look up the
description.
Fixes#2859, where a crash was caused by looking up the description by
the full command text rather than just the name.
Sometimes, we get another error with "Renderer process was killed" and the data:
URL for the error page. This is probably because the renderer process wasn't
restarted yet. This hopefully helps.
Override removeRows instead of removeRow.
> removeRow is not virtual in C++, so if this gets called by Qt
> internally for some reason, it wouldn't use the overloaded version -
> so I think it'd be better to implement removeRows and then use
> removeRow without overloading that
- The-Compiler
It doesn't make sense to have an active selection while you are
filtering by entering text. You should be in one of two states:
1. Tabbing through completions (valid selection)
2. Entering a filter pattern (invalid selection)
Fixes#2843, where a crash would occur after the following:
1. tab to an item other than the first
2. <backspace>
3. re-type last character
4. <ctrl-d>
This would try to delete an out of range index.
Even though no item was deleted, it was manipulating the completion
model because beginRemoveRows was called before the exception was
raised.
This fixes that problem by moving the removal logic (and delete_func
check) into the parent model, so it can check whether deletion is
possible before calling beginRemoveRows.
Fixes#2839.
We get no last_atime limit at all otherwise:
qutebrowser.misc.sql.SqlException: Failed to prepare query "SELECT url, title,
strftime('%Y-%m-%d', last_atime, 'unixepoch', 'localtime') FROM
CompletionHistory WHERE (url LIKE :pat escape '\' or title LIKE :pat escape '\')
AND last_atime >= ORDER BY last_atime DESC": "near "ORDER": syntax error Unable
to execute statement"
We need to tell sqlite to convert the timestamps to localtime during
formatting, otherwise it formats them as though you are in UTC.
Also fix up a few uses of mktime.
For some reason, calling search.clear() while no search is displayed causes the
backends to un-focus inputs, and with QtWebKit, even hinting can't focus them
again after that.
For performance, re-introduce web-history-max-items.
As the history query has now become a very specific multi-part query and
history completion was the only consumer of SqlCategory, SqlCategory is
now replaced by a HistoryCategory class.
The javascript history page was requesting the new start_time in ms, but
the python code was expecting seconds. This is fixed by removing all the
millisecond translations in the python code and only translating to
milliseconds in the javascript code that formats dates.
- Fix flake8
- history.clear should also clear completion table
- call _resize_columns in set_model, not set_pattern
- add more unit-testing for the history completion table
- Ignore invalid variable name in flake8 (pylint already checks this and
we don't want to have to double-ignore)
- Fix and test completion bug with `:set asdf `
- Remove unused import
- Use `assert not func.called` instead of `func.assert_not_called` for
backwards compatibility
- Fix comment and empty line check in _parse_entry
- connect layoutAboutToBeChanged signal
- assert sort_order is None if sort_by is None
- modify sql init failure message to ask about Qt sqlite support.
Check for interval being positive instead of checking for it to be
non-zero. So if somehow some unexpected thing happend and made
message-timeout negative, the bug doesn't cascade.
- remove outdated comment
- fix sql init error message
- clean up history text import code
- fix test_history file path in coverage check
- use real web history, not stub, for completion model tests
- use qtmodeltester in sql/list_category tests
- test url encoding in history tests
- fix test_clear by using a callable mock
- remove test_debug_dump_history_oserror as the check is now the same as
for the file not existing
- rename nonempty to data in test_completionmodel
- add more delete_cur_item tests
- test empty option/value completion
If creating the sql database fails, show an error dialog assuming sqlite
is not installed.
This removes the isDriverAvailable check as it was true even with sqlite
uninstalled.
sql.version now inits itself if sql is not already initialized and
prints 'UNAVAILABLE (<error message>)' if init fails. This is to avoid
cascading errors, where one error would create a crash dialog, which
calls sql.version, which would create another error.
This is also needed to make the docs environment work on Travis - as otherwise,
doc generation wasn't deterministic because of changing dict key order.
In Dict.to_str() and List.to_str() we use json.dump to get a value. However,
JSON includes surrogate escapes in the dumped values, which breaks round trips.
>>> yaml.load(json.dumps({'\U00010000': True}))
{'\ud800\udc00': True}
>>> yaml.load(json.dumps({'\U00010000': True}, ensure_ascii=False))
yaml.reader.ReaderError: unacceptable character #x10000: special characters are not allowed
See:
https://stackoverflow.com/a/38552626/2085149https://news.ycombinator.com/item?id=12798032
Now that the StyleSheetObserver is a child of the object it observes, it should
get cleaned up properly when the object is deleted.
This means this is hopefully not needed anymore, even on Qt 5.2.
This gets called a lot, and caused some :bind calls to take ~3s.
Stats after starting with a bit of :bind:
CacheInfo(hits=25917, misses=139, maxsize=256, currsize=139)
Now the "object" kind of value (like in YAML) is stored internally, and that's
the canonical value. The methods changed their meaning slightly, see the
docstring in configtypes.py for details.
- Fix outdated comments
- Use mock specs when possible
- More precise error message check in test_import_txt_invalid.
- Fix copyright message
- Tweak missing pyqt error message
- Dead code: remove group_by and where from sqlcategory.
With the new separate completion table, these are no longer used.
- Move test_history out of webkit/. History is no longer purely webkit
related, it could be webengine.
Show a graphical error box with install instructions if PyQt.QtSql is
not found, rather than failing with CLI errors. Also show an error box
if the sqlite driver is not available.
This ensures we actually know when an AttributeError happens.
It also changes most external code to use the correct environment, rather than
simply creating a jinja2.Template, which wouldn't use the more tightened
environment.
When binding a key, the first row will be the current binding if the key
is already bound. This should make it easier for users to tell when they
are binding a key that is already bound, and what it is bound to.
There were two issues here:
- The comparison was backwards, causing scroller.at_bottom() to always return
true.
- When zoomed in, jsret['px']['y'] can be a float, which means we can be
slightly off when checking the difference - math.ceil() fixes that.
With the new completion API, we no longer need a custom filterAcceptsRow
function. This was necessary to handle the tree structure of the model,
but now we use a separate QSortFilterProxyModel for each category, so
the data it filters is flat. We can simplify the code by using the
builtin setFilterRegExp.
This changes the behavior a little, as now all list categories filter on
all columns. This should be beneficial if anything. For example, help
topics are now filtered on description in addition to name.
This also seems to slightly speed up filtering, according to the url
model benchmark.
Before:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.2806 1.3817 1.3195 0.0390 1.3068 0.0487 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
After:
----------------------------------------------- benchmark: 1 tests ----------------------------------------------
Name (time in s) Min Max Mean StdDev Median IQR Outliers(*) Rounds Iterations
-----------------------------------------------------------------------------------------------------------------
test_url_completion_benchmark 1.1183 1.1508 1.1281 0.0132 1.1241 0.0142 1;0 5 1
-----------------------------------------------------------------------------------------------------------------
Taking the completion widget as an argument was overly complex.
The process now looks like:
1. CompletionView gets deletion request
2. CompletionView passes selected index to CompletionModel
3. CompletionModel passes the row data to the owning category
4. The category runs its custom completion function.
This also fixes a bug. With the switch to the hybrid (list/sql)
completion model, the view was no longer updating when items were
deleted. This fixes that by ensuring the correct signals are emitted.
The SQL model must be refreshed by running the query. We could try using
a SqlTableModel so we can call removeRows instead.
The test for deleting a url fails because qmodeltester claims the length
of the query model is still 3.
For QSqlQueryModel, the argument should always be an invalid index:
http://doc.qt.io/qt-5/qsqlquerymodel.html#canFetchMore
For a QStandardItemModel, it doesn't matter. Either way, passing the
top-level parent index was wrong.
_insert_query gets called with a query and dict of values such as:
{'val': 1, 'lucky': False, 'name': 'one'}
Via bindValues(), we only assign a placeholder in the query string to a value,
so we get a query with bindings like:
INSERT INTO Foo values(:lucky,:val,:name)
{':name': 'one', ':val': 1, ':lucky': False}
So what we're executing is something like:
INSERT INTO Foo values(false,1,"one")
However, if the column order in the database doesn't happen to be the order
we're passing the values in, we get the wrong values in the wrong columns.
Instead, we now do:
INSERT INTO Foo (lucky, val, name) values(false,1,"one")
Which inserts the values in the order we intended.
With Python 3.6, this just happened to work before because we always passed the
keyword arguments in the table column order, and in 3.6 dicts
(and thus **kwargs) happen to be ordered:
https://mail.python.org/pipermail/python-dev/2016-September/146327.html
On Travis CI we are sometimes seeing:
```
CompletionView.selection_changed[str].emit():
argument 1 has unexpected type 'int'
```
Cast the data to a string before emitting it just to be safe.
This allows replace to be a named parameter and allows consolidating
some duplicate code between various insert methods.
This also fixes some tests that broke because batch insert was broken.
No longer needed with sql backend. Query results build their own
namedtuple from the returned columns, and inserting new entries is just
done with named parameters.
Fix the issue where pressing `o<esc>o` would show a url completion
dialog where attempting to <Tab> select items would do nothing but show
a Qt warning.
The fix is to ensure we set _last_completion_func to None whenever we
clear completion (there was a case I missed).
It also ensures we always delete the old model and adds a safety to
prevent deleting an in-use model is set_model is called with the current
model.
This was changed during code review but was causing Qt errors while
TAB-completing in the selection view:
08:42:34 WARNING qt Unknown module:none:0 Can't select indexes from different model or with different parents
before
------
sqlite> SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
Run Time: real 0.072 user 0.063334 sys 0.010000
sqlite> explain query plan SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
0|0|0|SCAN TABLE History
0|0|0|USE TEMP B-TREE FOR ORDER BY
sqlite> explain query plan select url, title, strftime('%Y-%m-%d', last_atime, 'unixepoch') from CompletionHistory where (url like "%qute%" or title like "%qute%") order by last_atime desc;
0|0|0|SCAN TABLE CompletionHistory
0|0|0|USE TEMP B-TREE FOR ORDER BY
after
-----
sqlite> SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
Run Time: real 0.000 user 0.000000 sys 0.000000
sqlite> explain query plan SELECT * FROM History where not redirect and not url like "qute://%" and atime > ? and atime <= ? ORDER BY atime desc;
0|0|0|SEARCH TABLE History USING INDEX AtimeIndex (atime>? AND atime<?)
sqlite> explain query plan select url, title, strftime('%Y-%m-%d', last_atime, 'unixepoch') from CompletionHistory where (url like "%qute%" or title like "%qute%") order by last_atime desc;
0|0|0|SCAN TABLE CompletionHistory USING INDEX CompletionAtimeIndex
When loading heise.de, for some crazy reason QtWebKit calls historyContains
about 16'000 times.
With this cache (which we simply clear when *any* page has been loaded, as then
the links which have been visited can change), that's down to 250 or so...
- Show an error message when import fails, not a generic crash dialog
- Raise CommandError when debug-dump-history fails
- Check that the path exists for debug-dump-history
This is called often, hopefully a prepared query will speed it up.
This also modifies Query.run to return self for easier chaining, so you
can use `query.run.value()` instead of `query.run` ; query.value()`.
For real this time. A mistake on the last commit like this meant models
were still spuriously instantiated.
Now that the completion model is reused, the layoutChanged signal needs
to be forwarded through, otherwise the view will not update.
This seemed to have a significant performance impact. Removing it means
that instead of just seeing the most recent atime for a given url, you
will see multiple entries.
If the completion model would stay the same, just keep it and update the
filter pattern rather than instantiating a new model each time the
pattern changes.
Instead set this on inidividual categories, as that is where it actually
gets used. This makes it easier for SqlCompletionCategory to reuse a
prepared query (as it gets the filter field names in its constructor).
Trying to read from the sql database from another process was flaky.
This adds a debug-dump-history command which is used by the history BDD
tests to validate the history contents.
It outputs history in the old pre-SQL text format, so it might be
useful for those who want to manipulate their history as text.
Returning "next" was no longer possible as the SQL query does not fetch
more items than necessary. This is solved by using a start time, a
limit, and an offset. The offset is needed to prevent fetching duplicate
items if multiple entries have the same timestamp.
Two of the history tests that relied on qute://history were changed to
rely on qute://history/data instead to make them less failure-prone.
The history completion query is extended to pick only the most recent item for
a given url.
The tests in test_models now check for ordering of elements.
The old implementation was looping through the whole history list, which for
SQL was selecting every row in the database. The history benchmark was taking
~2s. If this is rewritten as a specialized SQL query, the benchmark takes
~10ms, an order of magnitude faster than the original non-SQL implementation.
Vulture exposed the following dead code:
- AppendLineParse was only used for reading the history text file, which is now
a sql database (and the import code for the old text file is simpler and does
not need a complex line parser)
- async_read_done is no longer used as importing the history text file is
synchronous (and should only happen once)
- config._init_key_config is unused as it was moved to keyconf.init
Calling sql.init() in version.version() would replace the existing sql
connection and cause a crash when accessed by opening qute://version.
Now version relies on sql already being initted, and app.py inits sql early if
the --version arg is given.
Turns out historyContains was getting called for the webkit backend multiple
times when the browser starts. This was calling `url in history`, which was
enumerating the entire history as `__contains__` was not defined.
Instead of skipping bad history lines during the import to sql, fail hard. We
don't want to delete the user's old history file if we couldn't parse all of
the lines.
Now that sql is only used for history (not quickmarks/bookmarks) a number of
functions are no longer needed. In addition, primary key support was removed as
we actually need to support multiple entries for the same url with different
access times. The completion model will have to handle this by selecting
something like (url, title, max(atime)).
This also fixes up a number of tests that were broken with the last few
sql-related commits.
If qutebrowser detects a history text file when it starts
(~/.local/share/qutebrowser/history by default on Linux), it will import this
file into the new sqlite database, then delete it.
The read is done as a coroutine as it can take some time.
Instead of reading sqlite history from a file and storing it in an in-memory
database, just directly use an on-disk database. This resolves#755, where
history entries don't pop in to the completion menu immediately as they are
still being read asynchronously for a few seconds after the browser starts.
Instead of add_list and add_sqltable, the completion model now supports
add_category, and callees either pass in a SqlCategory or ListCategory. This
makes unit testing much easier.
This also folds CompletionFilterModel into the ListCategory class.
The RFC on moving from plaintext to SQL storage (#2340) showed that many would
be upset if bookmarks and quickmarks were no longer stored in plaintext.
This commit uses list-based completion for quickmarks and bookmarks. Now the
history storage can be moved from plaintext to an on-disk SQL database while
leaving bookmarks and quickmarks as-is.
Now all completion models are of a single type called CompletionModel.
This model combines one or more categories. A category can either be a
ListCategory or a SqlCategory.
This simplifies the API, and will allow the use of models that combine simple
list-based and sql sources. This is important for two reasons:
- Adding searchengines to url completion
- Using an on-disk sqlite database for history, while keeping bookmarks and
quickmars as text files.
This was a performance optimization that shouldn't be needed with the new SQL
history backend. This also removes support for the LIMIT feature from SqlTable
as it only existed to support web-history-max-items.
Respond to the low-hanging code review fruit:
- Clean up some comments
- Remove an acidentally added duplicate init_autosave
- Combine two test_history tests
- Move test_init cleanup into a fixture to ensure it gets called.
- Name the _ argument of bind(_) to _key
- Ensure index is valid for first_item/last_item
- Move SqlException to top of module
- Rename test_index to test_getitem
- Return QItemFlags.None instead of None
- Fix copyright dates (its 2017 now!)
- Use * to force some args to be keyword-only
- Make some returns explicit
- Add sql to LOGGER_NAMES
- Add a comment to explain the sql escape statement
newest_slice is no longer needed after the completion refactor. Now that
history is based on the SQL backend, LIMIT is used instead.
StatusBar._option is not used, though I'm not sure why vulture only caught it
now.
The new function-based completion API introduced a circular import:
config -> keyconf -> miscmodels -> config.
config only depended on keyconf so it could initialize it as part of
config.init. This can be resolved by moving this to keyconf.init and
initializing keyconf as part of app.init.
Change the logging to report the completion function name and have the end2end
tests check for this.
Remove the tests for realtime completion, as it was decided this is not an
important feature and the code is much simpler without it.
This just forwards canFetchMore and fetchMore to the underlying tables.
It seems to be returning True and fetching in some cases (with a large
history), so I guess it is useful?
Instead of returning a regular tuple and trying to remember which index maps to
which field, return named tuples that allow accessing the fields by name.
Allow categories to specify a WHERE clause that applies in addition to the
pattern filter. This allows the url completion model to filter out redirect
entries.
This also fixed the usage of ESCAPE so it applies to all the LIKE statements.
A SQL completion category can now provide a customized column expression for
the select statement. This enables the url model to format timestamps, as well
as rearrange the name and url in the quickmark section.
This allows setting the query as a QSqlQuery instead of a string, which allows:
- Escaping quotes
- Using LIMIT (needed for history-max-items)
- Using ORDER BY (needed for sorting history)
- SELECTing columns (needed for quickmark completion)
- Creating a custom select (needed for history timestamp formatting)
From @TheCompiler:
To expand on this: I think it's fine to use KeyError on a lower level, i.e.
with the SqlTable object with a dict-like interface. However, on this higher
level, I think it makes sense to re-raise them as more specific exceptions.
For URL completion, time-based sorting is handled by the SQL model.
All the other models use simple alphabetical sorting. This allowed cleaning up
some logic in the sortfilter, removing DUMB_SORT, and removing the
completion.Role.sort.
This also removes the userdata completion field as it was only used in url
completion and is no longer necessary with the SQL model.
Store quickmarks and bookmarks in an in-memory sql database instead of a
python dict. Long-term storage is not affected, bookmarks and
quickmarks are still persisted in a text file.
The added and deleted signals were removed, as once sql completion
models are used the models will no longer need to update themselves.
This will set the stage for SQL-based history completion.
See #1765.
The browser-wide in-memory web history is now stored in an in-memory sql
database instead of a python dict. Long-term storage is not affected, it
is still persisted in a text file of the same format.
This will set the stage for SQL-based history completion.
See #1765.
When qutebrowser starts, it creates an in-memory sqlite database. One
can instantiate a SqlTable to create a new table in the database. The
object provides an interface to query and modify the table.
This intended to serve as the base class for the quickmark, bookmark,
and history manager objects in objreg. Instead of reading their data
into an in-memory dict, they will read into an in-memory sql table.
Eventually the completion models for history, bookmarks, and quickmarks
can be replaced with SqlQuery models for faster creation and filtering.
See #1765.
The new completion API no longer needs either of these. Instead of
referencing an enum member, cmdutils.argument.completion now points to
a function that returnsthe desired completion model.
This vastly simplifies the addition of new completion types. Previously
it was necessary to define the new model as well as editing usertypes
and completion.models.instances. Now it is only necessary to define a
single function under completion.models.
This is the next step of Completion Model/View Revamping (#74).
There was a circular import from
config -> keyconf -> miscmodels -> config.
This is resolved by scoping config's keyconf import to the one function
that uses it.
First step of Completion Model/View revamping (#74). Rewrite the
completion models as functions that each return an instance of a
CompletionModel class.
Caching is removed from all models except the UrlModel. Models other
than the UrlModel can be generated very quickly so caching just adds
needless complexity and can lead to incorrect results if one forgets to
wire up a signal.
When we open a background tab, it gets a hardcoded size (800x600 or so) because
it doesn't get resized by the layout yet.
By resizing it to the size it'll actually have later, we make sure scrolling to
an anchor in an background tab works, and JS also gets the correct size for
background tabs.
Fixes#1190Fixes#2495
See #1417
This continues the spirit of my previous PR and allows formatting tab
titles to designate when private mode is enabled. I didn't even realize
that tab title-format was a separate thing from window-title-format
(yes, it's in the name.. silly craftyguy), until now.
With per-domain settings, having a getter for a setting gets really complicated,
as there isn't one true value for a setting.
The only reason we needed those getters is to save away the default values for
some settings where we were unsure what the defaults are.
- For font setters, we can get the defaults from QFont, like QtWeb{Kit,Engine}
do.
- For font sizes, we hardcode the defaults QtWeb{Kit,Engine} hardcodes too.
- For maximum-page-in-cache, we hardcode 0, just like QtWebKit.
- For default-encoding, we hardcode iso-8559-1, like QtWeb{Kit,Engine}
- For offline-storage-default-quota, we hardcode 5MB, like QtWebKit
- For offline-web-application-cache-quota, we hardcode MAXINT as default value,
but we still keep the empty value in the config. It means "no quota"
internally in QtWebKit, but it's a too confusing value to have in the config.
- For object-cache-capacities it's a bit more complicated (the defaults are
calculated based on disk space), but let's just get rid of the setting
altogether in the next commit (see #1751).
Closes#2639.
This breaks things (with "ValueError: list.remove(x): x not in list") on
PyQt 5.9 (probably due to the destroyed object tracking it introduces?).
This was originally added in 0abb5cf738 to fix
some segfaults on exit, but things look much better with recent Qt versions.
This fixes an exception when trying to run :open-editor with a comment field
with QtWebKit:
16:37:51 DEBUG webelem webelem:is_editable:238 Checking if element is editable: <qutebrowser.browser.webkit.webkitelem.WebKitElement html='<div id="writer9997095275-writer" class="writer selectable no-lub put-art-here ui-droppable empty" style="min-height: 146px; width: 1169px;" contenteditable="true"></div>'>
16:37:51 ERROR misc crashsignal:exception_hook:205 Uncaught exception
Traceback (most recent call last):
File "/home/florian/proj/qutebrowser/git/qutebrowser/app.py", line 882, in eventFilter
return handler(event)
File "/home/florian/proj/qutebrowser/git/qutebrowser/app.py", line 842, in _handle_key_event
return man.eventFilter(event)
File "/home/florian/proj/qutebrowser/git/qutebrowser/keyinput/modeman.py", line 337, in eventFilter
return self._eventFilter_keypress(event)
File "/home/florian/proj/qutebrowser/git/qutebrowser/keyinput/modeman.py", line 168, in _eventFilter_keypress
handled = parser.handle(event)
File "/home/florian/proj/qutebrowser/git/qutebrowser/keyinput/basekeyparser.py", line 307, in handle
handled = self._handle_special_key(e)
File "/home/florian/proj/qutebrowser/git/qutebrowser/keyinput/basekeyparser.py", line 136, in _handle_special_key
self.execute(cmdstr, self.Type.special, count)
File "/home/florian/proj/qutebrowser/git/qutebrowser/keyinput/keyparser.py", line 44, in execute
self._commandrunner.run(cmdstr, count)
File "/home/florian/proj/qutebrowser/git/qutebrowser/commands/runners.py", line 275, in run
result.cmd.run(self._win_id, args, count=count)
File "/home/florian/proj/qutebrowser/git/qutebrowser/commands/command.py", line 525, in run
self.handler(*posargs, **kwargs)
File "/home/florian/proj/qutebrowser/git/qutebrowser/browser/commands.py", line 1600, in open_editor
tab.elements.find_focused(self._open_editor_cb)
File "/home/florian/proj/qutebrowser/git/qutebrowser/browser/webkit/webkittab.py", line 589, in find_focused
callback(webkitelem.WebKitElement(elem, tab=self._tab))
File "/home/florian/proj/qutebrowser/git/qutebrowser/browser/commands.py", line 1580, in _open_editor_cb
text = elem.value()
File "/home/florian/proj/qutebrowser/git/qutebrowser/browser/webkit/webkitelem.py", line 116, in value
assert isinstance(val, (int, float, str)), val
AssertionError: None
With general -> save-session on and only private windows open, we can easily get
a session file with "windows: []" in it. If we loaded such a file, we got no
windows at all when qutebrowser started.
Fixes#2664
We now automatically get out of fullscreen when switching away from a
fullscreened tab. This also means we can't get into a situation where we can't
leave fullscreen anymore.
Fixes#2379.
Turns out QWebEngineSettings.globalSettings() only sets things on the default
profile. We now get everything from the default profile settings, but set it on
both the default and the private profile.
Fixes#2638
(cherry picked from commit b11a4388cd10b6ff2fd917fca689ebdc50d581ae)
This tried to assert that we never create a DiskCache object when private
browsing is turned on. However, when initializing, we still create a global
DiskCache, so this will hit when qutebrowser is started with private browsing
turned on via the config.
We could just not create the DiskCache at all when started in private browsing
mode, however we might still need it later when opening a non-private window.
There's only one global DownloadManager with its own NAM (for downloads not
associated with a page). We can't really decide whether that should be private
or not, so as a best-effort approximation we simply make it private if private
browsing was turned on when starting qutebrowser.
If we simply use sorted() on a dict, we define insert before private-command,
which means the statusbar isn't going to be green when in insert mode while
private browsing.
Apart from checking for buttons with an href attribute (which made no sense at
all and should never return any element) this was identical to
webelem.Group.links.
There's actually no good reason to filter javascript links as we might want to
click them (or copy their URL) just like any other link - this fixes#2404.
With that being gone, we don't need FILTERS at all anymore, as we can check for
existence of the href attribute in the CSS selector instead.
This could happen for any of the attributes, but for tagName this actually
happens in the wild... Since elem.tagName is equal to elem.nodeName we just try
to use this.
Fixes#2569
This allows users to change the size of the favicon independently from
the size of the font/tab, in order to adjust the balance between
favicons and text. The drawing code is also adjusted to place the icon
relative to the text center, rather than the text top.
Works as expected even for values of 0.0 (which is equivalent to hiding
the favicon completely).
Closes#2549.
This makes those UI elements look the same on different platforms/OS styles,
with the small drawback of overriding the context menu style.
This most likely fixes#80 (though I couldn't reproduce that on Windows 10).
Before, we just returned the same data for both, but then we'll run into
same-origin restrictions as qute:history and qute:history/data are not the same
host.
Problem 1: Entering a command of `:::save` gives an error.
Problem 2: Entering a command of `:save\n` gives an error.
Both scenarios may seem a bit silly at first, but I encountered both by
copy/pasting a command:
1. Enter `:` in qutebrowser.
2. Copy a full line from a terminal starting with `:`.
3. You will now have both of the above problems.
Solution: Trim all whitespace and `:` of a command. This is also what
Vim does, by the way.
Problem: I like to edit `~/.config/qutebrowser/qutebrowser.conf`
manually with Vim. This works great, except that the current format is a
bit of a pain to deal with:
[section-name]
# section description
#
# [ Description of all the options]
actual options
So if I want to know the description or what the default value is, I
need to scroll up and back down.
Solution: change the order of the comments to:
# section description
[section-name]
# Option description
option = value
# Option description two
optiontwo = value
# Hello, world!
[section-two]
...
Which is much more convenient (and also what almost any other program
does).
(This patch changes much less code than it looks in the diff; I just
de-looped and moved `_str_option_desc` below `_str_items` as that makes
more sense since it gets called by `_str_items`).
Fixes#2304
In some cases, the finished handler fired before the error handler, e.g.
when downloading a 500 error page that is sent as attachment:
HTTP/1.1 500 Internal Server Error
Content-Type: application/octet-stream
Content-Disposition: inline; filename="attachment.jpg"
here we downloaded 0 bytes, fired the finished handler and after that
fired the error handler because of the 500 - but the finished handler
had already set our reply to None (and displayed the error message).
Otherwise, with 5ccafd62d4 the history starts
processing before the webview opened, and opening it is delayed until the whole
history is read.
Instead, call _process_args directly (I'm not even sure why it was using a 0ms
QTimer...) and schedule _init_late_modules after everything is really done.
Fixes#2296
By using a singleshot timer, we return fast from DownloadItem.open_file,
which in turn closes the prompt fast, which in turn doesn't allow a
second Ctrl-x to be registered, which in turn doesn't want to set the
filename twice.
Using focus() in JS there means that existing text in the field gets selected.
Move the cursor to the end after focusing it to prevent that.
Fixes#2359
Looks like the implementation was derived from Mozilla's
nsProxyAutoConfig.js, which is evaluated twice. It requires double
escaping.
In our case excessive escaping is harmful.
In particular it makes ip-matching regexp in isInNet() invalid and makes
it really slow as we go to dnsResolve() all the time, even when it's not
needed.
Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
The _find_nextprev function of browser/navigate.py only checks to see if
the rell attribute equals 'prev', 'previous', or 'next'. This patch
changes this to check for a set intersection between {'prev',
'previous'} or {'next'} and the set of the space separated list of the
rel attribute.
Otherwise, when checking later if the element has target=_blank set, we still
think it is.
This is not perfect as remove_blank_target also removes it from sub-elements,
but it's a good start.
Otherwise we call _tab_layout on something which is not a QStyleOptionTab, which
might cause AttributeErrors. In a crash report I couldn't reproduce, it got
called with a QStyleOptionMenuItem when right-clicking on the page.
I discovered this because of #2287 but it doesn't actually change anything
there. When we don't have a third column, subtract the scrollbar width from the
second one.
From the spec:
User agents should ensure, e.g. by means of an overlay, that the end user is
aware something is displayed fullscreen. User agents should provide a means of
exiting fullscreen that always works and advertise this to the user. This is
to prevent a site from spoofing the end user by recreating the user agent or
even operating system environment when fullscreen.
https://fullscreen.spec.whatwg.org/#security-and-privacy-considerations
We now use click() or focus() in JS if possible, or manually follow links in a
href attribute.
While this probably introduces some new corner cases, it fixes a handful of
older ones:
- window.open() in JS can now be handled correctly as we don't need hacks in
createWindow anymore.
- Focusing input fields with images now works - fixes#1613, #1879
- Hinting now works better on QtWebEngine with Qt 5.8 - fixes#2273
Also see #70.
- Added "QUTE_COMMANDLINE_TEXT" environment variable for userscripts.
- Updated documentation to include "QUTE_COMMANDLINE_TEXT" environment
variable for userscripts.
Ignore all keys with an empty .text() return value, not just modifier
keys. You can still use unusual things like ß for registers, but
XF86WakeUp is out. Fixes#2125.
Before, the module regexes didn't actually work properly, but we thought the
warnings were gone as they only were shown once because of __pycache__.
Now we instead don't filter by module, but simply hide those messages globally
during the earlyinit dependency import (which is the first import).
Fixes#2204
We didn't previously use PromptMode.download for MHTML download prompts
to avoid dealing with thinks like "Open download", but the new download
prompt is just way better than the old, which justifies the extra work.
This means that MHTML downloads can now also be opened directly.
- adds new `general` -> `default-open-dispatcher` setting (string)
- if set, will be used instead of QDesktopServices.openUrl (xdg-open backend)
- fixes the 'download handlers' part of issue #841
- note that this is only relevent to the `:download-open` command and other
methods of opening downloaded files from qutebrowser
Chooses the next value from the provided list of values (string-wise
comparison). Technically, the 'option!' syntax for toggling bools is now
redundant, but a translation from 'option!' to '--cycle option false
true' is kept for backwards compatibility.
The '--cycle' flag could also be technically optionally, since the only
thing that depends on it is preserving the error message for specifying
multiple values. (But I think it's best to keep it explicit, as a
principle-of-least-surprise thing)
Note: The business logic of picking the next value and setting it was
moved out to a separate function to avoid tripping pylint's
too-many-branches detector.
Fixes#47
Between v0.8.4 and now, the default keybinding for Ctrl-PgDown changed:
- ('tab-focus', ['J', '<Ctrl-PgDown>']),
+ ('tab-next', ['J', '<Ctrl-PgDown>']),
The existing keybinding was lower-cased in the config, but _is_new
didn't lowercase the new one, causing a conflict to show up.
Fixes#1835
See #1958
Supersedes #1986
This initially seemed like a nice feature, but it means 0 can't be bound
as a separate key anymore, and 0<Esc> gives weird error messages...
Reverts #1953.
Fixes#2032.
When starting a download due to unsupportedContent being emitted, we
need to use (and later adopt) the page's QNetworkAccessManager.
Since we need the whole adopting logic for that case anyways, let's keep
things as they were and always run downloads in per-tab QNAMs.
This reverts 53e360ec4b and fixes#2134.
When the user pressed esc during an SSL prompt, the message.ask call
returned None, which was handled fine by QtWebKit (which simply used an
'if') but failed with QtWebEngine (which returned the value to Qt).
Fixes#2123.
webelem.text() was only used without use_js=True from webelem.__str__.
Now we instead default to the use_js=True behavior and renamed the
method from text() to value().
The old behavior is instead directly implemented in __str__.
This changes how webelem.value (aka text) handles elements for which
is_content_editable() returns True, but I haven't found any cases where
this makes a difference.
This also fixes getting existing text from elements with QtWebEngine,
which closes#1957.
With QtWebEngine, handling a click (e.g. createWindow being called)
happens some time after a click event has been handled.
When setting override_target, that means we don't know when to set the
override target back, and we can't simply only unset it in createWindow
as elements not causing a new tab to be opened (like an input field) can
be hinted too.
Instead, we now only use override_target with QtWebKit, and simply
generate fake key events with the right modifiers to do the right thing
in createWindow for QtWebEngine.
Fixes#2119.
There were actually two issues here:
- The override_target was reset too early - now
acceptNavigationRequest/createWindow is responsible for resetting it.
- The QTBUG-54419 workaround opened any tabs in the background instead
of preserving their position/focus.
Fixes#2086.
Before, we used the same logic for QtWebKit and QtWebEngine, where we
simply set an attribute on the tab on a mousePressEvent and then handled
opening links in acceptNavigationRequest.
However, this caused random links to pop up in new tabs (probably to
things being much more async?) on QtWebEngine, so we now handle those
cases in createWindow and only use override_target from the tab there.
Fixes#2102.
Whenever we open a new tab in createWindow, we now set an attribute
marking it - as soon as its URL changes the first time, we then close
and re-open it in a new tab in TabbedBrowser.
When we use self._set_filename in self._set_fileobj, the file already
exists, so we need to force "overwriting" it.
Also, move temporary file handling to a dedicated _set_tempfile method,
so we can officially claim not supporting _set_fileobj with QtWebEngine
instead of supporting it with a hack.
This makes a lot of code eaiser, and we don't have per-tab settings yet
anyways. Also, with QtWebEngine, we can't honour any per-tab settings
for downloads...
Created configuration configdata.
Load and use template defined on configdata.
TODO: ability to conserve information between restart
TODO: ability to update title on configuration change
Moved pin information from BrowserTab to TabData.
Changed attribute from pin to pinned.
Changed "ifs" to implicit check boolen value.
Removed blancked line on before else statement.
Done so far:
Two new commands pin/unpin, both accept a index to help the organization
(maybe this should be more a flag and not exactly two commands)
Crtl+p to pin, Crtl+O to unpin (not sure which should a good default
shortcut)
If user tries to close a pinned tab it's asked to confirm
If user tries to open a URL in a pinned tab it receives a message with a
information that the tab is pinned and ignore the openurl command
Preserve the pinned information across restart if session is activated
Missing:
Visual indication of the tab being pinned
Tab appearance being distinct over other tabs
Make pinned tabs to be the firsts on the tab bar
This is not ready, but it would be good to get some feedback earlier
When we abort a question (e.g. by closing a tab with SSL errors) while
another prompt is queued, the right prompt gets closed by the question
being aborted, but that in turn aborts the question again, affecting the
next prompt.
We now make sure to hide prompts everywhere when the prompt mode was
left. We also make sure to hide the prompt widget before deleting, as it
might be deleted later.
