Commit Graph

2960 Commits

Author SHA1 Message Date
Ryan Roden-Corrent
a34df34208 Fix various test/flake8/pylint errors. 2017-07-03 08:12:47 -04:00
Ryan Roden-Corrent
22880926b1 Fix WebHistoryStub for delete argument change 2017-06-29 21:46:09 -04:00
Ryan Roden-Corrent
c1f5e77fc6 Implement "Current" completion for bind.
When binding a key, the first row will be the current binding if the key
is already bound. This should make it easier for users to tell when they
are binding a key that is already bound, and what it is bound to.
2017-06-29 21:44:44 -04:00
Ryan Roden-Corrent
262b028ee9 Match error message in lineparser test. 2017-06-29 20:49:05 -04:00
Ryan Roden-Corrent
c007f592b3 Use more intuitive argument order in sql.delete. 2017-06-29 20:43:42 -04:00
Ryan Roden-Corrent
6ac940fa32 Fix pylint/coverage errors.
Ensure 100% coverage for sqlcategory and history, and fix some linter
errors
2017-06-27 12:33:51 -04:00
Ryan Roden-Corrent
f06880c6e2 Fix history completion delete function.
In order to update SqlQueryModel's rowCount after re-running the query,
we must call setQuery again.
2017-06-27 08:42:10 -04:00
Ryan Roden-Corrent
62a849c2db Fix bugs introduced in test_models 2017-06-26 12:41:48 -04:00
Ryan Roden-Corrent
46161c3af0 Refactor delete_cur_item.
Taking the completion widget as an argument was overly complex.
The process now looks like:

1. CompletionView gets deletion request
2. CompletionView passes selected index to CompletionModel
3. CompletionModel passes the row data to the owning category
4. The category runs its custom completion function.

This also fixes a bug. With the switch to the hybrid (list/sql)
completion model, the view was no longer updating when items were
deleted. This fixes that by ensuring the correct signals are emitted.

The SQL model must be refreshed by running the query. We could try using
a SqlTableModel so we can call removeRows instead.

The test for deleting a url fails because qmodeltester claims the length
of the query model is still 3.
2017-06-26 08:57:36 -04:00
Ryan Roden-Corrent
866f4653c7 Fix spelling existant -> existent. 2017-06-25 22:14:38 -04:00
Ryan Roden-Corrent
0f585eda4f Bring history.py back to 100% coverage.
The code of debug_dump_history was tweaked to handle a possible
OSException that can be thrown by open, which I noticed while trying to
test it.
2017-06-20 21:41:43 -04:00
Florian Bruhin
4296a61b9a tests: Clean up check_history 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
891a6bcf14 Fix flake8 errors 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
c7a18a8b8d Fix tests for recent sql changes 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
4e87773d89 Use a dict instead of named params for insert.
This allows replace to be a named parameter and allows consolidating
some duplicate code between various insert methods.

This also fixes some tests that broke because batch insert was broken.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
f4f52ee204 Remove history.Entry.
No longer needed with sql backend. Query results build their own
namedtuple from the returned columns, and inserting new entries is just
done with named parameters.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
1fe1813431 Fix pylint errors. 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
cf23f42b99 Use splitlines in test_history_bdd again.
Just using read() returns a single string, and iterating over that
iterates over each character.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
679e001a48 Separate sqlcategory title from table name.
Also fix a number of sql/completion tests that were failing.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
fa39b82b3c Backup old history file after import.
Instead of removing it, move it to history.bak.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
a6a9ad72f9 Fix test_history_interface.
This was still using a history dict instead of SQL history.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
22b7b21d5a Use named placeholders for sql queries. 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
3a4ef09f58 More sql code review fixes 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
6fc61d12fc Assorted small fixes for sql code review. 2017-06-19 07:44:11 -04:00
Florian Bruhin
57d96a4512 Add a CompletionHistory instead of HistoryVisits table 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
309b6ba32c Move _import_history to history.py.
Also adjusts the history import test to operate at a higher level and
ensure the old text file is removed (or isn't, in the case of an error).
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
478a719f77 Use a prepared query for historyContains.
This is called often, hopefully a prepared query will speed it up.
This also modifies Query.run to return self for easier chaining, so you
can use `query.run.value()` instead of `query.run` ; query.value()`.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
565ba23f8c Don't instantiate completion models nedlessly.
For real this time. A mistake on the last commit like this meant models
were still spuriously instantiated.
Now that the completion model is reused, the layoutChanged signal needs
to be forwarded through, otherwise the view will not update.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
42243d3d97 Add more performance logging to completion. 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
a01c76db54 Remove 'group by' from url completion query.
This seemed to have a significant performance impact. Removing it means
that instead of just seeing the most recent atime for a given url, you
will see multiple entries.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
c297f047d2 Don't regenerate completion model needlessly.
If the completion model would stay the same, just keep it and update the
filter pattern rather than instantiating a new model each time the
pattern changes.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
2cd02be7b1 Remove CompletionModel.columns_to_filter.
Instead set this on inidividual categories, as that is where it actually
gets used. This makes it easier for SqlCompletionCategory to reuse a
prepared query (as it gets the filter field names in its constructor).
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
8fb6f45bec Don't set pattern in SqlCategory constructor.
This will be called by the Completer after construction anyways, this
was a duplicate call that could be expensive.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
cf89ffa971 Fix pylint/flake8 errors 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
2c501f7fb7 Fix url completion benchmark.
Still had old code from pre-SQL era.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
39b561a182 Fix BaseLineParser::test_double_open.
Don't tie the test to a particular error message. Ths failed because a
typo was fixed (AppendLineParser -> LineParser).
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
b1b521e0c2 Fix two history tests added recently.
These were added on master and needed to be adjusted slightly for the
new history check (which doesn't rely on reading a history file
anymore).
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
e67da51662 Use prepared SQL queries. 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
20000088de Add debug-dump-history and fix sql history tests.
Trying to read from the sql database from another process was flaky.
This adds a debug-dump-history command which is used by the history BDD
tests to validate the history contents.

It outputs history in the old pre-SQL text format, so it might be
useful for those who want to manipulate their history as text.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
87643040a4 Fix test_history for python < 3.6.
Mock.assert_called is only in python 3.6. For earlier versions we must
use `assert m.called`.

Weird errors only appearing in CI, trying to debug...
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
882da71397 Remove unused imports 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
eb61269068 Fix qute://history javascript for SQL.
Returning "next" was no longer possible as the SQL query does not fetch
more items than necessary. This is solved by using a start time, a
limit, and an offset. The offset is needed to prevent fetching duplicate
items if multiple entries have the same timestamp.

Two of the history tests that relied on qute://history were changed to
rely on qute://history/data instead to make them less failure-prone.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
71191f10a2 Only complete most recent atime for url.
The history completion query is extended to pick only the most recent item for
a given url.

The tests in test_models now check for ordering of elements.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
9d4888a772 Optimize qute://history for SQL backend.
The old implementation was looping through the whole history list, which for
SQL was selecting every row in the database. The history benchmark was taking
~2s. If this is rewritten as a specialized SQL query, the benchmark takes
~10ms, an order of magnitude faster than the original non-SQL implementation.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
784d9bb043 Remove code rendered dead by sql implementation.
Vulture exposed the following dead code:

- AppendLineParse was only used for reading the history text file, which is now
  a sql database (and the import code for the old text file is simpler and does
  not need a complex line parser)

- async_read_done is no longer used as importing the history text file is
  synchronous (and should only happen once)

- config._init_key_config is unused as it was moved to keyconf.init
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
a8ed9f1c2f Fix qute://version sql init bug.
Calling sql.init() in version.version() would replace the existing sql
connection and cause a crash when accessed by opening qute://version.

Now version relies on sql already being initted, and app.py inits sql early if
the --version arg is given.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
231bbe7c2b Fix race condition on history tests.
Two history end2end tests are failing because sqlite is not flushing to disk in
time to be read by the test process. My understanding is that sqlite should
take an exclusive lock while writing, so it is difficult to understand why this
is happening. This can be fixed by adding a delay, but that seems flaky.

I'm fixing it by checking qute://history instead of reading the database file.

See:
https://github.com/qutebrowser/qutebrowser/pull/2295#issuecomment-292786138
and the following discussion.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
44080b8ad4 Fix flake8 errors in test_history 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
e661fb7446 Fix test_history.
History doesn't depend on standarddir anymore, the history file path get passed
by app.py.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
f110cf4d53 Fix long hang after importing history.
Turns out historyContains was getting called for the webkit backend multiple
times when the browser starts. This was calling `url in history`, which was
enumerating the entire history as `__contains__` was not defined.
2017-06-19 07:44:11 -04:00