Commit Graph

183 Commits

Author SHA1 Message Date
Ryan Roden-Corrent
1e1335aa5e Make various SQL code review changes.
- Fix outdated comments
- Use mock specs when possible
- More precise error message check in test_import_txt_invalid.
- Fix copyright message
- Tweak missing pyqt error message
- Dead code: remove group_by and where from sqlcategory.
  With the new separate completion table, these are no longer used.
- Move test_history out of webkit/. History is no longer purely webkit
  related, it could be webengine.
2017-07-03 09:45:08 -04:00
Ryan Roden-Corrent
6ac940fa32 Fix pylint/coverage errors.
Ensure 100% coverage for sqlcategory and history, and fix some linter
errors
2017-06-27 12:33:51 -04:00
Ryan Roden-Corrent
866f4653c7 Fix spelling existant -> existent. 2017-06-25 22:14:38 -04:00
Ryan Roden-Corrent
0f585eda4f Bring history.py back to 100% coverage.
The code of debug_dump_history was tweaked to handle a possible
OSException that can be thrown by open, which I noticed while trying to
test it.
2017-06-20 21:41:43 -04:00
Ryan Roden-Corrent
1fe1813431 Fix pylint errors. 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
fa39b82b3c Backup old history file after import.
Instead of removing it, move it to history.bak.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
a6a9ad72f9 Fix test_history_interface.
This was still using a history dict instead of SQL history.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
3a4ef09f58 More sql code review fixes 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
6fc61d12fc Assorted small fixes for sql code review. 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
309b6ba32c Move _import_history to history.py.
Also adjusts the history import test to operate at a higher level and
ensure the old text file is removed (or isn't, in the case of an error).
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
20000088de Add debug-dump-history and fix sql history tests.
Trying to read from the sql database from another process was flaky.
This adds a debug-dump-history command which is used by the history BDD
tests to validate the history contents.

It outputs history in the old pre-SQL text format, so it might be
useful for those who want to manipulate their history as text.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
87643040a4 Fix test_history for python < 3.6.
Mock.assert_called is only in python 3.6. For earlier versions we must
use `assert m.called`.

Weird errors only appearing in CI, trying to debug...
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
eb61269068 Fix qute://history javascript for SQL.
Returning "next" was no longer possible as the SQL query does not fetch
more items than necessary. This is solved by using a start time, a
limit, and an offset. The offset is needed to prevent fetching duplicate
items if multiple entries have the same timestamp.

Two of the history tests that relied on qute://history were changed to
rely on qute://history/data instead to make them less failure-prone.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
44080b8ad4 Fix flake8 errors in test_history 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
e661fb7446 Fix test_history.
History doesn't depend on standarddir anymore, the history file path get passed
by app.py.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
f110cf4d53 Fix long hang after importing history.
Turns out historyContains was getting called for the webkit backend multiple
times when the browser starts. This was calling `url in history`, which was
enumerating the entire history as `__contains__` was not defined.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
024386d189 Fail on history file parsing errors.
Instead of skipping bad history lines during the import to sql, fail hard. We
don't want to delete the user's old history file if we couldn't parse all of
the lines.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
6412c88277 Clean up history module.
Eliminate out-of-date docstring and remove an unused signal.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
3e63b62d6e Fix pylint/flake8 for sql work. 2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
8ff45331df Clean up sql implementation.
Now that sql is only used for history (not quickmarks/bookmarks) a number of
functions are no longer needed. In addition, primary key support was removed as
we actually need to support multiple entries for the same url with different
access times. The completion model will have to handle this by selecting
something like (url, title, max(atime)).

This also fixes up a number of tests that were broken with the last few
sql-related commits.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
80647b062a Convert old history file to sqlite.
If qutebrowser detects a history text file when it starts
(~/.local/share/qutebrowser/history by default on Linux), it will import this
file into the new sqlite database, then delete it.

The read is done as a coroutine as it can take some time.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
de5be0dc5a Store history in an on-disk sqlite database.
Instead of reading sqlite history from a file and storing it in an in-memory
database, just directly use an on-disk database. This resolves #755, where
history entries don't pop in to the completion menu immediately as they are
still being read asynchronously for a few seconds after the browser starts.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
4296eed429 Fix test_history cleanup failure.
The test may be skipped if the PyQt5.QtWebKitWidget import fails, but the
cleanup was still running and trying to delete a nonexistant web-history.
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
56f3b3a027 Small review fixups for SQL implementation.
Respond to the low-hanging code review fruit:

- Clean up some comments
- Remove an acidentally added duplicate init_autosave
- Combine two test_history tests
- Move test_init cleanup into a fixture to ensure it gets called.
- Name the _ argument of bind(_) to _key
- Ensure index is valid for first_item/last_item
- Move SqlException to top of module
- Rename test_index to test_getitem
- Return QItemFlags.None instead of None
- Fix copyright dates (its 2017 now!)
- Use * to force some args to be keyword-only
- Make some returns explicit
- Add sql to LOGGER_NAMES
- Add a comment to explain the sql escape statement
2017-06-19 07:44:11 -04:00
Ryan Roden-Corrent
fc5fd6096a Revert "Initialize SQL for two failing tests."
This reverts commit 386e227ce7534f1e427db7ba6d4e53dc153a49f3.
The problem was really state leakage, initializing sql for these tests isn't
necessary.
2017-06-19 07:42:12 -04:00
Ryan Roden-Corrent
788babbb61 Further prevent state leakage from test_init.
test_history.test_init also leaked state by leaving the instantiated history as
the parent of the QApp, which was causing test_debug to fail because it was
trying to dump the history object left from test_history.
2017-06-19 07:42:12 -04:00
Ryan Roden-Corrent
d658378af3 Eliminate test interference from webkit history.
Initializing the qtwebkit history backend left some global state that was
leaking into other tests.
2017-06-19 07:42:12 -04:00
Ryan Roden-Corrent
490250f5be Initialize SQL for two failing tests.
test_selectors and test_get_all_objects were running fine on my machine, but
for some reason is failing with "Driver not loaded" on Travis. Let's try
initializing SQL and see what happens.
2017-06-19 07:42:12 -04:00
Ryan Roden-Corrent
839d49a8ac Fix up pylint/flake8 for completion revamp. 2017-06-19 07:42:12 -04:00
Ryan Roden-Corrent
a774647c26 Get test_models mostly working again.
- Adjust _check_completions to work for CompletionModel and SqlCompletionModel
- Move sql initialization into a reusable fixture
- Remove the bookmark/quickmark/history stubs, as they're now handled by sql
- Disable quickmark/bookmark model tests until their completion is ported to
  sql.
- Disable urlmodel tests for features that have to be implemented in SQL:
    - LIMIT (for history-max-items)
    - Configurable column order (for quickmarks)
    - Configurable formatting (for timestamp-format
2017-06-19 07:42:12 -04:00
Ryan Roden-Corrent
acea0d3c67 Use SQL completion for the open command.
Now that history, bookmark, and quickmark storage are SQL-backed, use
a sql completion model to serve url completions.
2017-06-19 07:42:12 -04:00
Ryan Roden-Corrent
9477a2eeb2 Use SQL for history storage.
The browser-wide in-memory web history is now stored in an in-memory sql
database instead of a python dict. Long-term storage is not affected, it
is still persisted in a text file of the same format.

This will set the stage for SQL-based history completion.
See #1765.
2017-06-19 07:42:12 -04:00
Florian Bruhin
ea2fbc427a tests: Use match= for pytest.raises 2017-05-23 09:36:00 +02:00
Florian Bruhin
af7923de4d tests: Use pytest.param 2017-05-23 08:08:46 +02:00
Florian Bruhin
f66c1a0e44 Merge commit '3d9729839d6d9b5ee5d38afdf6ddf410dfca2027' into abbradar/pac-fix 2017-05-19 08:36:39 +02:00
Nikolay Amiantov
3d9729839d Fix crash on PAC evaluation error 2017-05-18 16:54:49 +03:00
Florian Bruhin
00a7a0cee6 Reorganize pylint config
This removes various stuff we don't need anymoe, and also re-enables and fixes
the import order check.
2017-05-17 20:20:12 +02:00
Florian Bruhin
12520bf4ba Install PyQt from PyPI for pylint
This means we can be sure to have QtWebEngine available and won't have QtWebKit.
2017-05-17 19:08:59 +02:00
Florian Bruhin
9408babef5 Also remove test for removed assertion 2017-05-16 12:56:25 +02:00
Florian Bruhin
086139110d Merge branch 'new-private-browsing' 2017-05-16 06:32:15 +02:00
Florian Bruhin
65952ca290 Fix webkitelem tests 2017-05-12 13:57:33 +02:00
Florian Bruhin
9e2aa65c02 Remove webelem.Group.prevnext
Apart from checking for buttons with an href attribute (which made no sense at
all and should never return any element) this was identical to
webelem.Group.links.
2017-05-12 09:41:12 +02:00
Florian Bruhin
203a5dff74 Get rid of webelem.FILTERS
There's actually no good reason to filter javascript links as we might want to
click them (or copy their URL) just like any other link - this fixes #2404.

With that being gone, we don't need FILTERS at all anymore, as we can check for
existence of the href attribute in the CSS selector instead.
2017-05-12 09:41:12 +02:00
Florian Bruhin
c6e31391de Fix most tests/lint 2017-05-10 09:19:24 +02:00
Florian Bruhin
822623f2ed Finally update copyrights... 2017-05-09 21:37:03 +02:00
Florian Bruhin
11c026bf4c Reenable QtWebKit cache with Qt 5.9.
This was fixed here:
https://codereview.qt-project.org/#/c/190818/

See #2427
2017-04-24 12:27:00 +02:00
Florian Bruhin
db8b6d3e68 Add test for QNetworkReply.abort 2017-04-17 16:02:57 +02:00
Florian Bruhin
4ec5700cbf Redirect qute:foo to qute://foo
Before, we just returned the same data for both, but then we'll run into
same-origin restrictions as qute:history and qute:history/data are not the same
host.
2017-04-06 21:18:58 +02:00
Florian Bruhin
0de3b5460e Only disable the cache on Qt 5.7.1
I ended up bisecting it, and https://codereview.qt-project.org/#/c/153977/
causes this, which is not in 5.7.0.
2017-04-04 08:24:50 +02:00
Florian Bruhin
2eb365b146 Also disable cache on Qt 5.7 2017-04-03 20:22:54 +02:00
Florian Bruhin
b6642e66fa Fix cache tests on Qt 5.8 2017-04-03 19:41:37 +02:00
Florian Bruhin
65f407e926 Remove webelem.style_property()
It's not used anymore, and not possible to support with QtWebEngine without a
massive performance hit when serializing elements.
2017-03-01 17:54:27 +01:00
Florian Bruhin
deb59fc66e Don't strip info when loading PAC from a file 2017-03-01 14:19:23 +01:00
Florian Bruhin
9bb5c9fdab Remove UserInfo and path/query for PAC URLs 2017-03-01 14:19:13 +01:00
Florian Bruhin
ca4f249c30 Use three-argument form of monkeypatch.*attr 2017-03-01 11:33:41 +01:00
Florian Bruhin
9656449545 Merge branch 'master' of https://github.com/kiryl/qutebrowser into kiryl-master 2017-02-25 17:53:05 +01:00
Florian Bruhin
3cc32e0b6a Fix lint 2017-02-25 17:24:27 +01:00
Florian Bruhin
02c913238b Fix test_webkitelem 2017-02-24 13:51:45 +01:00
Kirill A. Shutemov
13213724b0 PAC: fix isPlainHostName()
Fix isPlainHostName() implementation and add test-case for it.

Signed-off-by: Kirill A. Shutemov <kirill@shutemov.name>
2017-02-21 12:16:46 +03:00
Florian Bruhin
97982df1cb Adjust history unittest 2017-02-09 13:40:44 +01:00
Florian Bruhin
601110761d Fix test_history.test_init without QtWebKIt 2017-02-09 00:50:38 +01:00
Florian Bruhin
8779a1367e Fix test collection without QtWebKit 2017-02-09 00:50:37 +01:00
Florian Bruhin
b1a95a3930 Add automatic backend selection in earlyinit 2017-02-05 17:09:04 +01:00
Florian Bruhin
de50f30b9b Replace all GitHub links 2017-02-05 00:13:11 +01:00
Florian Bruhin
b6e31d4172 Fix parametrizing 2017-02-04 19:03:59 +01:00
Florian Bruhin
0cc7f845e6 Simplify test 2017-02-04 18:10:34 +01:00
Nikolay Amiantov
aec002fa29 Add more tests for PAC 2017-01-03 13:47:13 +03:00
Florian Bruhin
6c1b7dcca1 Move proxy/pac out of QtWebKit folder 2016-12-22 13:54:11 +01:00
Florian Bruhin
a3d0ea7e01 Adjust skipped test 2016-12-22 09:04:23 +01:00
Nikolay Amiantov
27d64d3680 Add tests for PAC resolver and fetcher 2016-12-07 02:01:19 +03:00
Maciej Wołczyk
f6802272c7 Added a test for quopri with spaces at the end of the line 2016-12-01 01:17:00 +01:00
Maciej Wołczyk
2cbea50152 Fixed style issues, corrected unit test 2016-11-29 19:10:36 +01:00
Florian Bruhin
52df867030 Fix test_webkitelem 2016-11-16 13:09:16 +01:00
Florian Bruhin
964ddb472b Add urlutils.data_url 2016-11-15 22:25:51 +01:00
Florian Bruhin
81d67f8a2c Remove support for an empty data/config/cachedir
See #2115
2016-11-14 23:13:21 +01:00
Florian Bruhin
cf32aac111 Move usertypes.DownloadTarget to downloads module 2016-11-09 08:15:58 +01:00
Florian Bruhin
6eef79e180 Adjust imports/objreg 2016-11-04 18:49:24 +01:00
Florian Bruhin
dd537c9119 Really fix test_mthml 2016-10-05 10:12:38 +02:00
Florian Bruhin
8dab1cf58a Fix test_mhtml.py 2016-10-05 09:09:42 +02:00
Florian Bruhin
5bef7dc74c Use file with known mimetype for qutescheme test 2016-09-14 16:48:49 +02:00
Florian Bruhin
3a27c45ac9 More cleanup in test_webkitqutescheme 2016-09-14 12:08:35 +02:00
Florian Bruhin
5b527d0f1e Rename test_qutescheme to test_webkitqutescheme 2016-09-14 12:07:26 +02:00
Florian Bruhin
cc1e134f25 Fix test_qutescheme.py 2016-09-14 12:05:15 +02:00
Florian Bruhin
e6680c3c60 Also redirect $HOME for QtWebEngine tests
See https://github.com/The-Compiler/qutebrowser/pull/1637#issuecomment-243043811
2016-09-12 19:55:31 +02:00
Florian Bruhin
b949e4d73a Start splitting DownloadModel from DownloadManager
This mostly works, apart from remove_item/remove_items not being
available on the model.
2016-09-11 14:58:07 +02:00
Florian Bruhin
4c55963dc3 Fix lint 2016-09-10 17:32:04 +02:00
Florian Bruhin
0ba74ad023 Add Qt model test for download model 2016-09-10 16:42:26 +02:00
Florian Bruhin
3e1583bb1c Make WebKitElement._is_visible private
It makes a lot of sense for this to be in webkitelem.py, but it should
not be public API as it's only used internally and can't be implemented
here with QtWebEngine.
2016-09-07 11:58:39 +02:00
Florian Bruhin
60c86a08c4 Get rid of WebElement.debug_text 2016-09-07 11:47:39 +02:00
Florian Bruhin
e700d11447 Add check_vanished() in webkitelem.insert_text 2016-09-07 11:29:26 +02:00
Florian Bruhin
8f9cfcf232 Get rid of webelem.run_js_async 2016-09-07 11:27:37 +02:00
Florian Bruhin
78d64f4791 Get rid of webkitelem.focus_elem 2016-09-07 11:24:28 +02:00
Florian Bruhin
f6ba859896 Move tabhistory.TabHistoryItem to misc.sessions
This makes browser.webkit.tabhistory contain only QtWebKit-specific
code.
2016-09-06 09:50:55 +02:00
Florian Bruhin
cf44019475 Don't require QtWebEngine for history tests 2016-09-05 22:25:20 +02:00
Florian Bruhin
7592345b6e Make unittests work without QtWebKit 2016-09-05 18:45:50 +02:00
Florian Bruhin
df3733af54 tests: Use pytest.fixture instead of yield_fixture
See #1877
2016-08-22 07:40:24 +02:00
Florian Bruhin
4c2b64b573 tests: Clean up test_http 2016-08-20 22:07:12 +02:00
Florian Bruhin
63c66945a4 Add webelem.click() and webelem.hover() 2016-08-18 14:32:19 +02:00
Florian Bruhin
6f24221c54 Replace WebElement.frame() by .has_frame() 2016-08-18 12:40:21 +02:00
Florian Bruhin
62db0095d1 Revert "Make webelem.rect_on_view work async"
This reverts commit 4e11613d2df064b138532c18f88bbf278c64f347.

We can actually make this synchronous just fine by collecting that
information when searching for the elements...
2016-08-18 12:40:21 +02:00