Commit Graph

74 Commits

Author SHA1 Message Date
Bora M. Alper
ecd04a84a0 re-add info_hash_index creation in magneticod:persistence
As @Glandos pointed out, we use it in magneticod too for determining
whether an info hash is new or not.
2017-07-04 22:22:31 +03:00
Bora M. Alper
07fe0d3eb4 corrected naming 2017-06-17 15:20:06 +03:00
Bora M. Alper
0c6d01184b fixed wrong naming (query/response confusion) 2017-06-17 15:18:17 +03:00
Bora M. Alper
e624837059 create indices in magneticow rather than magneticod (rationale below)
It's better to create indices in magneticow rather than magneticod,
because magneticod does NOT need them, but magneticow does.
2017-06-16 12:08:30 +03:00
Bora M. Alper
d877ba2475 eliminate unnecessary calls to __decode_nodes in __on_FIND_NODE_resp.
fixes #102, thanks @Glandos!
2017-06-16 00:40:10 +03:00
Bora M. Alper
804fc10f1d fixed "sqlite3.ProgrammingError: Cannot operate on a closed database."
p.c. you should stop running what is using the database before closing
it!
2017-06-15 15:49:41 +03:00
Bora M. Alper
f4651fc6e3 fix mypy errors (although there are many # type: ignore lying around) 2017-06-15 13:40:03 +03:00
Bora M. Alper
6088ef6eec fix pylint errors (now it is rated 10.00/10 yeah!) 2017-06-14 18:05:14 +03:00
Bora M. Alper
235a42884f v0.6.0 is out!
v0.6.0, overall, tries to make magnetico more stable rather than
introducing new features.

## magneticod
* Removed better_bencode internal fork (but we are still using it so no
  loss at performance). This has caused some issues for some users and
  this release should solve the issue (e.g. #98).
* General cleanup, performance improvements, minor bug fixes...
  * Most notably among all, we improved our still-primitive congestion
    control support for BSD-based OSes, including OS X.

## magneticow
* There are no changes in magneticow, but version number is bumped to
  stay in-sync with magneticod.
2017-06-14 11:35:35 +03:00
Bora M. Alper
dea0778511 fixed checking errno in dht.py 2017-06-13 15:16:48 +03:00
Galambos Máté
ef9c8a6855 Removed bencode.pyx requirement from magneticod/setup.py 2017-06-12 20:11:32 +02:00
Galambos Máté
ee5d4d2724 Removed ext_modules section from magneticod/setup.py 2017-06-12 19:28:24 +02:00
Bora M. Alper
f1f0b9531d general cleanup, performance improvements, bug fixes
* Removed unnecessary functions such as those that just wraps a standard
  library function (e.g. '__random_bytes()' in SybilNode), and those
  that are wrongly abstracted (e.g. `cleanup` in __main__.py)

* Created `__build_GET_PEERS_query()` and `__build_ANNOUNCE_PEER()` in
  SybilNode to eliminate the cost of calling `bencode.dumps()` in these
  critical functions.

* Added some more comments to explain the rationale behind some
  decisions in-place.

* Improved our still-primitive congestion control support for BSD-based
  OSes, including OS X.
2017-06-11 15:27:31 +03:00
Bora M. Alper
a083bf40f9 ignore nodes with port 0, fixes #93 (presumably) 2017-06-10 22:10:33 +03:00
Bora M. Alper
ec708db607 removed better_bencode fork as I found a saner way for "loads2" 2017-06-10 22:04:23 +03:00
Bora M. Alper
643c72ee16 magnetico v0.5.0 is out!
Phew, what an effort it was to ship magnetico out this time!

CHANGELOG
=========

magneticod
----------
* Thanks to @richardkiss' heoric effort, magneticod now is ported to Python's
  asyncio (and uses uvloop for further performance improvements).

  CPU usage is reduced by 60%, and many older bugs are eliminated: #3, #55, #61

* Switched to better-bencode from bencoder.pyx for performance reasons. CPU
  utilization dropped by half.

magneticow
----------
* magneticow is finally became _fast_! Database indexes are created to speed-up
  file-list page (i.e. torrent info) and statistics.

Thanks to everyone who were involved in contributing to this release!
Special thanks to @ad-m who has been nagging me day and night to make me
take informed decision, I appreciate that.
2017-06-06 15:44:23 +03:00
Bora M. Alper
7fb2f19d2a improved uvloop support on different platforms (minor update) 2017-06-06 15:33:10 +03:00
Bora M. Alper
3960915812 fixes #62 "TypeError: argument of type 'int' is not iterable" 2017-06-06 14:33:11 +03:00
Bora M. Alper
a0ee71a81d fixes infinite-recursion error in bencode.py 2017-06-06 12:58:01 +03:00
Bora M. Alper
58364dfb3f replaced better_bencode with bencoder.pyx
fixes #72
2017-06-05 22:26:08 +03:00
Bora M. Alper
6e0461a4c7 small stylistic change 2017-06-05 12:42:54 +03:00
Richard Kiss
4e49b11f1d Big speed-up in magneticow when clicking through to file list. 2017-06-04 20:39:47 -07:00
Bora M. Alper
ae3de526ec another wave of bugfixes after merge 2017-06-02 17:00:49 +03:00
Bora M. Alper
fcecdba5b9 various bugfixes after merge 2017-06-02 16:49:39 +03:00
Bora M. Alper
5da4b6c67d Merge branch 'asyncio' of https://github.com/richardkiss/magnetico into richardkiss-asyncio
# Conflicts:
#	magneticod/magneticod/__main__.py
#	magneticod/magneticod/dht.py
2017-06-02 16:09:28 +03:00
Bora M. Alper
6a459d5e58 huge commit, code review done on asyncio port 2017-06-02 15:34:22 +03:00
Bora M. Alper
04e6d583f3 Merge branch 'asyncio' of https://github.com/richardkiss/magnetico into richardkiss-asyncio 2017-05-30 12:45:17 +03:00
Bora M. Alper
6bceb9c784 v0.4.0 is released!
Changes are below.

magneticod:
* Performance improvements
  * Unregistering WRITE event when possible
  * Using files instead of memory for temporary SQLite
    objects
* Stability improvements
  * Limit metadata size to prevent crashes due to
    malicious or malformed answers.
  * Flushing metadata buffer while exiting gracefully

magneticow:
* Stability improvements
  * Homepage crash due to count functionality when the
    database is empty, is fixed.
* Statistics page added, although very slow!
2017-05-30 12:28:34 +03:00
Bora M. Alper
e4c33ceb36 magneticod: store temp files in files and not in memory
(reduces the memory consumption)
2017-05-29 22:00:37 +03:00
Richard Kiss
0e389aa619 Query DB when checking if an infohash is new or not. 2017-05-27 16:11:29 -07:00
Richard Kiss
d7ead951a4 Refactor create_tasks out of main. 2017-05-27 16:05:22 -07:00
Richard Kiss
5d37737d0d Add some resource debug logging. 2017-05-27 12:20:24 -07:00
Richard Kiss
4dc11b047f Tidy up clean-up. Simplify fetch_metadata. 2017-05-24 14:50:26 -07:00
Richard Kiss
8df4015e06 Be a little smarter with task clean-up. 2017-05-24 12:51:25 -07:00
Richard Kiss
9b1bbfcaa1 Properly clean up fetch_metadata tasks. 2017-05-17 16:31:40 -07:00
Richard Kiss
29b99a338e ModuleNotFoundError (new in 3.6) => ImportError. 2017-05-17 12:10:40 -07:00
Richard Kiss
3e4eba740c Do explicit look-up of bootstrap nodes, and query all responses. 2017-05-15 15:58:13 -07:00
Richard Kiss
e6098ffb4a Use uvloop if available. 2017-05-15 15:44:36 -07:00
Richard Kiss
35c6176548 Move neighbour task work to tick task. 2017-05-15 00:06:02 -07:00
Richard Kiss
71f55f0c91 Reduce noise. 2017-05-14 14:13:34 -07:00
Richard Kiss
f38a796181 Add connection_lost. Properly handle shutdown of watch_q. 2017-05-14 14:04:46 -07:00
Richard Kiss
4b4c312fbe SybilNode now support pause_writing. 2017-05-14 14:00:50 -07:00
Richard Kiss
73d97d8188 Better cancel all outstanding tasks. 2017-05-14 13:55:19 -07:00
Richard Kiss
d04634b57b Improve dht shutdown. Notice writing pauses. 2017-05-14 13:43:15 -07:00
Richard Kiss
4515fa8b0a More cleanup. Only hit bootstrap if it seems necessary. 2017-05-14 13:37:22 -07:00
Richard Kiss
635fbe8cb1 More clean-up and simplification. 2017-05-14 13:03:01 -07:00
Richard Kiss
f3ae493308 More clean-up in error cases. 2017-05-14 12:03:50 -07:00
Richard Kiss
4aea5df886 Reduce leakage in bittorrent.py. 2017-05-14 11:41:11 -07:00
Richard Kiss
2f68ac3c7a Improve naming. 2017-05-13 22:55:49 -07:00
Richard Kiss
eb8a2c72e6 Fix conditional. 2017-05-13 22:52:43 -07:00