Commit Graph

70 Commits

Author SHA1 Message Date
Bora M. Alper
d877ba2475 eliminate unnecessary calls to __decode_nodes in __on_FIND_NODE_resp.
fixes #102, thanks @Glandos!
2017-06-16 00:40:10 +03:00
Bora M. Alper
804fc10f1d fixed "sqlite3.ProgrammingError: Cannot operate on a closed database."
p.c. you should stop running what is using the database before closing
it!
2017-06-15 15:49:41 +03:00
Bora M. Alper
f4651fc6e3 fix mypy errors (although there are many # type: ignore lying around) 2017-06-15 13:40:03 +03:00
Bora M. Alper
6088ef6eec fix pylint errors (now it is rated 10.00/10 yeah!) 2017-06-14 18:05:14 +03:00
Bora M. Alper
235a42884f v0.6.0 is out!
v0.6.0, overall, tries to make magnetico more stable rather than
introducing new features.

## magneticod
* Removed better_bencode internal fork (but we are still using it so no
  loss at performance). This has caused some issues for some users and
  this release should solve the issue (e.g. #98).
* General cleanup, performance improvements, minor bug fixes...
  * Most notably among all, we improved our still-primitive congestion
    control support for BSD-based OSes, including OS X.

## magneticow
* There are no changes in magneticow, but version number is bumped to
  stay in-sync with magneticod.
2017-06-14 11:35:35 +03:00
Bora M. Alper
dea0778511 fixed checking errno in dht.py 2017-06-13 15:16:48 +03:00
Galambos Máté
ef9c8a6855 Removed bencode.pyx requirement from magneticod/setup.py 2017-06-12 20:11:32 +02:00
Galambos Máté
ee5d4d2724 Removed ext_modules section from magneticod/setup.py 2017-06-12 19:28:24 +02:00
Bora M. Alper
f1f0b9531d general cleanup, performance improvements, bug fixes
* Removed unnecessary functions such as those that just wraps a standard
  library function (e.g. '__random_bytes()' in SybilNode), and those
  that are wrongly abstracted (e.g. `cleanup` in __main__.py)

* Created `__build_GET_PEERS_query()` and `__build_ANNOUNCE_PEER()` in
  SybilNode to eliminate the cost of calling `bencode.dumps()` in these
  critical functions.

* Added some more comments to explain the rationale behind some
  decisions in-place.

* Improved our still-primitive congestion control support for BSD-based
  OSes, including OS X.
2017-06-11 15:27:31 +03:00
Bora M. Alper
a083bf40f9 ignore nodes with port 0, fixes #93 (presumably) 2017-06-10 22:10:33 +03:00
Bora M. Alper
ec708db607 removed better_bencode fork as I found a saner way for "loads2" 2017-06-10 22:04:23 +03:00
Bora M. Alper
643c72ee16 magnetico v0.5.0 is out!
Phew, what an effort it was to ship magnetico out this time!

CHANGELOG
=========

magneticod
----------
* Thanks to @richardkiss' heoric effort, magneticod now is ported to Python's
  asyncio (and uses uvloop for further performance improvements).

  CPU usage is reduced by 60%, and many older bugs are eliminated: #3, #55, #61

* Switched to better-bencode from bencoder.pyx for performance reasons. CPU
  utilization dropped by half.

magneticow
----------
* magneticow is finally became _fast_! Database indexes are created to speed-up
  file-list page (i.e. torrent info) and statistics.

Thanks to everyone who were involved in contributing to this release!
Special thanks to @ad-m who has been nagging me day and night to make me
take informed decision, I appreciate that.
2017-06-06 15:44:23 +03:00
Bora M. Alper
7fb2f19d2a improved uvloop support on different platforms (minor update) 2017-06-06 15:33:10 +03:00
Bora M. Alper
3960915812 fixes #62 "TypeError: argument of type 'int' is not iterable" 2017-06-06 14:33:11 +03:00
Bora M. Alper
a0ee71a81d fixes infinite-recursion error in bencode.py 2017-06-06 12:58:01 +03:00
Bora M. Alper
58364dfb3f replaced better_bencode with bencoder.pyx
fixes #72
2017-06-05 22:26:08 +03:00
Bora M. Alper
6e0461a4c7 small stylistic change 2017-06-05 12:42:54 +03:00
Richard Kiss
4e49b11f1d Big speed-up in magneticow when clicking through to file list. 2017-06-04 20:39:47 -07:00
Bora M. Alper
ae3de526ec another wave of bugfixes after merge 2017-06-02 17:00:49 +03:00
Bora M. Alper
fcecdba5b9 various bugfixes after merge 2017-06-02 16:49:39 +03:00
Bora M. Alper
5da4b6c67d Merge branch 'asyncio' of https://github.com/richardkiss/magnetico into richardkiss-asyncio
# Conflicts:
#	magneticod/magneticod/__main__.py
#	magneticod/magneticod/dht.py
2017-06-02 16:09:28 +03:00
Bora M. Alper
6a459d5e58 huge commit, code review done on asyncio port 2017-06-02 15:34:22 +03:00
Bora M. Alper
04e6d583f3 Merge branch 'asyncio' of https://github.com/richardkiss/magnetico into richardkiss-asyncio 2017-05-30 12:45:17 +03:00
Bora M. Alper
6bceb9c784 v0.4.0 is released!
Changes are below.

magneticod:
* Performance improvements
  * Unregistering WRITE event when possible
  * Using files instead of memory for temporary SQLite
    objects
* Stability improvements
  * Limit metadata size to prevent crashes due to
    malicious or malformed answers.
  * Flushing metadata buffer while exiting gracefully

magneticow:
* Stability improvements
  * Homepage crash due to count functionality when the
    database is empty, is fixed.
* Statistics page added, although very slow!
2017-05-30 12:28:34 +03:00
Bora M. Alper
e4c33ceb36 magneticod: store temp files in files and not in memory
(reduces the memory consumption)
2017-05-29 22:00:37 +03:00
Richard Kiss
0e389aa619 Query DB when checking if an infohash is new or not. 2017-05-27 16:11:29 -07:00
Richard Kiss
d7ead951a4 Refactor create_tasks out of main. 2017-05-27 16:05:22 -07:00
Richard Kiss
5d37737d0d Add some resource debug logging. 2017-05-27 12:20:24 -07:00
Richard Kiss
4dc11b047f Tidy up clean-up. Simplify fetch_metadata. 2017-05-24 14:50:26 -07:00
Richard Kiss
8df4015e06 Be a little smarter with task clean-up. 2017-05-24 12:51:25 -07:00
Richard Kiss
9b1bbfcaa1 Properly clean up fetch_metadata tasks. 2017-05-17 16:31:40 -07:00
Richard Kiss
29b99a338e ModuleNotFoundError (new in 3.6) => ImportError. 2017-05-17 12:10:40 -07:00
Richard Kiss
3e4eba740c Do explicit look-up of bootstrap nodes, and query all responses. 2017-05-15 15:58:13 -07:00
Richard Kiss
e6098ffb4a Use uvloop if available. 2017-05-15 15:44:36 -07:00
Richard Kiss
35c6176548 Move neighbour task work to tick task. 2017-05-15 00:06:02 -07:00
Richard Kiss
71f55f0c91 Reduce noise. 2017-05-14 14:13:34 -07:00
Richard Kiss
f38a796181 Add connection_lost. Properly handle shutdown of watch_q. 2017-05-14 14:04:46 -07:00
Richard Kiss
4b4c312fbe SybilNode now support pause_writing. 2017-05-14 14:00:50 -07:00
Richard Kiss
73d97d8188 Better cancel all outstanding tasks. 2017-05-14 13:55:19 -07:00
Richard Kiss
d04634b57b Improve dht shutdown. Notice writing pauses. 2017-05-14 13:43:15 -07:00
Richard Kiss
4515fa8b0a More cleanup. Only hit bootstrap if it seems necessary. 2017-05-14 13:37:22 -07:00
Richard Kiss
635fbe8cb1 More clean-up and simplification. 2017-05-14 13:03:01 -07:00
Richard Kiss
f3ae493308 More clean-up in error cases. 2017-05-14 12:03:50 -07:00
Richard Kiss
4aea5df886 Reduce leakage in bittorrent.py. 2017-05-14 11:41:11 -07:00
Richard Kiss
2f68ac3c7a Improve naming. 2017-05-13 22:55:49 -07:00
Richard Kiss
eb8a2c72e6 Fix conditional. 2017-05-13 22:52:43 -07:00
Richard Kiss
8da8d20b53 First crack at porting to asyncio. 2017-05-13 22:49:14 -07:00
Bora M. Alper
818667b363 Merge pull request #58 from ad-m/patch-1
Limit metadata size, move constants to a new file, allow custom database paths
2017-05-01 09:24:09 +04:00
Adam Dobrawy
18f6617214 Add humanfriendly.format_path in --database-file help 2017-05-01 06:47:21 +02:00
Adam Dobrawy
17bd563909 Fix import error in persistence 2017-05-01 06:37:51 +02:00