Commit Graph

357 Commits

Author SHA1 Message Date
Bora M. Alper
804fc10f1d fixed "sqlite3.ProgrammingError: Cannot operate on a closed database."
p.c. you should stop running what is using the database before closing
it!
2017-06-15 15:49:41 +03:00
Bora M. Alper
f4651fc6e3 fix mypy errors (although there are many # type: ignore lying around) 2017-06-15 13:40:03 +03:00
Bora M. Alper
6088ef6eec fix pylint errors (now it is rated 10.00/10 yeah!) 2017-06-14 18:05:14 +03:00
Bora M. Alper
235a42884f v0.6.0 is out!
v0.6.0, overall, tries to make magnetico more stable rather than
introducing new features.

## magneticod
* Removed better_bencode internal fork (but we are still using it so no
  loss at performance). This has caused some issues for some users and
  this release should solve the issue (e.g. #98).
* General cleanup, performance improvements, minor bug fixes...
  * Most notably among all, we improved our still-primitive congestion
    control support for BSD-based OSes, including OS X.

## magneticow
* There are no changes in magneticow, but version number is bumped to
  stay in-sync with magneticod.
2017-06-14 11:35:35 +03:00
Bora M. Alper
5e5d152cc1 initial proposal for the magneticow REST-ful HTTP API! 2017-06-13 15:17:52 +03:00
Bora M. Alper
dea0778511 fixed checking errno in dht.py 2017-06-13 15:16:48 +03:00
Bora M. Alper
11f573a01d Merge pull request #96 from matega/master
Removed ext_modules section from magneticod/setup.py
2017-06-12 22:12:29 +03:00
Galambos Máté
ef9c8a6855 Removed bencode.pyx requirement from magneticod/setup.py 2017-06-12 20:11:32 +02:00
Galambos Máté
ee5d4d2724 Removed ext_modules section from magneticod/setup.py 2017-06-12 19:28:24 +02:00
Bora M. Alper
2ab518a240 Merge remote-tracking branch 'origin/master' 2017-06-11 15:28:01 +03:00
Bora M. Alper
f1f0b9531d general cleanup, performance improvements, bug fixes
* Removed unnecessary functions such as those that just wraps a standard
  library function (e.g. '__random_bytes()' in SybilNode), and those
  that are wrongly abstracted (e.g. `cleanup` in __main__.py)

* Created `__build_GET_PEERS_query()` and `__build_ANNOUNCE_PEER()` in
  SybilNode to eliminate the cost of calling `bencode.dumps()` in these
  critical functions.

* Added some more comments to explain the rationale behind some
  decisions in-place.

* Improved our still-primitive congestion control support for BSD-based
  OSes, including OS X.
2017-06-11 15:27:31 +03:00
Bora M. Alper
200b1b65fd fixed formatting in CONTRIBTUING.rst 2017-06-11 10:05:35 +03:00
Bora M. Alper
a083bf40f9 ignore nodes with port 0, fixes #93 (presumably) 2017-06-10 22:10:33 +03:00
Bora M. Alper
ec708db607 removed better_bencode fork as I found a saner way for "loads2" 2017-06-10 22:04:23 +03:00
Bora M. Alper
643c72ee16 magnetico v0.5.0 is out!
Phew, what an effort it was to ship magnetico out this time!

CHANGELOG
=========

magneticod
----------
* Thanks to @richardkiss' heoric effort, magneticod now is ported to Python's
  asyncio (and uses uvloop for further performance improvements).

  CPU usage is reduced by 60%, and many older bugs are eliminated: #3, #55, #61

* Switched to better-bencode from bencoder.pyx for performance reasons. CPU
  utilization dropped by half.

magneticow
----------
* magneticow is finally became _fast_! Database indexes are created to speed-up
  file-list page (i.e. torrent info) and statistics.

Thanks to everyone who were involved in contributing to this release!
Special thanks to @ad-m who has been nagging me day and night to make me
take informed decision, I appreciate that.
2017-06-06 15:44:23 +03:00
Bora M. Alper
7fb2f19d2a improved uvloop support on different platforms (minor update) 2017-06-06 15:33:10 +03:00
Bora M. Alper
c96cb7de0e created index for discovered_on to speedup statistics 2017-06-06 15:13:26 +03:00
Bora M. Alper
3960915812 fixes #62 "TypeError: argument of type 'int' is not iterable" 2017-06-06 14:33:11 +03:00
Bora M. Alper
a0ee71a81d fixes infinite-recursion error in bencode.py 2017-06-06 12:58:01 +03:00
Bora M. Alper
58364dfb3f replaced better_bencode with bencoder.pyx
fixes #72
2017-06-05 22:26:08 +03:00
Bora M. Alper
3ed5a86c46 Added contribution guidelines. 2017-06-05 17:54:11 +03:00
Bora M. Alper
a73d872825 disable reports for pylint 2017-06-05 17:03:55 +03:00
Bora M. Alper
94a9fb730e sixth attempt to set up Travis CI... 2017-06-05 16:56:19 +03:00
Bora M. Alper
9dee183669 fifth attempt to set up Travis CI...
I am really sorry for this madness.
2017-06-05 16:53:59 +03:00
Bora M. Alper
5f855e1f2a fourth attempt to set up Travis CI 2017-06-05 16:48:35 +03:00
Bora M. Alper
e8e5b73b67 third attempt to set up Travis CI 2017-06-05 16:44:40 +03:00
Bora M. Alper
002d50ac89 second attempt to set up Travis CI 2017-06-05 16:38:06 +03:00
Bora M. Alper
ac86d48a31 first* attempt to implement Travis CI
*: and probably unsuccessful. =)
2017-06-05 14:38:36 +03:00
Bora M. Alper
6e0461a4c7 small stylistic change 2017-06-05 12:42:54 +03:00
Bora M. Alper
2c9de4691d Merge pull request #88 from richardkiss/master
Big speed-up in magneticow when clicking through to file list.
2017-06-05 12:37:18 +03:00
Richard Kiss
4e49b11f1d Big speed-up in magneticow when clicking through to file list. 2017-06-04 20:39:47 -07:00
Bora M. Alper
ae3de526ec another wave of bugfixes after merge 2017-06-02 17:00:49 +03:00
Bora M. Alper
fcecdba5b9 various bugfixes after merge 2017-06-02 16:49:39 +03:00
Bora M. Alper
0c9adb95d4 Merge branch 'richardkiss-asyncio' 2017-06-02 16:09:48 +03:00
Bora M. Alper
5da4b6c67d Merge branch 'asyncio' of https://github.com/richardkiss/magnetico into richardkiss-asyncio
# Conflicts:
#	magneticod/magneticod/__main__.py
#	magneticod/magneticod/dht.py
2017-06-02 16:09:28 +03:00
Bora M. Alper
d628269d67 Merge branch 'master' of github.com:boramalper/magnetico 2017-06-02 15:35:27 +03:00
Bora M. Alper
c6388f33e8 Merge branch 'richardkiss-asyncio' 2017-06-02 15:34:39 +03:00
Bora M. Alper
6a459d5e58 huge commit, code review done on asyncio port 2017-06-02 15:34:22 +03:00
Bora M. Alper
536ac2c4dc Removed "incoming finals" notice from README.md
I am free!
2017-06-01 12:13:08 +03:00
Bora M. Alper
04e6d583f3 Merge branch 'asyncio' of https://github.com/richardkiss/magnetico into richardkiss-asyncio 2017-05-30 12:45:17 +03:00
Bora M. Alper
6bceb9c784 v0.4.0 is released!
Changes are below.

magneticod:
* Performance improvements
  * Unregistering WRITE event when possible
  * Using files instead of memory for temporary SQLite
    objects
* Stability improvements
  * Limit metadata size to prevent crashes due to
    malicious or malformed answers.
  * Flushing metadata buffer while exiting gracefully

magneticow:
* Stability improvements
  * Homepage crash due to count functionality when the
    database is empty, is fixed.
* Statistics page added, although very slow!
2017-05-30 12:28:34 +03:00
Bora M. Alper
273237bd39 magneticow: fix 30-days period issue in stats
Previously magenticow calculated 30-days period as
literally 30 days (30 * 24 * 60 * 60 seconds) from NOW,
which was counter-intuitive for the user as it did not
correspond to whole days. Now that is fixed.
2017-05-30 12:01:51 +03:00
Bora M. Alper
e4c33ceb36 magneticod: store temp files in files and not in memory
(reduces the memory consumption)
2017-05-29 22:00:37 +03:00
Richard Kiss
0e389aa619 Query DB when checking if an infohash is new or not. 2017-05-27 16:11:29 -07:00
Richard Kiss
d7ead951a4 Refactor create_tasks out of main. 2017-05-27 16:05:22 -07:00
Richard Kiss
5d37737d0d Add some resource debug logging. 2017-05-27 12:20:24 -07:00
Richard Kiss
4dc11b047f Tidy up clean-up. Simplify fetch_metadata. 2017-05-24 14:50:26 -07:00
Richard Kiss
8df4015e06 Be a little smarter with task clean-up. 2017-05-24 12:51:25 -07:00
Richard Kiss
9b1bbfcaa1 Properly clean up fetch_metadata tasks. 2017-05-17 16:31:40 -07:00
Richard Kiss
29b99a338e ModuleNotFoundError (new in 3.6) => ImportError. 2017-05-17 12:10:40 -07:00