Oh, forgot to talk about compression. You can run the stem script against compressed tarballs but python didn't add lzma support until python 3.3...
https://stem.torproject.org/faq.html#how-do-i-read-tar-xz-descriptor-archive...
I suppose we could run over bz2 or gz tarballs, or upgrade python. But can't say the compressed benchmark is overly important.
Cheers! -Damian
On Thu, Jan 14, 2016 at 2:23 AM, Karsten Loesing karsten@torproject.org wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 13/01/16 20:42, Damian Johnson wrote:
This was Stem commit c01a9cda4e7699c7f4bd642c8e81ed45aab7a29b and Python version 2.7.10.
Great, thanks! Also what was the metrics-lib and zoossh commits?
metrics-lib: 8767f3e3bb8f6c9aa8cdb4c9fb0e9f2b545a7501 java version "1.7.0_51"
Zoossh: 2380e557e35532fd25870c8fc7a84a3fc951dbfc go version go1.5.2 darwin/amd64
Or should we add these performance tests for metrics-lib, Stem, and Zoossh to their own repository that also comes with scripts to fetch data from CollecTor? (Not sure if this is a smart thing to do, but I figured I should ask before adding things to the metrics-lib repository.)
Sinister plan #572 is that I'll add these results and scripts to the page we have with the library comparison. Probably tomorrow. If you'd care to clean up the metrics-lib examples that would be great. Otherwise I'll include what we have and you can send me patches later with what you'd like.
Sounds like a good plan. Thanks for doing this!
Here's one thing I realized when composing this message. When I pasted Zoossh results earlier, I compared them to the results for metrics-lib and Stem processing tarballs. But Zoossh can only process extracted tarballs. I just re-ran metrics-lib and Stem with extracted tarballs and included all results below:
server-descriptors-2015-11.tar.xz:
- metrics-lib: 0.334261 ms
server-descriptors-2015-11.tar:
- metrics-lib: 0.285430 ms
- Stem: 1.02 ms (357%)
server-descriptors-2015-11/:
- metrics-lib: 0.682293 ms
- Zoossh: 0.458566 ms (67%)
- Stem: 1.11 ms (163%)
extra-infos-2015-11.tar.xz:
- metrics-lib: 0.274610 ms
extra-infos-2015-11.tar:
- metrics-lib: 0.215500 ms
- Stem: 0.68 ms (316%)
consensuses-2015-11.tar.xz:
- metrics-lib: 255.760446 ms
consensuses-2015-11.tar:
- metrics-lib: 246.713092 ms
- Stem: 1393.10 ms (565%)
consensuses-2015-11/:
- metrics-lib: 283.910864 ms
- Stem: 1303.53 ms
- Zoossh: 83 ms
microdescs-2015-11.tar.xz[*]:
- metrics-lib: 0.099397 ms
microdescs-2015-11.tar[*]:
- metrics-lib: 0.066566 ms
- Stem: 0.66 ms (991%)
[*] The microdescs* tarballs contain microdesc consensuses and microdescriptors, but I only cared about the latter; what I did is extract tarballs, delete microdesc consensuses, and re-create and re-compress tarballs
I'm attaching a slightly updated version of the metrics-lib code. It's not cleaned up, but it's what I used to perform the measurements above.
All the best, Karsten
-----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org
iQEcBAEBAgAGBQJWl3cmAAoJEJD5dJfVqbCrew8H/1JnOCbZAXD9dBQUIYsZPu// jngO1Khf2lcggcHcxNMkhZS67KMmdKnyu0ZdjBHGdRNLZbQhhbDf5kQVtdx4xGim h4+cMgNEWqVp2gvfr/pvF6luns1rXfKREN/uxs9B1zrk5nEqWjAuPWqdgFbqA3Ui 03VOd1inUNZ7fWFwJpvSzb5RuZHusmLAWsDXO/607cJ/Of99QUIU5NWEBwwVTvgl ATd1+Slo2KIsNSpVPgtIbv345X7kTs2Jvt/ZvJsotQuRzn18d0A2ZayCddpQ896Z gffIp+xY+19tr1x27bwbDS44Jhb3p1Y1cKqvAUjZXzk8iSecxL+kSn5JCEdl2YY= =7nr/ -----END PGP SIGNATURE-----