Re: [tor-dev] Making microdescriptor tarballs available on metrics.tpo

Damian, can you try to parse these descriptors using stem, to see if the descriptor annotations are correct and if stem can parse them without issues?

Hi Karsten, sorry about the delay! Yup, stem parses them just fine (though processing compressed tarballs still takes an unpleasantly long time)...

% du -h microdescs-2014-01.tar.bz2 1.8M microdescs-2014-01.tar.bz2

% cat parse.py from stem.descriptor.reader import DescriptorReader

counter = 0

with DescriptorReader(["microdescs-2014-01.tar.bz2"]) as reader: for desc in reader: counter += 1

print "Found %i microdescriptors" % counter

% time python parse.py Found 14999 microdescriptors

real 67m15.022s user 65m50.259s sys 1m13.717s

Cheers! -Damian