On 28 Sep 2014, at 02:12, Tom Ritter tom@ritter.vg wrote:
why not also change the consensus and related document formats to be something more efficient than ASCII text? Taking the latest consensus and doing some rough estimates, I found the following:
Original consensus, xz-ed: 407K Change flags to uint16: ~399K +Removing names: 363K +Compressing IPv6 to 16Bytes + 4 Bytes - 360K +Compressing IPv4 to 4 Bytes + 4Bytes + 4bytes - 315K +Compressing the Datetime to 4 bytes - 291K +Compressing the Version string to 4bytes - 288K +Replacing reject 1-65K to a single byte - 287K +Replacing Bandwidth=# with a 4 byte - 273K
These numbers are optimistic - you won't see quite this much gain, but if I'm understanding you correctly that the consensus is painful, it seems like you could save at least 50K-70K out of 400K with relatively straightforward changes.
This analysis doesn't make much sense, I'm afraid. We use compression on the wire, so repeating flags as human-readable strings has a much lower overhead than you estimate, for example. Re-doing your estimates with actually compressed consensuses might make sense, but probably you'll see a lot less value.
Cheers Sebastian