On 8/4/12 1:45 AM, Karsten Loesing wrote:
Hi Norman,
On 8/3/12 9:42 PM, Norman Danner wrote:
If I understand the weights documents correctly, there is no one consensus weight for a given router. What exactly is the ordering we should use when the request asks to order by consensus weight?
It seems like determining the consensus weights of the selected relays could be a relatively slow process, because at least with the filesystem-backed approach we are using right now, each individual weights file has to be opened and read. Is this correct, or am I missing something here?
Ordering by consensus weight (or any other field) is independent of the returned document type, like weights documents. The fact that weights documents contain consensus weight histories doesn't change that. You'd order by the latest known consensus weight of a relay. You don't have to open and parse all the weights files for that, which would be a performance disaster.
The way how the current Java Onionoo works is that we do the filtering and ordering only based on the summary helper file. The result is an ordered list of fingerprints. Only in the last step we look at the requested document type and fetch documents from disk by fingerprint.
The Python Onionoo should probably do the same thing but using a database. There would be a single database table that can be used to filter and sort results. Those results can then be joined with document tables by fingerprint to fetch all the JSON strings to return.
This is the structure I was imagining we would follow. Thanks.
- Norman