Hi Karsten,
A few questions on the Onionoo protocol specification:
If multiple parameters are specified in the GET request, does that imply a logical and or a logical or (or something else)? E.g., suppose the GET request has type=relay and lookup=FFFF... . To me it seems like that should return the relay with fingerprint FFFF..., if there is such a relay. But it could also mean return all relays as well as the router with fingerprint FFFF... .
The search parameter is described as "Return only relays with the parameter value matching the beginning of a nickname, (possibly $-prefixed) fingerprint, or IP address..." Do you mean that the parameter value matches the beginning of a nickname, the beginning of a (possibly $-prefixed) fingerprint, or is one of the relay IP addresses?
For ordering, what do you mean by "possible fields for ordering are: consensus_weight." This is really two questions. How is the consensus_weight ordering defined when producing a summary document (which doesn't include a consensus weight field)? Can clients not request that summary documents be ordered by any of the 'n', 'f', 'a', or 'r' fields?
Thanks!
- Norman
Hi Norman,
On 8/3/12 8:22 PM, Norman Danner wrote:
A few questions on the Onionoo protocol specification:
If multiple parameters are specified in the GET request, does that imply a logical and or a logical or (or something else)? E.g., suppose the GET request has type=relay and lookup=FFFF... . To me it seems like that should return the relay with fingerprint FFFF..., if there is such a relay. But it could also mean return all relays as well as the router with fingerprint FFFF... .
It's a logical and. (Will clarify that in the spec.)
The search parameter is described as "Return only relays with the parameter value matching the beginning of a nickname, (possibly $-prefixed) fingerprint, or IP address..." Do you mean that the parameter value matches the beginning of a nickname, the beginning of a (possibly $-prefixed) fingerprint, or is one of the relay IP addresses?
It's the beginning of any of the three. (Will clarify that in the spec, too.)
For ordering, what do you mean by "possible fields for ordering are: consensus_weight." This is really two questions. How is the consensus_weight ordering defined when producing a summary document (which doesn't include a consensus weight field)? Can clients not request that summary documents be ordered by any of the 'n', 'f', 'a', or 'r' fields?
The ordering is, like all parameters, defined independent of the returned document type. (Will try to clarify that in the spec, too.)
At the moment, clients can only request ordering by consensus weight, because that's the first and only thing we needed so far. In the future, clients should be able to order by nickname, fingerprint, address and maybe even the running bit. Related to the first answer, clients wouldn't request ordering by the "n" field, but by the "nickname", regardless of the document they request. Details documents probably give a better idea of which fields could be used for ordering.
Hope that makes sense.
Best, Karsten
Hi Karsten,
On 8/3/12 3:33 PM, Karsten Loesing wrote:
At the moment, clients can only request ordering by consensus weight, because that's the first and only thing we needed so far. In the future, clients should be able to order by nickname, fingerprint, address and maybe even the running bit. Related to the first answer, clients wouldn't request ordering by the "n" field, but by the "nickname", regardless of the document they request. Details documents probably give a better idea of which fields could be used for ordering.
Hope that makes sense.
Yes, it does, though now I have a couple more questions.
If I understand the weights documents correctly, there is no one consensus weight for a given router. What exactly is the ordering we should use when the request asks to order by consensus weight?
It seems like determining the consensus weights of the selected relays could be a relatively slow process, because at least with the filesystem-backed approach we are using right now, each individual weights file has to be opened and read. Is this correct, or am I missing something here?
- Norman
Hi Norman,
On 8/3/12 9:42 PM, Norman Danner wrote:
If I understand the weights documents correctly, there is no one consensus weight for a given router. What exactly is the ordering we should use when the request asks to order by consensus weight?
It seems like determining the consensus weights of the selected relays could be a relatively slow process, because at least with the filesystem-backed approach we are using right now, each individual weights file has to be opened and read. Is this correct, or am I missing something here?
Ordering by consensus weight (or any other field) is independent of the returned document type, like weights documents. The fact that weights documents contain consensus weight histories doesn't change that. You'd order by the latest known consensus weight of a relay. You don't have to open and parse all the weights files for that, which would be a performance disaster.
The way how the current Java Onionoo works is that we do the filtering and ordering only based on the summary helper file. The result is an ordered list of fingerprints. Only in the last step we look at the requested document type and fetch documents from disk by fingerprint.
The Python Onionoo should probably do the same thing but using a database. There would be a single database table that can be used to filter and sort results. Those results can then be joined with document tables by fingerprint to fetch all the JSON strings to return.
Best, Karsten
On 8/4/12 1:45 AM, Karsten Loesing wrote:
Hi Norman,
On 8/3/12 9:42 PM, Norman Danner wrote:
If I understand the weights documents correctly, there is no one consensus weight for a given router. What exactly is the ordering we should use when the request asks to order by consensus weight?
It seems like determining the consensus weights of the selected relays could be a relatively slow process, because at least with the filesystem-backed approach we are using right now, each individual weights file has to be opened and read. Is this correct, or am I missing something here?
Ordering by consensus weight (or any other field) is independent of the returned document type, like weights documents. The fact that weights documents contain consensus weight histories doesn't change that. You'd order by the latest known consensus weight of a relay. You don't have to open and parse all the weights files for that, which would be a performance disaster.
The way how the current Java Onionoo works is that we do the filtering and ordering only based on the summary helper file. The result is an ordered list of fingerprints. Only in the last step we look at the requested document type and fetch documents from disk by fingerprint.
The Python Onionoo should probably do the same thing but using a database. There would be a single database table that can be used to filter and sort results. Those results can then be joined with document tables by fingerprint to fetch all the JSON strings to return.
This is the structure I was imagining we would follow. Thanks.
- Norman
-
Karsten Loesing -
Norman Danner