For those on tor-dev@, this is a thread where Karsten and I have been talking about the future of Tor Weather. Present thoughts are to do one of the following...
a. Attempt to tap the community to find a maintainer (the service has been unmaintained for years).
b. Hire a developer from odesk to address its present issues. Karsten and I aren't especially thrilled about this idea since it would mean continuing to have an unmaintained service linger on.
c. Replace the functionality Weather provides with something else (that's most of what our recent discussion has been about).
d. Simply deprecate and later shut down the service.
- We redesign Weather based on stem.
As mentioned earlier this is already done to the extent that a patch exists. Weather's use of TorCtl was pretty simple so it was easy to map to stem, making some general improvements to the codebase in the process. If we opt for any plan involving the continuation of Weather than I'd like to see us do this since it's both easy, and will let Weather move off a deprecated dependency. We can then shift the service to Onionoo or rewrite it afterwards.
There's a patch for this? Sounds like a new Weather maintainer should apply this patch as their first action. (However, I'm not sure if we should apply it without having a maintainer, because if it breaks, who's going to pick up the pieces? Unless you'd want to do that?)
Yup, a patch but untested (I didn't invest time into starting up a new Weather instance). So we definitely do need a maintainer before applying it.
- (Even smarter ideas go here)
What about my earlier suggestion? I'd like to replace weather with a new torrc parameter, say...
SendNotifications me@some_address.com
... which is then included in the extrainfo descriptors. DocTor (or a new Weather if we really don't want to merge services) could then send notifications when a relay that previously set this goes down, or has reached a point of deserving a t-shirt. The only trouble with this is that it would expose real addresses in descriptor data (and hence, possibly spam).
Maybe there's something smart we can do to overcome this issue? Providing relay operators with a method for including real contact information would be helpful (as mentioned with the upgrade scenario earlier in the thread).
But we already have ContactInfo lines. What's the difference between the two? I'd find this rather confusing as a relay operator.
Differences are...
* This option would be an opt-in for receiving notifications about their relay.
* It resides in extrainfo descriptors rather than server descriptors (not sure if that'll lessen spam, but ContactInfo's definitely in the wrong spot).
* We've already trained users to throw garbage information into the ContactInfo. Since SendNotifications would be for automated systems like DocTor we could clearly state that it requires un-munged addresses, lest it doesn't work (and hence is pointless to set).
This said, without a solution to the spam issue I don't think this is a good idea. I'm hoping others on this list will have some ideas around this since a solution for the spam problem is the only thing standing in our way for having a good replacement for Weather.
Also, I dislike the fact that we're adding a config option to tor that is mostly used by the Weather service. What if we want to discontinue the Weather service? The config option will be around for another two years after that, but it won't do anything, which will make people think the tor program is broken.
Actually, DocTor rather than Weather - I want to shut Weather down. In theory this is a more general field for 'hey, automated services may send me notices regarding my relay', maybe also with 'here's tags for the notices I want'. Anyway, food for thought - an opt-in flag and email address is really most of what Weather provides us.
Also also, should this discussion happen on tor-dev@? :)
Perfectly fine with me - added.
Cheers! -Damian
Hey Damian,
I thought about your email for a while (though that's not the reason it took me so long to reply), but I didn't really come up with a good solution. But I have a few comments on details, so I decided to reply anyway.
On 10/11/13 9:57 AM, Damian Johnson wrote:
For those on tor-dev@, this is a thread where Karsten and I have been talking about the future of Tor Weather. Present thoughts are to do one of the following...
a. Attempt to tap the community to find a maintainer (the service has been unmaintained for years).
b. Hire a developer from odesk to address its present issues. Karsten and I aren't especially thrilled about this idea since it would mean continuing to have an unmaintained service linger on.
c. Replace the functionality Weather provides with something else (that's most of what our recent discussion has been about).
d. Simply deprecate and later shut down the service.
Just to be clear about shutting down the service, it's already scheduled to be shut down in 3 months from now, unless we find a maintainer:
https://trac.torproject.org/projects/tor/wiki/org/operations/Infrastructure
(Along with the Tor website, by the way. Fun!)
- We redesign Weather based on stem.
As mentioned earlier this is already done to the extent that a patch exists. Weather's use of TorCtl was pretty simple so it was easy to map to stem, making some general improvements to the codebase in the process. If we opt for any plan involving the continuation of Weather than I'd like to see us do this since it's both easy, and will let Weather move off a deprecated dependency. We can then shift the service to Onionoo or rewrite it afterwards.
There's a patch for this? Sounds like a new Weather maintainer should apply this patch as their first action. (However, I'm not sure if we should apply it without having a maintainer, because if it breaks, who's going to pick up the pieces? Unless you'd want to do that?)
Yup, a patch but untested (I didn't invest time into starting up a new Weather instance). So we definitely do need a maintainer before applying it.
- (Even smarter ideas go here)
What about my earlier suggestion? I'd like to replace weather with a new torrc parameter, say...
SendNotifications me@some_address.com
... which is then included in the extrainfo descriptors. DocTor (or a new Weather if we really don't want to merge services) could then send notifications when a relay that previously set this goes down, or has reached a point of deserving a t-shirt. The only trouble with this is that it would expose real addresses in descriptor data (and hence, possibly spam).
Maybe there's something smart we can do to overcome this issue? Providing relay operators with a method for including real contact information would be helpful (as mentioned with the upgrade scenario earlier in the thread).
But we already have ContactInfo lines. What's the difference between the two? I'd find this rather confusing as a relay operator.
Differences are...
- This option would be an opt-in for receiving notifications about their relay.
I still think a) that relay operators will be confused that they can enter their email address in two different places and b) that we shouldn't integrate Weather (or DocTor or however we call it) into little-t-tor by moving configuration options there.
- It resides in extrainfo descriptors rather than server descriptors
(not sure if that'll lessen spam, but ContactInfo's definitely in the wrong spot).
In theory, there's no big difference between adding these fields to server descriptors or extra-info descriptors. Recent clients only download microdescriptors which don't contain contact lines, AFAIK. And spammers can download tarballs of either descriptor type to learn contained email addresses.
- We've already trained users to throw garbage information into the
ContactInfo. Since SendNotifications would be for automated systems like DocTor we could clearly state that it requires un-munged addresses, lest it doesn't work (and hence is pointless to set).
This said, without a solution to the spam issue I don't think this is a good idea. I'm hoping others on this list will have some ideas around this since a solution for the spam problem is the only thing standing in our way for having a good replacement for Weather.
I disagree that spam is what prevents us from replacing Weather. The Weather codebase is unmaintained, which is why it needs to be replaced or cleaned up. The model of people opting in by registering their email address is not a bad one. We can even dump the part where Weather sends out welcome mails to new relay operators and instead make Weather more visible on the Tor website.
Also, I dislike the fact that we're adding a config option to tor that is mostly used by the Weather service. What if we want to discontinue the Weather service? The config option will be around for another two years after that, but it won't do anything, which will make people think the tor program is broken.
Actually, DocTor rather than Weather - I want to shut Weather down.
Okay, I guess that's just a question of scope. The old Java DocTor was only a consensus-health checker written for directory authority operators. If you want to extend scope to relay operators, sure, why not.
In theory this is a more general field for 'hey, automated services may send me notices regarding my relay', maybe also with 'here's tags for the notices I want'. Anyway, food for thought - an opt-in flag and email address is really most of what Weather provides us.
Agreed, Weather doesn't do much. Though, as I said earlier, that opt-in flag and email address don't belong into torrc, IMHO.
Also also, should this discussion happen on tor-dev@? :)
Perfectly fine with me - added.
Thanks.
All the best, Karsten
- This option would be an opt-in for receiving notifications about their relay.
I still think a) that relay operators will be confused that they can enter their email address in two different places and b) that we shouldn't integrate Weather (or DocTor or however we call it) into little-t-tor by moving configuration options there.
Ok. Personally I'm not overly in love with the additional torrc option, but like it better than maintaining a database just to track email addresses and opt-ins. If we're gonna opt for maintaining a Weather website and database then it should certainly have a maintainer to continue.
Actually, DocTor rather than Weather - I want to shut Weather down.
Okay, I guess that's just a question of scope. The old Java DocTor was only a consensus-health checker written for directory authority operators. If you want to extend scope to relay operators, sure, why not.
Gotcha. I've defined DocTor's scope as 'notifications based on the ongoing collection of present descriptor data', which overlaps with Weather (minus a website for opt-ins). DocTor has already grown a tad larger than just consensus-health since it now includes checks for descriptor validity and sybil attacks.
Cheers! -Damian
Hi, I have recently begun looking to volunteer development time to Tor, and when I stated this on #tor-dev, it was suggested among other things that I look at maintaining Weather. I have been reading through the code and looked at some of the tickets, but haven't made any fixes yet.
On 10/23/2013 10:02 AM, Damian Johnson wrote:
Actually, DocTor rather than Weather - I want to shut Weather down.
Okay, I guess that's just a question of scope. The old Java DocTor was only a consensus-health checker written for directory authority operators. If you want to extend scope to relay operators, sure, why not.
Gotcha. I've defined DocTor's scope as 'notifications based on the ongoing collection of present descriptor data', which overlaps with Weather (minus a website for opt-ins). DocTor has already grown a tad larger than just consensus-health since it now includes checks for descriptor validity and sybil attacks.
So, it seems that the desire is to replace whatever services Weather was supplying with increased DocTor robustness. In this case, then, what is the time scale for migration? I suppose I could still volunteer my maintainer services (watever they are) for Weather maintenance until its end, although I must say that I've never worked in an OSS project before and claim gross newbie ignorance and immaturity.
In particular, I'm not sure what a maintainer of Weather would do in this scenario. Am I just generally available for indefinite whatever, or am I actively working through bug tickets, trying to resolve them and pushing fixes to the site?
Also, seperate from and previous to my Weather proclivities, upon encountering this page
https://www.torproject.org/getinvolved/volunteer.html.en#Coding
I was intrigued by project a. PyDoctor, which '[is] about rewriting DocTor in Python'. So if DocTor is undergoing active development now, is this project idea now retracted?
I have no preference on the items I have discussed, and I am greatly interested in making Tor better in the most effective way I can, but I would appreciate a little guidance in doing so, at least for now. Thanks.
Justin
Hi, I have recently begun looking to volunteer development time to Tor, and when I stated this on #tor-dev, it was suggested among other things that I look at maintaining Weather. I have been reading through the code and looked at some of the tickets, but haven't made any fixes yet.
Awesome, we'd appreciate the help! A lot of services could certainly use some love. :)
I was intrigued by project a. PyDoctor, which '[is] about rewriting DocTor in Python'. So if DocTor is undergoing active development now, is this project idea now retracted?
Oops, that project is done - the DocTor codebase is now python...
https://gitweb.torproject.org/doctor.git/tree/refs/heads/master
I'll drop it from the site tomorrow.
Cheers! -Damian