Hey guys,
So I just started running a non-exit relay on a Raspberry Pi, and have hit a problem where it seems huge numbers of circuits are being created which overwhelms the system and can cause tor to crash. I read here ( https://blog.torproject.org/blog/how-to-handle-millions-new-tor-clients) that upgrading to 0.2.4.x might help with this, so I followed the instructions here:
https://www.torproject.org/docs/debian#development
And ran apt-get update, apt-get upgrade to get the latest version which turned out to be 0.2.4.11-alpha-1~wheezy+1
There seems to be a problem with this though as when the install finishes (or when I try to restart tor), I get this error:
pi@torberrypi /etc/tor $ sudo /etc/init.d/tor restart /etc/init.d/tor: line 111: 9701 Illegal instruction $DAEMON $VERIFY_ARGS > /dev/null [FAIL] Checking if tor configuration is valid ... failed! /etc/init.d/tor: line 111: 972 https://www.torproject.org/docs/debian#development4 Illegal instruction $DAEMON $VERIFY_ARGS 1>&2
Tor is not starting and I'm unsure how to proceed - help appreciated! :)
Chris
On Mon, Oct 14, 2013 at 08:26:52PM +0100, Chris Whittleston wrote:
So I just started running a non-exit relay on a Raspberry Pi, and have hit a problem where it seems huge numbers of circuits are being created which overwhelms the system and can cause tor to crash. I read here ( https://blog.torproject.org/blog/how-to-handle-millions-new-tor-clients) that upgrading to 0.2.4.x might help with this
No, 0.2.4.x means that you make things better for users, but alas it does not magically make your relay receive fewer circuit create requests.
, so I followed the instructions here:
Read the bold words on that page that start with "Raspbian is not Debian."
And ran apt-get update, apt-get upgrade to get the latest version which turned out to be 0.2.4.11-alpha-1~wheezy+1
There seems to be a problem with this though as when the install finishes (or when I try to restart tor), I get this error:
pi@torberrypi /etc/tor $ sudo /etc/init.d/tor restart /etc/init.d/tor: line 111: 9701 Illegal instruction $DAEMON $VERIFY_ARGS > /dev/null [FAIL] Checking if tor configuration is valid ... failed! /etc/init.d/tor: line 111: 972 https://www.torproject.org/docs/debian#development4 Illegal instruction $DAEMON $VERIFY_ARGS 1>&2
Tor is not starting and I'm unsure how to proceed - help appreciated! :)
You might like http://tor.stackexchange.com/questions/242/how-to-run-tor-on-raspbian-on-the...
--Roger
Aha - makes sense, I'll just build it myself. Thanks for the quick response.
So - the new handshake in 0.2.4.x doesn't help with the ongoing issues? On that page I linked it was suggested it might...
Chris
On 14 October 2013 20:33, Roger Dingledine arma@mit.edu wrote:
On Mon, Oct 14, 2013 at 08:26:52PM +0100, Chris Whittleston wrote:
So I just started running a non-exit relay on a Raspberry Pi, and have
hit
a problem where it seems huge numbers of circuits are being created which overwhelms the system and can cause tor to crash. I read here ( https://blog.torproject.org/blog/how-to-handle-millions-new-tor-clients) that upgrading to 0.2.4.x might help with this
No, 0.2.4.x means that you make things better for users, but alas it does not magically make your relay receive fewer circuit create requests.
, so I followed the instructions here:
Read the bold words on that page that start with "Raspbian is not Debian."
And ran apt-get update, apt-get upgrade to get the latest version which turned out to be 0.2.4.11-alpha-1~wheezy+1
There seems to be a problem with this though as when the install finishes (or when I try to restart tor), I get this error:
pi@torberrypi /etc/tor $ sudo /etc/init.d/tor restart /etc/init.d/tor: line 111: 9701 Illegal instruction $DAEMON $VERIFY_ARGS > /dev/null [FAIL] Checking if tor configuration is valid ... failed! /etc/init.d/tor: line 111: 972 https://www.torproject.org/docs/debian#development4 Illegal instruction $DAEMON $VERIFY_ARGS 1>&2
Tor is not starting and I'm unsure how to proceed - help appreciated! :)
You might like
http://tor.stackexchange.com/questions/242/how-to-run-tor-on-raspbian-on-the...
--Roger
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On Mon, Oct 14, 2013 at 08:36:58PM +0100, Chris Whittleston wrote:
Aha - makes sense, I'll just build it myself. Thanks for the quick response.
So - the new handshake in 0.2.4.x doesn't help with the ongoing issues? On that page I linked it was suggested it might...
It does help! It helps clients a lot.
The change is that your relay now prioritizes requests from 0.2.4 clients, and then spends whatever time it has left handling requests from 0.2.3 clients.
--Roger
I see - so I'll probably still see the problem with a huge number of circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can overclock the CPU...
Chris
On 14 October 2013 20:48, Roger Dingledine arma@mit.edu wrote:
On Mon, Oct 14, 2013 at 08:36:58PM +0100, Chris Whittleston wrote:
Aha - makes sense, I'll just build it myself. Thanks for the quick
response.
So - the new handshake in 0.2.4.x doesn't help with the ongoing issues?
On
that page I linked it was suggested it might...
It does help! It helps clients a lot.
The change is that your relay now prioritizes requests from 0.2.4 clients, and then spends whatever time it has left handling requests from 0.2.3 clients.
--Roger
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
On 2013-10-14 22:01, Chris Whittleston wrote:
I see - so I'll probably still see the problem with a huge number of circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can overclock the CPU...
Only option that I know of is to reduce the bandwidth you advertise to the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload.
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme m7527@abc.se wrote:
On 2013-10-14 22:01, Chris Whittleston wrote:
I see - so I'll probably still see the problem with a huge number of circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can overclock the CPU...
Only option that I know of is to reduce the bandwidth you advertise to the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload.
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests. It *will* affect the *average* bandwidth used by your relay, but every once in a while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still an unresolved issue.
On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote:
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme <m7527@abc.se mailto:m7527@abc.se> wrote:
On 2013-10-14 22:01, Chris Whittleston wrote: > I see - so I'll probably still see the problem with a huge number of > circuits being created after I've finished building 0.2.4. Is there > any way to limit this, I'm guessing reducing the bandwidth wouldn't > actually help? I guess I'll look into how much further I can overclock > the CPU... Only option that I know of is to reduce the bandwidth you advertise to the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload. _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays-- *Dr Chris Whittleston 栗主* Department of Chemistry University of Cambridge Lensfield Road, Cambridge, CB2 1EW Email: csw34@cam.ac.uk mailto:csw34@cam.ac.uk Tel: +44 (0)1223 336423
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9
Do you think it might help to restart tor every 24 hours or so using cron Dan - or would that adversely affect the network too much/not actually help? On 14 Oct 2013 22:32, "Dan Staples" danstaples@disman.tl wrote:
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests. It *will* affect the *average* bandwidth used by your relay, but every once in a while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still an unresolved issue.
On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote:
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme <m7527@abc.se mailto:m7527@abc.se> wrote:
On 2013-10-14 22:01, Chris Whittleston wrote: > I see - so I'll probably still see the problem with a huge numberof
> circuits being created after I've finished building 0.2.4. Is there > any way to limit this, I'm guessing reducing the bandwidth wouldn't > actually help? I guess I'll look into how much further I can overclock > the CPU... Only option that I know of is to reduce the bandwidth you advertiseto
the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload. _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays-- *Dr Chris Whittleston 栗主* Department of Chemistry University of Cambridge Lensfield Road, Cambridge, CB2 1EW Email: csw34@cam.ac.uk mailto:csw34@cam.ac.uk Tel: +44 (0)1223 336423
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9 _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
I don't think it would make a difference. When my relay is experiencing a circuit-creation storm, I've restarted Tor, and even stopped it for a few minutes, but as soon as it comes back online, it gets slammed again. Only leaving it offline for an hour or so and then turning it back on seems to mitigate the problem.
On Mon 14 Oct 2013 06:11:10 PM EDT, Chris Whittleston wrote:
Do you think it might help to restart tor every 24 hours or so using cron Dan - or would that adversely affect the network too much/not actually help?
On 14 Oct 2013 22:32, "Dan Staples" <danstaples@disman.tl mailto:danstaples@disman.tl> wrote:
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests. It *will* affect the *average* bandwidth used by your relay, but every once in a while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still an unresolved issue. On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote: > Thanks Logforme - yeah I was trying that before I sent the first email > in this chain, but maybe I didn't go low enough with the advertised > bandwidth. When the 0.2.4 compilation is done (it's still chugging > along) I'll try going lower and see if it helps. > > Chris > > > On 14 October 2013 21:38, Logforme <m7527@abc.se <mailto:m7527@abc.se> > <mailto:m7527@abc.se <mailto:m7527@abc.se>>> wrote: > > On 2013-10-14 22:01, Chris Whittleston wrote: > > I see - so I'll probably still see the problem with a huge number of > > circuits being created after I've finished building 0.2.4. Is there > > any way to limit this, I'm guessing reducing the bandwidth wouldn't > > actually help? I guess I'll look into how much further I can > overclock > > the CPU... > Only option that I know of is to reduce the bandwidth you advertise to > the network. The more bandwidth you advertise the more circuits > the tor > network will throw at your relay. The following flags in the torrc > file > can be used (with my current understanding of them): > BandwidthRate : The max bandwidth you provide over a long period > of time > BandwidthBurst : The max bandwidth you provide over a short period > of time > MaxAdvertisedBandwidth : The max bandwidth you tell the tor > network about > So you can set BandwidthRate to the real max you want to provide and > then set MaxAdvertisedBandwidth to a number low enough to prevent > circuit overload. > > _______________________________________________ > tor-relays mailing list > tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> > <mailto:tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org>> > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays > > > > > -- > *Dr Chris Whittleston 栗主* > Department of Chemistry > University of Cambridge > Lensfield Road, Cambridge, CB2 1EW > Email: csw34@cam.ac.uk <mailto:csw34@cam.ac.uk> <mailto:csw34@cam.ac.uk <mailto:csw34@cam.ac.uk>> > Tel: +44 (0)1223 336423 <tel:%2B44%20%280%291223%20336423> > > > _______________________________________________ > tor-relays mailing list > tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays -- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9 _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9
I run a few Rasp PI relays. If you use Tor 2.3.5 you need to limit bandwidth to 100KB or less and make sure the CPU is not overclocked to avoid heating issues unless you stick it in the fridge or something.
2.4.17 is much better at dealing with circuit storms, i can push 3Mbps upstream with no issues.
try setting up cron if it keeps crashing. eg.
*/15 * * * * /etc/init.d/tor start
On 14 October 2013 23:42, Dan Staples danstaples@disman.tl wrote:
I don't think it would make a difference. When my relay is experiencing a circuit-creation storm, I've restarted Tor, and even stopped it for a few minutes, but as soon as it comes back online, it gets slammed again. Only leaving it offline for an hour or so and then turning it back on seems to mitigate the problem.
On Mon 14 Oct 2013 06:11:10 PM EDT, Chris Whittleston wrote:
Do you think it might help to restart tor every 24 hours or so using cron Dan - or would that adversely affect the network too much/not actually help?
On 14 Oct 2013 22:32, "Dan Staples" <danstaples@disman.tl mailto:danstaples@disman.tl> wrote:
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests. It *will* affect the *average* bandwidth used by your relay, but every once ina
while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still anunresolved
issue. On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote: > Thanks Logforme - yeah I was trying that before I sent the first email > in this chain, but maybe I didn't go low enough with the advertised > bandwidth. When the 0.2.4 compilation is done (it's still chugging > along) I'll try going lower and see if it helps. > > Chris > > > On 14 October 2013 21:38, Logforme <m7527@abc.se <mailto:m7527@abc.se> > <mailto:m7527@abc.se <mailto:m7527@abc.se>>> wrote: > > On 2013-10-14 22:01, Chris Whittleston wrote: > > I see - so I'll probably still see the problem with a huge number of > > circuits being created after I've finished building 0.2.4. Is there > > any way to limit this, I'm guessing reducing the bandwidth wouldn't > > actually help? I guess I'll look into how much further I can > overclock > > the CPU... > Only option that I know of is to reduce the bandwidth you advertise to > the network. The more bandwidth you advertise the more circuits > the tor > network will throw at your relay. The following flags in the torrc > file > can be used (with my current understanding of them): > BandwidthRate : The max bandwidth you provide over a longperiod
> of time > BandwidthBurst : The max bandwidth you provide over a short period > of time > MaxAdvertisedBandwidth : The max bandwidth you tell the tor > network about > So you can set BandwidthRate to the real max you want to provide and > then set MaxAdvertisedBandwidth to a number low enough to prevent > circuit overload. > > _______________________________________________ > tor-relays mailing list > tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> > <mailto:tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org>> >https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
> > > > > -- > *Dr Chris Whittleston 栗主* > Department of Chemistry > University of Cambridge > Lensfield Road, Cambridge, CB2 1EW > Email: csw34@cam.ac.uk <mailto:csw34@cam.ac.uk> <mailto:csw34@cam.ac.uk <mailto:csw34@cam.ac.uk>> > Tel: +44 (0)1223 336423 <tel:%2B44%20%280%291223%20336423> > > > _______________________________________________ > tor-relays mailing list > tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays -- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9 _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org <mailto:tor-relays@lists.torproject.org> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9 _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Chris Whittleston:
Do you think it might help to restart tor every 24 hours or so using cron Dan - or would that adversely affect the network too much/not actually help?
Generally restarting a Tor relay is something you want to do as little as possible. I'm not sure if a quick graceful restart will ruin your Stable flag, but if you do have a Stable flag, you're killing every circuit through you when you restart.
So, try to keep tor up 24/7 rather than restarting it a lot.
Best, - -Gordon M.
On 14 Oct 2013 22:32, "Dan Staples" danstaples@disman.tl wrote:
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests. It *will* affect the *average* bandwidth used by your relay, but every once in a while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still an unresolved issue.
On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote:
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme <m7527@abc.se mailto:m7527@abc.se> wrote:
On 2013-10-14 22:01, Chris Whittleston wrote:
I see - so I'll probably still see the problem with a huge number
of
circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can
overclock
the CPU...
Only option that I know of is to reduce the bandwidth you advertise
to
the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload.
Thanks for the replies guys - I've not has a storm in the last few days, but if/when my Pi gets knocked offline again, I'll dig into those mitigation strategies. Think in the meantime I need to do a bit more reading about how the network maintains stability etc
Best,
Chris
On 20 October 2013 19:02, Gordon Morehouse gordon@morehouse.me wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Chris Whittleston:
Do you think it might help to restart tor every 24 hours or so using cron Dan - or would that adversely affect the network too much/not actually help?
Generally restarting a Tor relay is something you want to do as little as possible. I'm not sure if a quick graceful restart will ruin your Stable flag, but if you do have a Stable flag, you're killing every circuit through you when you restart.
So, try to keep tor up 24/7 rather than restarting it a lot.
Best,
- -Gordon M.
On 14 Oct 2013 22:32, "Dan Staples" danstaples@disman.tl wrote:
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests. It *will* affect the *average* bandwidth used by your relay, but every once in a while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still an unresolved issue.
On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote:
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme <m7527@abc.se mailto:m7527@abc.se> wrote:
On 2013-10-14 22:01, Chris Whittleston wrote:
I see - so I'll probably still see the problem with a huge number
of
circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can
overclock
the CPU...
Only option that I know of is to reduce the bandwidth you advertise
to
the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload.
-----BEGIN PGP SIGNATURE-----
iQEcBAEBCgAGBQJSZBrOAAoJED/jpRoe7/ujnf0H/i+LnIirKcAaceALJOuBasQX LczVJiuIG027mqEA6xid6lkiMMVyhIbYbLCL965RJiVm/P8OYfb6woxxUCaOG2s4 N+pzFDZpg5toZOYgp378oq84GDYpvXdeTxTwx+itATsoGBPg28bYA3YTXGfmTiJr /K+cn7j+0QlJsJEgv2taTcnHVgpm4/pm0cfji7/Gg2sGJTuQmRH/V1QMy95fdLUR 9dklGpCHEFNOWcDR+MGRTqrks3qG3iMvxuw0HgQ6l5wJSGi1g1ovV3yI0JZNJKQq vBAHIaZ+yqUHkGux0cd1FxUe+HOVbLfuKFFBNTuuu2riXdboMyI65aepezRqSQU= =h+np -----END PGP SIGNATURE----- _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Dan Staples:
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests.
This is absolutely correct, and I've been working on this problem with the Pi for months now.
It *will* affect the *average* bandwidth used by your relay, but every once in a while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still an unresolved issue.
Correct.
I have an ongoing project to support Tor on small single-board computers, including the Raspberry Pi.[1]
In the contrib/90_slowboards dir, I've included some iptables rules that drastically reduce, but *do not eliminate*, Tor's vulnerability to running out of physical RAM and being killed during a SYN flood/circuit creation storm.
I've also included some fail2ban[2] rules that ban hosts for short periods of time when they ignore the iptables SYN throttling. However (and I'll push a commit in the next hour changing this), I'd suggest you use a "findtime" of 60 and "bantime" of 90, not 300 and 600 as I first pushed to the repo. Please see this post[3] for more technical talk/questions, which I *just* made before reading yours and writing this reply. Please be aware that the fail2ban fallback is much less battle-tested (on my relay) than the iptables SYN throttling.
Incidentally, I host binary .debs compiled *for Raspbian* at [1] as well, if you trust random internet dudes and don't want to compile it yourself. A deterministic build is a (very long-term) goal.
1. https://github.com/gordon-morehouse/cipollini
2. http://www.fail2ban.org/wiki/index.php/Main_Page
3. https://lists.torproject.org/pipermail/tor-relays/2013-October/003074.html
Best, - -Gordon M.
On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote:
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme <m7527@abc.se mailto:m7527@abc.se> wrote:
On 2013-10-14 22:01, Chris Whittleston wrote:
I see - so I'll probably still see the problem with a huge number of circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can
overclock
the CPU...
Only option that I know of is to reduce the bandwidth you advertise to the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload.
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org mailto:tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- *Dr Chris Whittleston 栗主* Department of Chemistry University of Cambridge Lensfield Road, Cambridge, CB2 1EW Email: csw34@cam.ac.uk mailto:csw34@cam.ac.uk Tel: +44 (0)1223 336423
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9 _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Thanks Gordon, I've been following your posts about the circuit-creation storms with interest. I recently upgraded my Pi to Tor v0.2.4, and haven't witnessed a storm yet (they are relatively rare for me). But I am certainly interested in trying out your mitigation strategies.
Regarding your deb packages, have you considered talking with any Debian or Raspbian devs? Getting them to sign your public key, and possibly even accepting your packages upstream, would create a web of trust so folks would be more willing to install what you build.
On 10/20/2013 12:55 PM, Gordon Morehouse wrote:
Dan Staples:
In my experience, setting the bandwidth advertising options does nothing to stop the "storms" of circuit creation requests.
This is absolutely correct, and I've been working on this problem with the Pi for months now.
It *will* affect the *average* bandwidth used by your relay, but every once in a while, I'll still get circuit-creation storms that completely overwhelm my RPi and knock it offline (I'm talking continuous 3Mbps bandwidth use for several hours when MaxAdvertisedBandwidth is 200 kbps). It seems from past discussions on the mailing list, this is still an unresolved issue.
Correct.
I have an ongoing project to support Tor on small single-board computers, including the Raspberry Pi.[1]
In the contrib/90_slowboards dir, I've included some iptables rules that drastically reduce, but *do not eliminate*, Tor's vulnerability to running out of physical RAM and being killed during a SYN flood/circuit creation storm.
I've also included some fail2ban[2] rules that ban hosts for short periods of time when they ignore the iptables SYN throttling. However (and I'll push a commit in the next hour changing this), I'd suggest you use a "findtime" of 60 and "bantime" of 90, not 300 and 600 as I first pushed to the repo. Please see this post[3] for more technical talk/questions, which I *just* made before reading yours and writing this reply. Please be aware that the fail2ban fallback is much less battle-tested (on my relay) than the iptables SYN throttling.
Incidentally, I host binary .debs compiled *for Raspbian* at [1] as well, if you trust random internet dudes and don't want to compile it yourself. A deterministic build is a (very long-term) goal.
https://lists.torproject.org/pipermail/tor-relays/2013-October/003074.html
Best, -Gordon M.
On Mon 14 Oct 2013 04:43:50 PM EDT, Chris Whittleston wrote:
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme <m7527@abc.se mailto:m7527@abc.se> wrote:
On 2013-10-14 22:01, Chris Whittleston wrote:
I see - so I'll probably still see the problem with a huge number of circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can
overclock
the CPU...
Only option that I know of is to reduce the bandwidth you advertise to the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload.
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org mailto:tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
- -- *Dr Chris Whittleston 栗主* Department of Chemistry University
of Cambridge Lensfield Road, Cambridge, CB2 1EW Email: csw34@cam.ac.uk mailto:csw34@cam.ac.uk Tel: +44 (0)1223 336423
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9 _______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
_______________________________________________ tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
- -- http://disman.tl OpenPGP key: http://disman.tl/pgp.asc Fingerprint: 2480 095D 4B16 436F 35AB 7305 F670 74ED BD86 43A9
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Dan Staples:
Thanks Gordon, I've been following your posts about the circuit-creation storms with interest. I recently upgraded my Pi to Tor v0.2.4, and haven't witnessed a storm yet (they are relatively rare for me). But I am certainly interested in trying out your mitigation strategies.
They certainly got rarer (or stopped rising to the level of being noticeable in logs so often) with the upgrade to 0.2.4.17-rc.
Regarding your deb packages, have you considered talking with any Debian or Raspbian devs? Getting them to sign your public key, and possibly even accepting your packages upstream, would create a web of trust so folks would be more willing to install what you build.
It's on the list[1]. :)
1. https://www.pivotaltracker.com/s/projects/917796
Best, - -Gordon M.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Am 20.10.2013 17:25, schrieb Dan Staples:
I recently upgraded my Pi to Tor v0.2.4,
How did you do that? Is there some deb package for the Pi available?
- -- Mit besten Grüßen Jochen Fahrner
Gordon has pre-built packages[1], but I just downloaded the source and built it myself: https://www.torproject.org/download/download.html.en (unstable/alpha version).
-Dan
[1] https://github.com/gordon-morehouse/cipollini/tree/master/raspbian_packages
On 10/20/2013 01:49 PM, Jochen Fahrner wrote:
Am 20.10.2013 17:25, schrieb Dan Staples:
I recently upgraded my Pi to Tor v0.2.4,
How did you do that? Is there some deb package for the Pi available?
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Ok - I have 0.2.4.17-rc built and running on the Pi - I'll give it 24-48 hours to see if the same problem with sudden increases in the number of circuits happens again, then try the MaxAdvertisedBandwidth setting in case it helps.
Will drop feedback here in a couple of days - thanks for the help all!
Chris
On 14 October 2013 21:43, Chris Whittleston csw34@cam.ac.uk wrote:
Thanks Logforme - yeah I was trying that before I sent the first email in this chain, but maybe I didn't go low enough with the advertised bandwidth. When the 0.2.4 compilation is done (it's still chugging along) I'll try going lower and see if it helps.
Chris
On 14 October 2013 21:38, Logforme m7527@abc.se wrote:
On 2013-10-14 22:01, Chris Whittleston wrote:
I see - so I'll probably still see the problem with a huge number of circuits being created after I've finished building 0.2.4. Is there any way to limit this, I'm guessing reducing the bandwidth wouldn't actually help? I guess I'll look into how much further I can overclock the CPU...
Only option that I know of is to reduce the bandwidth you advertise to the network. The more bandwidth you advertise the more circuits the tor network will throw at your relay. The following flags in the torrc file can be used (with my current understanding of them): BandwidthRate : The max bandwidth you provide over a long period of time BandwidthBurst : The max bandwidth you provide over a short period of time MaxAdvertisedBandwidth : The max bandwidth you tell the tor network about So you can set BandwidthRate to the real max you want to provide and then set MaxAdvertisedBandwidth to a number low enough to prevent circuit overload.
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-- *Dr Chris Whittleston 栗主* Department of Chemistry University of Cambridge Lensfield Road, Cambridge, CB2 1EW Email: csw34@cam.ac.uk Tel: +44 (0)1223 336423
On Mon, Oct 14, 2013 at 08:36:58PM +0100, Chris Whittleston wrote:
Aha - makes sense, I'll just build it myself. Thanks for the quick response.
So - the new handshake in 0.2.4.x doesn't help with the ongoing issues? On that page I linked it was suggested it might...
It would help except that the vast numbers of 0.2.3 clients out there still use the older TAP creates. I'm still seeing roughly 130 TAP handshakes for every 1 NTor.
-t
tor-relays@lists.torproject.org
-
Chris Whittleston -
Chris Whittleston -
Dan Staples -
Gordon Morehouse -
Jochen Fahrner -
Logforme -
Roger Dingledine -
Thomas Hand -
Troy Arnold