Hi Folks,
im running many relays which are doing a nice job. But i have also some smaller relays outside which have just 256mb. They are performing quite quell - but on one relay im facing issues which i think you may help me.
this relay seems to ran out of memory from time to time (2 to 5 days) and i found out via dmesg, that ubuntu itself is killing the process to free up memory. the swap is used, but far away from full.
as this stops tor immediately its bit annoying that i need to restart the releay without any reason.
is there any way to tell ubuntu or to prevent that the tor process is killed for freeing up memory?
i dont care about other processes as this machine is for tor purposes only.
thanks for your expertise! Geri
Hi Geri!
You may adjust the Linux OOM killer's settings on a per process basis with the proc fs; see here: http://askubuntu.com/questions/60672/how-do-i-use-oom-score-adj
If you have multiple numa cores then it also might be helpful to set the process to use numa interleaved memory instead of just it's local numa memory bank... see numactl for more information about that. Jeremy Cole wrote an interesting article about tuning mysql with numactl a while back.
Some people advise keeping a sacrificial lamb process that gets oom-killed first.
Also I always like to disable swap completely (e.g. swapoff -a)... but sometimes I meet sys admins that try to argue that having some swap is a good idea; i am not convinced. swap is so 1992.
I didn't explain everything or go into all the details here... so feel free to ask me questions if you have some problems or if something is unclear.
Cheers!
David
On Sat, Feb 1, 2014 at 12:41 PM, toxi roxi toxiroxi@gmail.com wrote:
Hi Folks,
im running many relays which are doing a nice job. But i have also some smaller relays outside which have just 256mb. They are performing quite quell - but on one relay im facing issues which i think you may help me.
this relay seems to ran out of memory from time to time (2 to 5 days) and i found out via dmesg, that ubuntu itself is killing the process to free up memory. the swap is used, but far away from full.
as this stops tor immediately its bit annoying that i need to restart the releay without any reason.
is there any way to tell ubuntu or to prevent that the tor process is killed for freeing up memory?
i dont care about other processes as this machine is for tor purposes only.
thanks for your expertise! Geri
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi David,
thank you very much for your hints - i got now a better picture on how this could work on my ubuntu relay. Very interesting infos - thanks! I've setup some tweaks now - lets see how it works. If you are interested what i have did and if it helped - just let me know - i will drop you a message in a week or two.
Thanks! Geri
2014-02-01 David Stainton dstainton415@gmail.com:
Hi Geri!
You may adjust the Linux OOM killer's settings on a per process basis with the proc fs; see here: http://askubuntu.com/questions/60672/how-do-i-use-oom-score-adj
If you have multiple numa cores then it also might be helpful to set the process to use numa interleaved memory instead of just it's local numa memory bank... see numactl for more information about that. Jeremy Cole wrote an interesting article about tuning mysql with numactl a while back.
Some people advise keeping a sacrificial lamb process that gets oom-killed first.
Also I always like to disable swap completely (e.g. swapoff -a)... but sometimes I meet sys admins that try to argue that having some swap is a good idea; i am not convinced. swap is so 1992.
I didn't explain everything or go into all the details here... so feel free to ask me questions if you have some problems or if something is unclear.
Cheers!
David
On Sat, Feb 1, 2014 at 12:41 PM, toxi roxi toxiroxi@gmail.com wrote:
Hi Folks,
im running many relays which are doing a nice job. But i have also some smaller relays outside which have just 256mb. They are performing quite quell - but on one relay im facing issues
which i
think you may help me.
this relay seems to ran out of memory from time to time (2 to 5 days)
and i
found out via dmesg, that ubuntu itself is killing the process to free up memory. the swap is used, but far away from full.
as this stops tor immediately its bit annoying that i need to restart the releay without any reason.
is there any way to tell ubuntu or to prevent that the tor process is
killed
for freeing up memory?
i dont care about other processes as this machine is for tor purposes
only.
thanks for your expertise! Geri
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
Hi Geri,
Oh good! Glad to hear you are experimenting.
It's nice to have some time series graphs (i can't recommend it but the kids these days use a heavy weight time series graphing system called graphite) displaying various memory related properties of your server such as those values found in /proc/meminfo You can read about what these fields mean in: linux/Documentation/filesystems/proc.txt
Often times it is possible to look at a stacked time series graph with all these properties... and it tells you a story about what is going on. Then you can make more informed decisions with regards to fixing memory related problems.
Yes... I (and probably others on the tor-relays list) would be interested in hearing from you if you figure out a solution to your Linux memory management problems.
Cheers!
David
On Sun, Feb 2, 2014 at 2:17 PM, toxi roxi toxiroxi@gmail.com wrote:
Hi David,
thank you very much for your hints - i got now a better picture on how this could work on my ubuntu relay. Very interesting infos - thanks! I've setup some tweaks now - lets see how it works. If you are interested what i have did and if it helped - just let me know - i will drop you a message in a week or two.
Thanks! Geri
2014-02-01 David Stainton dstainton415@gmail.com:
Hi Geri!
You may adjust the Linux OOM killer's settings on a per process basis with the proc fs; see here: http://askubuntu.com/questions/60672/how-do-i-use-oom-score-adj
If you have multiple numa cores then it also might be helpful to set the process to use numa interleaved memory instead of just it's local numa memory bank... see numactl for more information about that. Jeremy Cole wrote an interesting article about tuning mysql with numactl a while back.
Some people advise keeping a sacrificial lamb process that gets oom-killed first.
Also I always like to disable swap completely (e.g. swapoff -a)... but sometimes I meet sys admins that try to argue that having some swap is a good idea; i am not convinced. swap is so 1992.
I didn't explain everything or go into all the details here... so feel free to ask me questions if you have some problems or if something is unclear.
Cheers!
David
On Sat, Feb 1, 2014 at 12:41 PM, toxi roxi toxiroxi@gmail.com wrote:
Hi Folks,
im running many relays which are doing a nice job. But i have also some smaller relays outside which have just 256mb. They are performing quite quell - but on one relay im facing issues which i think you may help me.
this relay seems to ran out of memory from time to time (2 to 5 days) and i found out via dmesg, that ubuntu itself is killing the process to free up memory. the swap is used, but far away from full.
as this stops tor immediately its bit annoying that i need to restart the releay without any reason.
is there any way to tell ubuntu or to prevent that the tor process is killed for freeing up memory?
i dont care about other processes as this machine is for tor purposes only.
thanks for your expertise! Geri
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays mailing list tor-relays@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
tor-relays@lists.torproject.org
-
David Stainton -
toxi roxi