The problem got resolved, once I closely analyzed the tor INFO logs.
Main problem was that the exit nodes in the circuit were refusing connections due to their exit policies, because our application ran on some random ports.
Running out application on tor exit supported ports worked for a while.

But a new problem arised once the above was resolved i.e. tor was dropping connections again giving the reason to be "We tried for 10 seconds to connect to <server_ip> using exit <exit-node-fingerprint>~<exit-node-nick> at <exit-node-ip>. Retrying on a new circuit." 

I've been trying to analyse the INFO level logs, but haven't been able to find more information as why this error is happening. 

Are there any possible reasons as to why this might be happening and any possible fixes for this situation?

On Wed, 20 Feb 2019, 18:08 Piyush Kumar Sharma, <piyushs@iiitd.ac.in> wrote:
After a lot of debugging, it seems that the reason for streams failing is as follows :
As soon as a new stream is made, it goes into the NEW state according to Torctl logs.
Then with stem or carml running, they try to attach it to the specified circuit.
As soon as the stream is attached, it moves to the SENTCONNECT, and if it doesn't succeed, the state of that stream again changes to NEW after which the connection fails (the stream waits for the 120s then gets closed automatically with error code REASON_TIMEOUT).
A point i noted was that, a stream is in the SENTCONNECT state for a very short duration (fraction of seconds).

Other times, when the above process does not happen, the streams successfully move to the SUCCEEDED state after SENTCONNECT and are served through the specified circuit.
But this behavior seems to be completely random. Is there a way to resolve this issue?

@meejah : carml had the same problem as with stem as already explained above.
@Nick Mathewson : Yes there are some instances where the circuit creation itself takes a lot of time and fails sometimes. But those cases are already handled. The main problem as we could drill down is explained above.


Regards
Piyush



On Wed, Feb 20, 2019 at 7:40 AM Nick Mathewson <nickm@alum.mit.edu> wrote:
On Mon, Feb 18, 2019 at 1:04 PM Piyush Kumar Sharma <piyushs@iiitd.ac.in> wrote:
>
> Hello all,
>
> I am a PhD student, and am working on some measurements in Tor.
> I am stuck at a point where i need to send multiple applications(streams) traffic through a single circuit.
> I am currently using torsocks/torify to send traffic of these multiple applications through Tor.
> The main problem is that, despite trying many different ways to achieve the same (sending multiple streams through a single circuit), i am not successful.
>
> Things i have tried :
> 1.) Force Tor process to create only a single circuit at a time preventing any new circuit creation, so that any new stream would be attached to this only available circuit. To acheive this i have set the following Tor options :
> set __DisablePredictedCircuits to 1
> set MaxClientCircuitsPending to 1
> set newcircuitperiod to 999999999
> set maxcircuitdirtiness to 999999999
> The problem with the above method is that it seems to work sometimes randomly. But most of the times for some reason, a new circuit is still created.
>
> 2.) Next, i assumed that maybe running torify multiple times for each application is the culprit, as it may try to create new circuit for each run. So i created a new bidirectional stream using socat, which listens on a local TCP port, and forwards the data to the Tor SOCKS port assuming that it will lead to a singe connection to local SOCKS.
> Even this did not work and still new circuits were created randomly.
>
> 3.) Next i tried to attach streams to circuits manually, using the stem library following the link : https://stem.torproject.org/tutorials/to_russia_with_love.html#custom-path-selection . This seemed to work initially, but then after every 4-5 runs, the streams seemed to detach automatically. Moreover, the original circuit crashed too.

The stem approach (#3) ought to work in general -- the information
here isn't enough to tell what the problem is, exactly. Is it possible
that the circuit you are constructing is failing for some reason?

--
Nick
_______________________________________________
tor-dev mailing list
tor-dev@lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev