On 7/6/12 10:34 AM, Sathyanarayanan Gunasekaran wrote:>> OK; Megan and Erik, after you incorporate the export function into
Descriptor in stem, please start reading through the Django tutorial.
I'm not sure if Django is a good choice for this project. We don't require such a heavy web framework with a templating engine, auth, etc. I'd rather use Tornado or Cyclone+Twisted. Since tor2web, APAF use Twisted, it would make sense to have onionoo use the same thing.
I've been hacking on this since morning and I have a very simple prototype which uses Cyclone that parses only the summary documents and provides an API for that("/summary") - I will clean it up and put on github in a bit
Is this available yet?
We'll start working out milestones on Monday (I'm away until Monday
morning,
but I'll probably have occasional e-mail access); Sathyanarayanan, you should probably chime in too.
So we need to decide on both some milestones and also what various people ought to work on so we avoid stepping on each other's toes.
It seems like a natural way to keep some of our work disjoint is to have us primarily work on the back end and Sathyanarayanan on the front end, or vice-versa. Sathyanarayanan, does this sound reasonable to you? Do you have a preference? It sounds like you're already working on the front end, though I was hoping we'd have a chance to play with and learn twisted ourselves a bit.
- Norman
Is this available yet?
Yep - https://github.com/gsathya/pyonionoo It's pretty hacky(it was meant to be a prototype to see if Cyclone was a good idea - and well, i like it) and will probably have to be refactored.
So we need to decide on both some milestones and also what various people ought to work on so we avoid stepping on each other's toes.
It seems like a natural way to keep some of our work disjoint is to have us primarily work on the back end and Sathyanarayanan on the front end, or vice-versa. Sathyanarayanan, does this sound reasonable to you? Do you have a preference? It sounds like you're already working on the front end, though I was hoping we'd have a chance to play with and learn twisted ourselves a bit.
Either way works fine with me.
On 7/9/12 12:09 PM, Sathyanarayanan Gunasekaran wrote:
Is this available yet?
Yep - https://github.com/gsathya/pyonionoo It's pretty hacky(it was meant to be a prototype to see if Cyclone was a good idea - and well, i like it) and will probably have to be refactored.
Based on a quick look, it seems like Cyclone provides a slightly nicer way to specify how to handle the various requests than does a plain Twisted web application. Are there any other advantages to using Cyclone as opposed to plain Twisted?
To me, there is a trade-off:
Cyclone+Twisted: slightly nicer way to write the web application. More dependencies for Onionoo.
Plain Twisted: not quite as nice a way to write the application. Fewer dependencies for Onionoo.
Also, the documentation for Cyclone seems...minimal. It might be straightforward for someone used to using Tornado, but that doesn't describe us over here...
- Norman
On 7/10/12 5:36 PM, Norman Danner wrote:
On 7/9/12 12:09 PM, Sathyanarayanan Gunasekaran wrote:
Is this available yet?
Yep - https://github.com/gsathya/pyonionoo It's pretty hacky(it was meant to be a prototype to see if Cyclone was a good idea - and well, i like it) and will probably have to be refactored.
Based on a quick look, it seems like Cyclone provides a slightly nicer way to specify how to handle the various requests than does a plain Twisted web application. Are there any other advantages to using Cyclone as opposed to plain Twisted?
On APAF (anonymous python application framework) GSoc project, to build cross-platform Windows, OSX, Linux applications including Tor there is the use of Twisted+Cyclone.
If you use such kind of framework, it should be relatively easy to hook them to APAF, to build your application and deliver it also for Windows and OSX.
-naif
On APAF (anonymous python application framework) GSoc project, to build cross-platform Windows, OSX, Linux applications including Tor there is the use of Twisted+Cyclone.
I'm still not clear though - what is the advantage of providing Onionoo as a hidden service? We shouldn't add dependencies, especially on a new project like APAF, without clear and substantial benefits.
On 7/10/12 6:18 PM, Damian Johnson wrote:
On APAF (anonymous python application framework) GSoc project, to build cross-platform Windows, OSX, Linux applications including Tor there is the use of Twisted+Cyclone.
I'm still not clear though - what is the advantage of providing Onionoo as a hidden service? We shouldn't add dependencies, especially on a new project like APAF, without clear and substantial benefits.
The Tor Hidden Service support of APAF is just a "small part" of what it does provide.
At first it does provide you an environment to build a Desktop Applications, with an easy-build-system integrating Tor and nicely looking UI elements for Windows and OSX, including an embedded browser.
So basically if you would like to make a desktop application including Onionoo & Atlas together to provide an end-user an application that he can download to query/search/analyze the Consensus, APAF it's a good choice. If you just need to do it for a single application that does not need to be installed and/or distributed hundreds/thousands of time, then the APAF framework may not be the right choice.
-naif
So basically if you would like to make a desktop application including Onionoo & Atlas together to provide an end-user an application that he can download to query/search/analyze the Consensus, APAF it's a good choice. If you just need to do it for a single application that does not need to be installed and/or distributed hundreds/thousands of time, then the APAF framework may not be the right choice.
I don't think APAF is a good choice for this. It's just a web service that's going to run one of the Tor VM's. We're not going to bundle this with Atlas as a desktop application.
Based on a quick look, it seems like Cyclone provides a slightly nicer way to specify how to handle the various requests than does a plain Twisted web application. Are there any other advantages to using Cyclone as opposed to plain Twisted?
Cyclone provides a much nicer API than Twisted with routing done right. I haven't played around with Twisted much, but I'm liking how Cyclone works. Cyclone does a bunch of stuff like converting a dict to a json and setting the proper response headers automagically(which was a "Aha" moment for me, I'm not sure if Twisted does all this).
Cyclone+Twisted: slightly nicer way to write the web application. More dependencies for Onionoo.
Plain Twisted: not quite as nice a way to write the application. Fewer dependencies for Onionoo.
I'd definitely prefer Cyclone+Twisted. Cyclone ships with a couple of Debian scripts that make it easier to run as well.
Also, the documentation for Cyclone seems...minimal. It might be straightforward for someone used to using Tornado, but that doesn't describe us over here...
Well, that's true. There isn't much documentation, but it uses almost the same API as Tornado so you can pretty much use Tornado's docs. But that isn't much either so I kinda ended up reading Cyclone's source and that worked. It's actually pretty easy to follow. But, I can see how this can become a real pain.
-Sathya
Norman Danner ndanner@wesleyan.edu writes:
Based on a quick look, it seems like Cyclone provides a slightly nicer way to specify how to handle the various requests than does a plain Twisted web application. Are there any other advantages to using Cyclone as opposed to plain Twisted?
From what I understand of the two systems, Twisted's "web stuff" isn't
really intended as a "web application framework" (yet?) and you'd probably want Nevow or Twisted.web2 for that anyway. Cyclone and Tornado, OTOH, *are* Web frameworks. The advantage of Cyclone being that it's based on the core Twisted async "stuff" rather than Tornado's homebrew (and hence supports more things, has tests, cross-platform support, etcetera).
Using Cyclone would also allow you to use txtorcon, as it's based on Twisted too :)
I agree, Twisted is not a web application framework. So the question is this: do we want/need a web application framework for Onionoo? It seems like it is such a straightforward service (process one of a very few different request types, send back a JSON response) that maybe any web application framework is overkill.
According to Sathya, Cyclone does have some functionality for converting dicts to JSON (of course, so does the json standard library module) and sets response headers properly automatically. So that also makes some of the coding easier and possibly more maintainable. The latter is definitely not to be sneezed at; I just don't have enough experience with either Twisted or Cyclone to know whether it makes up for the additional dependency.
- Norman
On 7/10/12 3:04 PM, meejah@meejah.ca wrote:
Norman Danner ndanner@wesleyan.edu writes:
Based on a quick look, it seems like Cyclone provides a slightly nicer way to specify how to handle the various requests than does a plain Twisted web application. Are there any other advantages to using Cyclone as opposed to plain Twisted?
From what I understand of the two systems, Twisted's "web stuff" isn't really intended as a "web application framework" (yet?) and you'd probably want Nevow or Twisted.web2 for that anyway. Cyclone and Tornado, OTOH, *are* Web frameworks. The advantage of Cyclone being that it's based on the core Twisted async "stuff" rather than Tornado's homebrew (and hence supports more things, has tests, cross-platform support, etcetera).
Using Cyclone would also allow you to use txtorcon, as it's based on Twisted too :)
On 07/10/2012 05:36 PM, Norman Danner wrote:
Based on a quick look, it seems like Cyclone provides a slightly nicer way to specify how to handle the various requests than does a plain Twisted web application. Are there any other advantages to using Cyclone as opposed to plain Twisted?
To me, there is a trade-off:
Cyclone+Twisted: slightly nicer way to write the web application. More dependencies for Onionoo.
Plain Twisted: not quite as nice a way to write the application. Fewer dependencies for Onionoo.
Maintaining a REST API in pure twisted is *very* painful. The syntax for defining the API in twisted is very hackish and unclean. Using cyclone makes your code more structured and cleaner. The hackiness is very clear if you are interested in doing things like matching to a certain functionality based on a regexp. Cyclone makes this easy as it's based on tornado that was designed with the creation of an API for friendfeed in mind.
Also, the documentation for Cyclone seems...minimal. It might be straightforward for someone used to using Tornado, but that doesn't describe us over here...
The documentation of cyclone is the documentation of tornado. Cyclone is basically just a fork of it so anything that you are used to doing in tornado you can do in cyclone.
- Art.
-
Arturo Filastò -
Damian Johnson -
Fabio Pietrosanti (naif) -
meejah＠meejah.ca -
Norman Danner -
Sathyanarayanan Gunasekaran