On 8/29/11 10:48 AM, Karsten Loesing wrote:
The new deadline is
Sunday, September 4, 2011
Dear Tor developers and volunteers who haven't responded yet: If you have an opinion on using Trac, please let us know!
First of all, thanks to the 13 people for taking part in the survey!
In this mail I'm trying to summarize the results from part 1 of the survey and derive some suggestions for making Trac better. (I'm leaving out the results from part 2 for now, because it turned out the questions weren't as useful as I thought. I may use the replies to design a second survey in a month or two from now, but for the moment I think it's best to just focus on our Trac usage.)
Below are the results, plus some suggestions in the paragraphs starting with "[Suggestion: " to separate my suggestions from result summaries.
Feel free to comment on the suggestions or add your own!
Depending on whether this thread is going to explode, I'll write another summary in a few days to list the suggestions that people seem to agree on and that we're going to implement.
Best, Karsten
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
From all the replies, there's 1 person using 5 reports, 2 persons using
3 reports, 1 person using 2 reports, 4 persons using 1 report, and 5 persons using no reports at all. There's only 1 report being viewed by more than one person. 5 persons are not using reports at all.
The following reports are viewed: 7, 8, 12 (mentioned twice), 14, 22, 23 (mentioned twice), 27, 28, 34, 35, 36, 38, 39, 40 (mentioned twice). Hence, the following reports are not viewed: 1, 2, 3, 4, 5, 6, 10, 11, 15, 16, 17, 18, 19, 20, 21, 24, 29, 30, 31, 32, 33, 37.
[Suggestion: Backup and delete all reports with a component name in them. The current list of Available Reports list is mostly useless for newcomers who don't care much about components, but who are interested in finding something to work on. Developers and volunteers can bookmark custom queries or put links to them on a wiki page belonging to a component. For example, report 12 "Tor: Active Tickets by Milestone" is https://trac.torproject.org/projects/tor/query?status=!closed&group=mile... ]
1.2 What are typical custom queries that you run?
There were 11 replies to this question. 4 persons start from a small set of queries that are always the same, similar to stored ticket queries, and refine them as needed. 4 persons run many different queries. 4 persons don't use the custom query feature at all.
1.3 How do you use milestones and roadmaps?
7 out of 11 people use milestones. The components for which milestones are used are Tor (4 mentions), Vidalia (1 mention), TBB (1 mention), and sponsors deadlines (1 mention). 4 persons use wiki pages or parent tickets instead of or in addition to milestones to classify when tickets should be completed. Nobody uses the roadmap view of milestones.
Unrelated to this survey, here are some statistics on the milestones in our Trac: We have 224 milestones in total, 37 of which having at least 1 ticket assigned. That leaves 187 milestones with zero tickets assigned, most of which are a left-over from the flyspray migration.
[Suggestion: Delete all milestones that have no tickets assigned to them or that seem useful even without assigned tickets.]
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
11 persons are subscribed to tor-bugs, but most people filter the mails to only learn about components they care about. 5 persons read tor-wiki-changes, but at least 3 persons would be glad if they contained diffs.
[Suggestion: Try harder to include diffs in tor-wiki-changes notifications. Maybe Trac 0.12 will make this easier.]
1.5 Which wiki pages do you read/edit most often?
8 out of 11 persons replying to this question use the wiki. Mentioned wiki pages are (multiple selections allowed): sponsor pages (5 mentions), specific pages in doc/ (3 mentions), various FAQ (2 mentions), dev meeting pages (2 mentions), roadmaps in org/roadmaps (1 mention).
1.6 How do you search for wiki pages?
People use at least four methods to search the wiki (multiple selections allowed): TitleIndex (5 mentions), external search engine (4 mentions), browser auto-completion or history (3 mentions), Trac search (3 mentions).
1.7 What are typical search terms that you use when using the search features?
3 persons search Trac using key words and 1 person types in ticket numbers in the search field. The rest doesn't use the search feature.
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
1 person uses Agile iteration tracking keywords (D'oh, there goes the anonymity!) and 2 persons use the "easy" keyword sometimes. At least 1 person was surprised that there's a tag page. The rest doesn't use keywords, nor the tags page.
1.9 How relevant are the following ticket fields for you?
Here are the ticket fields sorted by conceived relevance in decreasing order, plus some comments. The numbers in brackets are (useful, somewhat useful, not useful):
The Component field (10, 1, 1) is used to find/filter tickets and guess who's paying attention to a ticket. 1 person said that the many Tor components make it hard to refer to the software "tor" and that it's easier to look at milestones itself.
The Owned by field (10, 1, 1) is important for most people to decide whether they're responsible for fixing or finding someone else to fix something.
The Milestone field (9, 0, 2) is used for release planning and sponsor deliverables, and it also determines the ticket priority to some extent.
The Reported by field (8, 2, 2) is used by some to estimate how useful a reported defect or suggested enhancement is, and it is useful to know whether someone will be mailed by Trac when the ticket changes.
The Cc field (7, 1, 3) is used to add people whose input is desired and to learn who will be mailed when the ticket changes. There's the problem that only users with certain permissions can add other persons to the Cc field of existing tickets.
[Suggestion: Allow all developers and volunteers to edit the Cc field.]
The Priority field (6, 5, 1) is important to some people, but is highly dependent on who sets the priority.
The Parent ID field (6, 1, 4) is used for complex tickets with many subtasks and when a task spans multiple components.
The Version field (3, 3, 5) is not used by many components and is considered not very useful, because bug reporters get versions wrong in most cases anyway. Also, current versions of products are never in the list.
[Suggestion: Delete all obsolete versions from the list, and try harder to add new versions.]
The Keywords field (2, 1, 8) is only used for agile iteration tracking and to tag tickets as "easy."
The Points (1, 0, 10) and Actual Points (1, 0, 10) fields are actively used by only 1 person and read by 1 other person.
[Suggestion: Investigate whether it's possible to suppress email notifications for changes to the Points and Actual Points fields.]
1.10 How relevant are the following ticket statuses for you?
Here are the ticket statuses sorted by relevance from most important to least important. Numbers in brackets are (useful, somewhat useful, not useful):
The closed status (8, 0, 2) is considered useful by pretty much everyone.
The needs_review status (7, 2, 1) is used to determine which tickets to read and respond to first.
The needs_information status (6, 2, 0) is new, but considered promising and says that one cannot do more without feedback.
The new (2, 1, 4), assigned (1, 3, 5), reopened (1, 2, 5), and accepted status (1, 1, 6) have slightly different meanings for some people, but are mostly equivalent for the majority. There are 5 persons who'd like to see these four statuses merged into one, e.g., assigned. 1 person suggests to narrow down statuses to three statuses: assigned, pending (with a combo box for reasons: requester information, code review, code push, schedule), and resolved.
[Suggestion: Investigate whether we can merge "new," "assigned," "reopened," and "accepted" into a single "assigned" status.]
1.11 What other features do you use in Trac?
4 out of 11 persons mention use of extended features. These features include admin functions, e.g., deleting users and spam tickets, embedded queries (see #3410), and RSS feeds for custom queries, tickets, wiki pages, and the timeline (2 mentions).
1.12 What features are you missing in Trac?
Better search: The current search is ridiculous. I need to specify things like "I want these three keywords definitely, only in torbutton tickets that are in status needs-review".
Git commit hooks (2 mentions): We should be able to mention "Bug #XXX" at the start of a commit message and have that commit get posted as a comment to trac w/ a link to the gitweb url for the commit. I believe we have a trac plugin for this, but it is not yet properly configured. At other employers, commits could only go into stable releases if they referenced a bug # in the first line. I thought this was a good policy. Back in the SVN days, this property was actually enforcable on the server. Git may make this bit more difficult..
[Suggestion: Investigate Git integration in Trac 0.12.]
Trac 0.12-stable: I would like some of the embedded query syntax that is available in 0.12.x-stable of trac. In particular, 0.12 would make it much easier to compute the rate of Opened tickets against a given project in a certain period of time.
[Suggestion: Try out Trac 0.12 on a VM and find out.]
Better handling of duplicate bugs: The "dup" feature should have a bug field that causes both bugs to be updated automatically with links to each other.
Field updates should perhaps not send email: Some field updates don't require emailing everybody on the ticket. I *think* 0.12 might provide the ability to solve this one, too, but I am not sure.
Prevent trac from emailing you about your own changes (2 mentions): always_notify_updater = false in trac.ini is close to this, but it is not exactly what I want. I think it is useful to automatically notify previous updaters via email, but it seems silly to send me mail for my own changes. Yet there is only one option for both behaviors in trac.. I could see setting it to false and just being more careful about adding people to the Cc line as a stopgap.
Embargoed/Security tickets: For serious security issues, it would be nice to be able to make tickets not viewable by accounts not on the Cc, owner, or reporter lines of the ticket. I don't think trac can do this at all.
Component-specific email notifications: I would have preferred the ability to receive emails for any changes related to a specific component (i.e. TorCtl).
[Suggestion: Find out if r2e can do the trick and explain this on the wiki somewhere.]
Better anonymous reporting (3 mentions): (First reporter) I'd like better anonymous reporting and commenting. (Second reporter) I've bolded the 'New Ticket' and cypherpunks info on the landing page, though creating a ticket via cypherpunks really should be a fist sized cherry red button on that page... (Third reporter) I think the 'cypherpunk' user should have to add an email address in the Cc field. If they put mailinator in there, so be it. But they at least need to know we need to talk to them, or their bugs will probably just sit around forever.
Voting on issues.
Tor-detection: It would be nice to have Tor detection built into tor - so we can see if a reporting user is actually using Tor!
Multiple milestones: I wish I could assign bugs to multiple milestones.
[Suggestion: Only assign defects and enhancements to software product milestones and only assign tasks and projects to sponsor milestones.]
Large-scale ticket management: One of the main problems we have in general is the proliferation of tickets. Originally that was a good thing -- "have something we need to do? Make a ticket, then we'll remember." But now we have so many darn tickets, especially in some components, that they're tough to manage, which leads to less active maintenance, which leads to having it get even more out of control. I'm not sure what to do about it, but sometimes it makes me less excited to add more tickets since I feel like I'm adding to the problem. So some feature to do large-scale ticket management would be good.
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
No email notifications for certain fields: points, actual points are annoying because changes in them cause email to be sent as an update. cc list is the same.
Separate system for users and developers: Trac is used for user support too much. I wish we had a system that was JUST developer centric, no user wiki pages or other crap that turns up in searches.
Landing page: Our landing page is a nightmare for newcomers.
[Suggestion: Write a new landing page.]
Single "tor" component: I don't know what confuses others, but IMO the proliferation of components that are all "tor" doesn't help me, and makes stuff slightly harder.
Get rid of wiki: Most of the contents of the wiki consist of obsolete and/or bad advice. These pages should be archived in a tarball where they will not be readily accessible to web browsers and search engines, then purged from our Trac installation. The rest of the pages in the Trac wiki should be moved to a wiki which allows a user who edit a page concurrently with another user to merge his/her/its changes into the wiki page. Perhaps we should just use Git and give up on browser-based wikis.