The YAML reports had two time fields: start_time: timestamp of the start of ooni-probe run test_start_time: timestamp of the start of each individual test Within a single report file, start_time was constant, while test_start_time would advance with each successive test, depending on how long each test took to run.
The JSON format reports have just one of the fields, test_start_time, but it confusingly appears to have the same meaning as start_time in the old YAML reports (it doesn't change within a report file): test_start_time: timestamp of the start of ooni-probe run
It might be because of this code: https://github.com/TheTorProject/ooni-pipeline/blob/355ac1780f1f05eefb9ea3bf... entry['test_start_time'] = datetime.fromtimestamp(entry.pop('start_time', 0)).strftime("%Y-%m-%d %H:%M:%S")
Some tests can take many minutes or hours to run, so by the end, the JSON test_start_time might be far off from the real time when it was run. Is there a way we could get both timestamps in each record again?
What I'm doing now is incrementing a counter according to the test_runtime field of each record, and adding that counter to the test_start_time in order to estimate the individual test's start time. But I feel that is only approximate, and some older reports do not have test_runtime.