Looks like the primary CollecTor instance had a problem between 22:00 and 08:00 UTC. It works again now, as does Onionoo.
Karsten, thanks for the fast reaction.
We didn't lose any data, because the primary CollecTor instance obtained all descriptors it had missed earlier from the backup CollecTor instance.
Since I'm archiving onionoo data I'm "loosing" data (causing blind spots) everytime a "relays_published" timestamp is skipped. In theory one could spin up an onionoo instance to generate data for skipped timestamps but in practice this is hard (requires lots of resources). (I know, you are probably talking about not loosing any raw CollecTor data, but wanted to mention that nonetheless.)
Do you monitor onionoo for such problems ("relays_published" timestamp remaining unchanged for >1-2 hours)? Would you find something like that useful?
Thanks for keeping it running besides all the other things you do.
I'm wondering if the admin team would be available to cover such cases to reduce the operations load for developers.
kind regards, nusenu