Search unavailable
Incident Report for Futureproofs
Resolved
TL;DR Search is now working properly again.

The re-index for the search database is now finally complete and your proofs should be searchable again, and any new proofs should have a search index created as normal. I know that the delay in completing this task has caused problems for some users, and I'm very sorry about that. We've talked to those most affected but do let me know if you'd like to discuss how this has affected you and your team.
Posted 9 months ago. Nov 03, 2018 - 08:53 GMT
Update
TL;DR It's taking much longer than we'd hoped to rebuild the search index, which means new proofs you upload won't get a working search function until the whole job is complete.

The re-indexing is unfortunately taking many days to complete, even though we've limited the number of proofs being re-indexed, as I shared previously. This is because our database has been rate-limited by our provider - because we're writing so much data to it, they've slowed down our access (presumably, to persuade us not to write so much data again in future!), which just makes the process take even longer. The indexing continues but there is a very unfortunate consequence: because newly uploaded proofs use the same queue as the existing proofs that are being re-indexed, any new proofs you upload during this process go onto the end of the queue and so won't be indexed until the whole queue is cleared, which could be several days yet. This means that, although you can mark up new proofs as usual, you won't be able to search them yet.

I'm very sorry about this!
Posted 9 months ago. Oct 22, 2018 - 10:33 BST
Update
All newly created proofs will now be indexed correctly for search. We are rebuilding the index for all proofs 6 months old or newer. Older proofs will not be searchable but are very unlikely to be part of live projects so we hope this will be no inconvenience. (We've taken this decision because indexing everything would take a very long time - and could also seriously affect the performance of other parts of the system because it would hit our databases hard.)
Posted 9 months ago. Oct 11, 2018 - 21:45 BST
Update
We have identified and rectified the original issue with the search subsystem, and will shortly be reindexing the entire database so that search functionality is restored. Unfortunately, because there are a lot of proofs containing a *lot* of words, this will take a while! We will update this page as we have more information.
Posted 9 months ago. Oct 11, 2018 - 21:32 BST
Identified
While we were working on our search database, in order to resolve a problem with the indexing of new proofs, our team unintentionally triggered a re-indexing of the whole database. This was cancelled, but has caused the search system to be unavailable at present. We are working to restore this from a recent backup and will bring search back as soon as possible. We're very sorry for this problem.
Posted 9 months ago. Oct 11, 2018 - 07:15 BST
This incident affected: Our servers (Our search database).