For at most 30 minutes yesterday (June 20), candidate profiles were inaccessible in Ashby. While we quickly resolved the issue once we became aware of it, it simply shouldn't have happened - viewing candidate profiles is an important part of every team's recruiting process.
Why did this happen, and what have we put in place to prevent it from happening in the future?
Ashby has powerful features that let you analyze large amounts of data. We put safeguards in our app to prevent malicious actors from using that power to request unreasonably resource-intensive data from our servers to overload them. Yesterday, we shipped a change to the candidate profile that inadvertently triggered one of those safeguards for normal usage of Ashby.
Once we became aware of the issue, we immediately updated our status page, and our team was able to identify and ship a fix within 15 minutes. To prevent this from happening again, we added additional monitoring and automated checks to block code from being deployed that could inadvertently trigger this safeguard. These additional changes were deployed within 8 hours of the incident.
Please let us know if you have any questions or concerns.
Abhik Pramanik
Co-Founder and VP of Engineering