Dear 0x00sec Users,
We are writing to you with important information regarding a recent security incident involving your personal information from https://0x00sec.org. We became aware of the incident September 7th, 2020, when a security researcher from Thug Crowd privately disclosed to us that our S3 bucket containing database backups was publicly accessible. The S3 bucket was publicly accessible for a total of 63 days, from July 6th - September 7th, 2020, and contained usernames, email addresses, direct messages and salted PBKDF2 password hashes.
[Although we are unaware of any actual misuse of your information, we are providing notice to you and other potentially affected users about the incident.]
What happened?
On July 6th we ran a routine update of Discourse that introduced a bug that overwrote images and mangled object prefixes (folders). The bug affected us because we did not separate buckets for uploads and backups. Instead, we used ACLs to allow public access to the entire bucket to store user uploads with the exception of a subfolder that blocks public access which we used for backups.
Example:
Allow Public Access s3_upload_bucket: my-backups-bucket/
Block Public Access s3_backup_bucket: my-backups-bucket/backups
Immediately after the update, images started linking to invalid s3 objects causing 403 Forbidden errors to be returned. We decided to delete our s3 backups folder and restore from local backups. We then checked for backup files that might have allowed public access and didn’t find any.
What we didn’t know at the time is that using the AWS web console for searching s3 buckets does not search recursively and after the update, a new maintenance job intended to run solely on the contents of the backup bucket affected contents of the uploads because their values were the same. In consequence, folders and objects from /backups were copied to the parent folder with no ACLs restricting public access.
September 7th, 2020, Thug Crowd notified us of the exposure and we immediately isolated the environment by disabling S3 and locked off all ACLs to investigate further.
We deprovisioned and redeployed our containers on new infrastructure, created a new bucket with proper ACLs and restored our user database from a known safe backup. We removed s3 entirely from the backup process and moved to a local solution instead.
We continued our investigation of the issue as well as the possible impact to any data stored in the S3 buckets. The key thing was to ensure that we had a solid plan to add additional protections and a procedure for securing site backups. This included performing a server migration, SIEM tuning and extra logging enabled where possible. This was a time-consuming process but it was important to set this up properly to avoid any issues in the future.
Upon completion of the remediation activities, a full user password reset performed as a precautionary measure and all members notified.
What information was involved?
The information exposed relates to user data such as plaintext usernames, email addresses, direct messages, profile data and salted PBKDF2 password hashes. PBKDF2 passwords hashes, when salted, with a 10 character minimum password requirement like 0x00sec, are very hard to crack, however, still not impossible.
What’s being done?
As stated, we deprovisioned and redeployed our containers on new infrastructure, created a new s3 bucket with proper ACLs and restored our user database from a known safe backup. We added end-to-end encryption for direct messages and completely revamped our security posture by fine-tuning our SIEM and IDR, as well as enhanced our logging. A full password reset has also been enacted as a precautionary measure.
What you can do?
As a precaution we temporarily disabled all user accounts, the only action required is to reset your password to regain access.
Where to find more information?
If you have any questions or concerns, please feel free to DM or email me at [email protected]. I’ll do my best to respond in a timely manner.
Timeline:
A full timeline of events have been provided for maximum transparency:
- July 6th 2020 9:15am - Ran a routine Discourse Update, a bug overwrote bunch of images on Amazon S3 causing us to lose a bunch of article images, occurred because an automatic backup overwrote all the images leaving a site backup on the bucket.
- July 6th 2020, 10:25am - Same day, the bucket was searched for backups, S3 search is not recursive, Pry missed this and first backups were exposed from this point on.
- July 6th 2020, 11:47 am - Same day, images were now functional
- 63 days passed
- September 7th, 2020, 5:36 PM - ThugCrowd Reported Vulnerability to 0x00sec
- September 7th, 2020, 5:42 PM - The vulnerability was remediated. 0x00sec remediated the exposure within 10 minutes of reporting and entirely disabled S3 and locked off all ACLs to investigate further.
- September 12th, 2020 Afternoon - Further investigation as to nature of the issue was performed, plans were made to add additional protections, site backups will no longer be made to any additional cloud platforms.
- September 26th, 2020 2:00pm - 11:45pm - Further planned remediation work performed including server migration, SIEM tuning, extra logging enabled where possible.
- September 30th 2020 5:00pm - Completion of planned remediation activities, full user password reset performed as a precautionary measure and all members notified
Is there any evidence these backups been obtained by anybody other than the security researcher who reported it?
- No, all evidence currently suggests that we’re the first to catch this, however, no logging was configured on S3 as the bucket was only intended for public site images. 0x00sec has pulled logs for everything where possible, and 0x00sec have searched multiple S3 indexing tools.
What was exposed?
- Usernames
- Password Hashes (Salted PBKDF2, Min password length: 10 chars)
- Email addresses
- User DM’s
How long was this exposed?
- The backups were exposed on the bucket for exactly 63 days after creation.
Who does this affect?
- This only affects users with accounts on the forum, 0x00sec.org. As most 0x00sec users are on the Discord, and this is separate, this will not affect any Discord users.
What do I need to do about it?
- All user passwords have been reset, all you need to do is reset your user password if you wish to regain access to your account.
What’s the good news?
- Password hashes in use are quite strong, and the user direct message functionality is largely unused as most of the community use Discord - especially in the past year. This event has prompted an array of upgrades to the community infrastructure including encrypted direct messages (coming soon).
Extra Notes:
When resetting your password, please check your junk box and wait up to 10 minutes as we have recently changed email providers.
Extra Precautions:
- Reset all user passwords
- Extra monitoring on accounts for abuse
- Extra logging in place (including all s3 & image buckets just in case)
- End-End Encryption (In-the-browser key generation and storage) for Direct Messages
- Reconfigure S3 and removed backups entirely from external-cloud sources
Update: We’re having some trouble sending notification emails to all the users, they will be arriving shortly. All user passwords have been reset, if you’re reading this all you have to do is request a new password to regain access to your account.
If you have any further questions, please email [email protected] and we’ll reply as soon as we can.