Skip to Main Content

MongoByte MongoDB Logo

Welcome to the new MongoDB Feedback Portal!

{Improvement: "Your idea"}
We’ve upgraded our system to better capture and act on your feedback.
Your feedback is meaningful and helps us build better products.

Status Submitted
Categories Database
Created by Guest
Created on Oct 31, 2023

Allow disabling auto removal for capped collections

Capped collections work as a circular buffer - when adding new documents after the size limit is reached, old documents are removed. I would like to have an option for the collection, where this behavior can be changed - instead of removing old documents, the insert should fail with a specific error (eg: collection full). This would allow implementing log-rotation, by creating a new collection whenever the old one is full. My motivation: I have two services, one creating documents (events), and another one processing them. I'm looking for a reliable way to: - process all documents from start, without skipping or double-processing any of them - after catching up, process all newly inserted documents - avoid updating the documents by the processor (eg: avoid adding a "processed" field to the processed documents) In order to be able to iterate all documents, and be able to continue from where I left off during an error/re-deployment of the processor service, the documents need to have some kind or order, where newly inserted documents can't get added before an already processed one. My options right now: - use a normal collection, with a global auto-increment id - this is what i'm using, but it's a bit complicated with mongodb - use a capped collection, where the insertion order is kept - there is a risk of "losing" documents if the processor gets too behind, that's not acceptable - using a "large enough" storage is not scalable
  • Attach files