String Json Parser

lately, I imported a big CSV file, containing fields with values of Json object in String format, the simplest way to parse those strings to object it was by doing find().forEach( JSON.pase then update) it taken too much time to update the collection the only think that i have could done to reduce the latency is putting write priority to 0 which isn't very advisable with high throughput of updates, I don't really know for sure I'am not the expert here but working with big data demand a lot of cleaning, and not being able to do the cleaning on the server only on the client side, high latency make working very difficult and stressful, aggregation give you the possibility to get information from your data and storing the result in different collection, except for update pipelines but you can't do much in them, i think more string operation like replace with, json parser for example, cleaning features out and merge to the same collection that would be one of the best features. ps sorry 🙏 for my poor english

Attach files

Enter a subject

Guest

Oct 8, 2020

Hi Ali, in MongoDB 4.4 you can run JavaScript function inside the aggregation stage, so you can avoid the network roundtrip for each document. For example: {$set: { user: { $function: { arguments: ["$user"], body: function(user) { return JSON.parse(user); }, language: 'js' } } }} Also, there are $replaceOne and $replaceAll operators for strings, and $merge can now output to the same collection. Release Notes: https://docs.mongodb.com/manual/release-notes/4.4/index.html#aggregation

Reply
Hide replies
Like

Please enter your email address

RELATED FEEDBACK

String Json Parser