Windows version crashes when exporting to csv

CN shared this problem 4 years ago
Cannot Reproduce

Hello,


When trying to export around 150k records from a collection using the non-Mongoexport method, NoSQLBooster crashes on the reading stage at about 130k records. Collection size is .5 GB with 151k records.


Running version 3.4.10 of MongoDB and 4.3.0 of NoSQLBooster.


Thanks.

Replies (4)

photo
1

Thank you for your feedback.

I did a quick local test, collection size is 3 GB with 10M records, but can't recall the issue. I doubt that the issue is related to specific system environment or data content. If possible, could you please provide me the sample data and detailed steps to reproduce?

20106ad006083f22622abb12f02cd8ab

photo
1

Thanks for the quick response.


Unfortunately I can't provide sample data due to confidentiality reasons. What I can tell you is that the collection has 328 fields (various datatypes including arrays) and our database was designed by developers with limited MongoDB experience.


Steps to reproduce are as follows:

1. Double-click on collection from left pane to open shell that displays results of db.abc.find({})

2. Export-> Export to JSON, BSON...

3. In Export window:


a. select CSV as format

b. select Target file

c. Click off "All Fields"

d. Deselect fields that are are primarily arrays. Note that some fields were created with the same name but different datatypes. For example, there are multiple fields with the following types: Array(Empty)-2.1%; String-95.4%; Null-0.3%). Though these fields still have a few records with arrays, I'm still trying to export them.

e. Click Export.

f. Export slows at around 120k records, then NoSQLBooster becomes unresponsive and crashes with message from OS asking to close, debug, etc.


I can also provide you with a schema analysis document by email if you let me know where to send it (separate issue but may be relevant...when trying to run a schema analysis on the same collection with Analyze Type = All, I'm getting the same behavior with NoSQLBooster becoming unresponsive and then crashing).


Thank you for your assistance.

photo
1

It is difficult to locate the issue by guessing.

Please try to replace the following line

from "let csvContent = mb.docsToCSV({ docs, fields, delimiter, withColumnTitle: isFirstRead });" to

let csvContent

try{
  csvContent = mb.docsToCSV({ docs, fields, delimiter, withColumnTitle: isFirstRead });
} catch(ex){
   console.log("docs to csv error", docs=>docs._id)
}
If it can catch error. Could you please give me the JSON content of docs which caused the error.


And, you can also export CSV with mongoexport utility. Toolbar->Export button -> Mongoexport utility (JSON,CSV, faster...)

photo
1

Hello,

I tried replacing the export code as listed above, but no errors were caught and the application still crashed.

I also tried your suggestion of using the mongoexport utility, and the export was successful! This addresses my needs.

Thank you.

Leave a Comment
 
Attach a file