Export to CSV doesn't work

Dominik Bill shared this problem 4 years ago
Solved

I orderd the commercial Version so I can use the export functionality.Poorly the exports doesn't work. If I want to export a bigger amount of entriesthe NoSQLBooster sometimes aborts the job or the complete software crashes. Ifit tries to export data it takes much time


I am using csv exports and try to export around 700.000 entries out of amongoDB with 62.7M entries and a size of 0.12TB.


Used version: 5.1.5

Replies (6)

photo
1

Thank you for your feedback. The current ability to export query result is not very suitable for exporting large amounts of data. In order to achieve simplicity, all the data in the resulting dataset is read into memory at once and then exported in batches, and if the data is large, there may not be enough export memory.

We'll take a good look at the problem and try to solve it in the next update.

A workaround solution, save query result as a view then export the view.

1. use "saveAsView" method to save the query result as a read-only view. e.g. db.collection.find({status:1}).saveAsView("query_result",{dropIfExists:true})

2. Refresh DB node in the object explorer

3. Select the newly added "query_result" node

4. Popup right-click menu

5. Select "Export Collection"

/396014dd8c8e28c6bb440b2343dcf034

photo
2

We have worked out a test build to solve the export crash issue, please download and give it a try.

Windows: https://s3.mongobooster.com/download/releasesv5/nosqlbooster4mongo-5.1.6-beta.1.exe

MacOS: https://nosqlbooster.com/s3/download/releasesv5/nosqlbooster4mongo-5.1.6-beta.1.dmg

photo
1

Thanks for the new version. The Bosster does not crashes any more. Poorly it still doesn't export all entries it finds with the db.collection.find() command. If it finds 7440 entries it says that it exportet this amount of entries but in the csv are only 6440 with a Batch size of 1000 or 4000 with a bacht size of 100. Please fix this Problem.

photo
1

I tried a few times, but can't recall the issue. The problem of not being able to reproduce locally is difficult to locate. If possible, please give us the test-purpose JSON data or accessible MongoDB URL and complete reproduced steps.

photo
1

Poorly I cannot give you some data because they contain userdata.

The problem appears at nearly every export so i am wondering that you cannot reproduce that problem.

photo
1

I tried it a few more times ( even specifying to export 7,440 data), but I still can't reproduce the problem.

Since it can appear every time in your machine, could you construct some data for testing and give the detailed reproduction steps? If you really can't give a reproduction step, could you give me a screen recording?

Thank you.

Leave a Comment
 
Attach a file