Skip to content

Instantly share code, notes, and snippets.

@anton-x-t
Last active May 3, 2024 10:03
Show Gist options
  • Save anton-x-t/11cf0848a2a9219dc53a913922ba3322 to your computer and use it in GitHub Desktop.
Save anton-x-t/11cf0848a2a9219dc53a913922ba3322 to your computer and use it in GitHub Desktop.
MongoDB Import Slowly PowerShell Script
# MIT Licence
# MongoDB Import Slowly PowerShell Script
# Sources below!!! Thank you all persons and entities!!!
$i = 0
$lines = Get-Content .\data.json
$lineCount = ($lines | Measure-Object -line).lines
Write-Output "Starting import of $lineCount lines."
$hostColonPort = <hostColonPort>
$username = <username>
$password = <password>
$db = <db>
$collection = <collection>
$lines | ForEach-Object {
$i++
$linesPart += "$_`n"
if ($i -eq $lineCount) {
$linesPart | Set-Content .\temp.json
Start-Sleep -Milliseconds 5000
mongoimport --host $hostColonPort `
-u $username `
-p $password `
--db $db --collection $collection `
--ssl --type json --writeConcern="{w:0}" `
--file temp.json
Write-Output "$i / $lineCount"
Start-Sleep -Milliseconds 5000
break
}
if ($i % 8 -eq 0) {
$linesPart | Set-Content .\temp.json
Start-Sleep -Milliseconds 5000
mongoimport --host $hostColonPort5 `
-u $username `
-p $password `
--db $db --collection $collection `
--ssl --type json --writeConcern="{w:0}" `
--file temp.json
Write-Output "$i / $lineCount"
Start-Sleep -Milliseconds 5000
$linesPart = ""
}
}
# Sources
# MongoDB, Write Concern,
# thank you MongoDB, Inc.,
# https://www.mongodb.com/docs/manual/reference/write-concern/
# Powershell counting lines in file incorrectly when there is only one line in the file?,
# thank you TheMadTechnician,
# https://stackoverflow.com/a/25532421/6533028
# how to overwrite file content with power shell,
# thank you Shiko,
# https://stackoverflow.com/a/48217922/6533028
# Find the number of lines in a project with powershell,
# thank you Alex,
# https://stackoverflow.com/a/562546/6533028
# How to identify the last item from pipeline foreach-object cmdlet,
# thank you Noah Sparks,
# https://stackoverflow.com/a/25731050/6533028
# PowerShell, Start-Sleep,
# thank you Microsoft, Inc.,
# https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/start-sleep
# PowerShell, Out-File,
# thank you Microsoft, Inc.,
# https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/out-file
# Powershell append line to variable in for loop,
# thank you Keith Hill,
# https://stackoverflow.com/a/12959692/6533028
# PowerShell, about_Variables,
# thank you sdwheeler,
# https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_variables
# Read file line by line in PowerShell,
# thank you Mathias R. Jessen et al,
# https://stackoverflow.com/a/33511982/6533028
# Mongodb foreach for nested collection to update/copy documents to another collection,
# thank you chridam,
# https://stackoverflow.com/a/33913765/6533028
# How to Import Data into MongoDB with mongoimport,
# thank you Mark Smith,
# https://www.mongodb.com/developer/products/mongodb/mongoimport-guide/
# Multiline copy-paste to mongo shell,
# thank you darkace,
# https://stackoverflow.com/q/37817961/6533028
# How do you throttle an update script for MongoDB?,
# thank you med116,
# https://stackoverflow.com/a/33294449/6533028
# How do you throttle an update script for MongoDB?,
# thank you jpaljasma,
# https://stackoverflow.com/a/33294041/6533028
# How do you throttle an update script for MongoDB?,
# thank you med116,
# https://stackoverflow.com/a/33294449/6533028
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment