Skip to content

Instantly share code, notes, and snippets.

@mmodrow
Last active May 3, 2023 13:58
Show Gist options
  • Save mmodrow/9628e5d698a637991ad6c3bf7f5d43ee to your computer and use it in GitHub Desktop.
Save mmodrow/9628e5d698a637991ad6c3bf7f5d43ee to your computer and use it in GitHub Desktop.
"Big Data" creator (creates big json files with an object array)
[CmdletBinding()]
param (
[Parameter()]
[String]
$filePath,
[long]
$targetSizeInBytes = 500000000, #4294967296 Bit = 512MiB
[switch]
$deleteFile = $false
)
function addData {
param (
[string] $filePath,
[long]$targetSizeInBytes,
[int]$index,
[string] $randomString
)
if (!(Test-Path $filePath)) {
New-Item -ItemType File -Path $filePath
Set-Content -Value "[" -Path $filePath
}
$file = Get-Item $filePath
$fileSizeInBytes = $file.Length
if ($fileSizeInBytes -ge $targetSizeInBytes) {
Add-Content -Value ("`n]") -Path $filePath -NoNewline
return $false
}
if ($index -gt 0) {
Add-Content -Value ",`n" -NoNewline -Path $filePath
}
$json = ConvertTo-Json (@{
id = $index;
targetSize = $targetSizeInBytes;
currentSize = $fileSizeInBytes;
path = $filePath;
random = $randomString
})
Add-Content -Value ($json) -Path $filePath -NoNewline
return $true
}
if ($deleteFile -and (Test-Path $filePath)) {
Remove-Item -Path $filePath
}
$randomString = (-join (0..5000 | ForEach-Object {((65..90) + (97..122) | Get-Random -Count 1 | ForEach-Object {[char]$_})}))
for ($i = 0; (addData -filePath $filePath -targetSizeInBytes $targetSizeInBytes -index $i -randomString $randomString); $i++) {
Write-Host ("" + $i + " elements written to " + $filePath)
}
Write-Host ($filePath + " is now " + (Get-Item $filePath).Length + " Bytes long.")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment