Skip to content

Instantly share code, notes, and snippets.

@darrenjrobinson
Last active February 4, 2019 22:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save darrenjrobinson/49e570f7cb6c1bc47b24a97452eb5cae to your computer and use it in GitHub Desktop.
Save darrenjrobinson/49e570f7cb6c1bc47b24a97452eb5cae to your computer and use it in GitHub Desktop.
Load dataset into Table Storage in Batches using Powershell. Associated Blog Post https://blog.darrenjrobinson.com/loading-and-querying-data-in-azure-table-storage-using-powershell/
$subscriptionName = "myAzure Subscription"
$resourceGroupName = "myResourceGroup"
$storageAccountName = "myStorageAccount"
$tableName = "NICVendors"
$partitionKey = "Manufacturers" # Partition Key
# Log on to Azure and set the active subscription
Add-AzureRMAccount
Select-AzureRmSubscription -SubscriptionName $subscriptionName
# Get the storage key for the storage account
$storageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $resourceGroupName -Name $storageAccountName).Value[0]
# Get a storage context
$ctx = New-AzureStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey
# Get a reference to the table
$table = Get-AzureStorageTable -Name $tableName -Context $ctx
[int]$entryCount = 1
[int]$rowCount = 0
# Azure Table Storage Batch Operations
[Microsoft.WindowsAzure.Storage.Table.TableBatchOperation]$batchOperation = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.TableBatchOperation
foreach ($line in $vendors) {
$rowCount++
$entity = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.DynamicTableEntity -ArgumentList $partitionKey, $entryCount
$entryCount++
Write-Host -ForegroundColor cyan "$($partitionKey), $($line.vendor)"
$entity.Properties.Add("Vendor", $line.vendor)
$entity.Properties.Add("Base16Address", $line.base16)
$entity.Properties.Add("HexAddress", $line.hex)
if ($rowCount -le 100) {
# Add to Batch. Batches max of 100 rows
$batchOperation.InsertOrReplace($entity)
}
else {
# Commit 100 rows to the Table
$table.CloudTable.ExecuteBatch($batchOperation)
[Microsoft.WindowsAzure.Storage.Table.TableBatchOperation]$batchOperation = New-Object -TypeName Microsoft.WindowsAzure.Storage.Table.TableBatchOperation
$batchOperation.InsertOrReplace($entity)
$rowCount = 1
}
}
if ($batchOperation.Count -ne 0) {
$table.CloudTable.ExecuteBatch($batchOperation)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment