Skip to content

Instantly share code, notes, and snippets.

@negeric
Last active August 31, 2018 13:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save negeric/7c4fcc5cef16c8dcb9641f7fe3c42a71 to your computer and use it in GitHub Desktop.
Save negeric/7c4fcc5cef16c8dcb9641f7fe3c42a71 to your computer and use it in GitHub Desktop.
<#
.SYNOPSIS
This script will enumerate through a Storage Account and delete files that are older than x
.DESCRIPTION
The script requires you to log into your account then automatically pulls your Account Keys.
Using the Storage Account Keys, the script searches for files older than the provided date
.PARAMETER ResourceGroup
Name of the Resource Group that the Storage Account is in - Required
.PARAMETER StorageAccount
Name of the Storage Account that the Container is in - Required
.PARAMETER Container
Name of the Container holding the files/blobs - Required
.PARAMETER FileExtension
File extension filter. Script will search for files matching this extension
.PARAMETER DaysToKeep
Delete files older than X days
.PARAMETER DryRun
String param, if provided, and true, do not delete, just log what would be deleted
.INPUTS
None
.OUTPUTS
Outputs to the console
.NOTES
Version: 1.0
Author: Jeff Anderson
Creation Date: February 7th, 2018
Purpose/Change: Initial script development
.EXAMPLE
.\Remove-AzureBlobStorage.ps1 -ResourceGroup "SQLBackupsRG" -StorageAccount "Backups" -Container "SQLBackups" -FileExtension ".bak" -DaysToKeep 31
#>
param (
[Parameter(Mandatory=$true)]
[String] $ResourceGroup,
[Parameter(Mandatory=$true)]
[String] $StorageAccount,
[Parameter(Mandatory=$true)]
[String] $Container,
[Parameter(Mandatory=$false)]
[String] $FileExtension = "*",
[Parameter(Mandatory=$true)]
[int] $DaysToKeep,
[Parameter(Mandatory=$true)]
[String] $DryRun = "false"
)
function ValidateFileExtension {
param (
[Parameter(Mandatory=$true)]
[String]$Extension
)
if ($Extension -eq "*") {
return $Extension;
} else {
if ($Extension.StartsWith(".")) {
return "*$Extension";
} else {
return "*.$Extension"
}
}
}
$Now = [DateTime]::Now
$Extension = ValidateFileExtension -Extension $FileExtension
## Authentication
Write-Output ""
Write-Output "----------------------- Starting Script ------------------------"
Write-Output "$Now"
Write-Output "Resource Group: $ResourceGroup"
Write-Output "Storage Account: $StorageAccount"
Write-Output "Container: $Container"
Write-Output "File Extension: $Extension"
Write-Output "Days to Keep: $DaysToKeep"
if ($DryRun.ToLower() -eq "true") {
Write-Output "DRY RUN - NOT DELETING ANYTHING"
}
Write-Output "------------------------ Authentication ------------------------"
Write-Output "Logging in to Azure ..."
try
{
$ConnectionName = "AzureRunAsConnection"
# Get the connection "AzureRunAsConnection "
$ServicePrincipalConnection = Get-AutomationConnection -Name $ConnectionName
$null = Add-AzureRmAccount `
-ServicePrincipal `
-TenantId $ServicePrincipalConnection.TenantId `
-ApplicationId $ServicePrincipalConnection.ApplicationId `
-CertificateThumbprint $ServicePrincipalConnection.CertificateThumbprint
Write-Output "Successfully logged in to Azure."
}
catch
{
if (!$ServicePrincipalConnection)
{
$ErrorMessage = "Connection $ConnectionName not found."
throw $ErrorMessage
}
else
{
Write-Error -Message $_.Exception
throw $_.Exception
}
}
## End of authentication
## Get Storage Account Key for Storage Account
Write-Output ""
Write-Output ""
Write-Output "---------------------------- Status ----------------------------"
Write-Output "Getting Storage Account Keys for $StorageAccount ..."
try
{
$StorageAccountKeys = Get-AzureRmStorageAccountKey -ResourceGroupName $ResourceGroup -AccountName $StorageAccount
## Create a Storage Account Context
## Try using Key1 then Key2
try {
$StorageAccountKey = $StorageAccountKeys.Key1
$Context = New-AzureStorageContext -StorageAccountName $StorageAccount -StorageAccountKey $StorageAccountKey
Write-Output "Using Storage Account Key 1"
} catch {
Write-Output "Failed to create Storage Context with Key 1, trying Key 2"
try {
$StorageAccountKey = $StorageAccountKeys.Key2
$Context = New-AzureStorageContext -StorageAccountName $StorageAccount -StorageAccountKey $StorageAccountKey
Write-Output "Using Storage Account Key 2"
} catch {
$Context = $null;
Write-Output "Failed to create Storage Context with Key 2, exiting"
Write-Error "Error creating Storage Context using either Account Key"
throw $_.Exception
}
}
## Loop through the list of possible deletes
$DeleteDate = [DateTime]::Today.AddDays(-$DaysToKeep)
$PossibleDeletes = 0;
$Deleted = 0;
$TotalSize = 0;
$DeletedSize = 0;
Write-Output "Searching for actionable Blobs in Container"
$Blobs = Get-AzureStorageBlob -Context $Context -Container $Container $Extension | Where-Object { $_.LastModified -lt $DeleteDate }
foreach ($Blob in $Blobs) {
$PossibleDeletes++
$TotalSize = $TotalSize + $Blob.Length
Write-Output "Working on " $Blob.Name
try {
if ($DryRun.ToLower() -eq "false") {
Remove-AzureStorageBlob -Blob $Blob.Name -Container $Container -Context $Context
} else {
Write-Output "Dry Run - " $Blob.Name
}
$Deleted++
$DeletedSize = $DeletedSize + $Blob.Length
} Catch {
Write-Output "Failed to delete " - $Blob.Name
}
}
$TotalSizeHR = [System.Math]::Round($TotalSize / 1GB, 2)
$DeletedSizeHR = [System.Math]::Round($DeletedSize / 1GB, 2)
$FailedToDelete = ($PossibleDeletes - $Deleted)
Write-Output "Deleted $Deleted of $PossibleDeletes Blobs"
Write-Output "Cleaned up $DeletedSizeHR GB"
Write-Output "Failed to delete $FailedToDelete Blobs"
}
catch
{
Write-Output "Error fetching Storage Account Key"
Write-Error -Message $_.Exception
throw $_.Exception
}
@chield
Copy link

chield commented Aug 31, 2018

thanks for the awesome script! One issue that I have is that when I try to run it for example on a storage account with a blob folder-structure like this: \sqldbauditlogs\customname\auditdate\year\day it crashes.

Do you know the solution for this so I can ran this for example on \sqldbauditlog.... ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment