Skip to content

Instantly share code, notes, and snippets.

@phbits
Last active November 18, 2022 00:41
Show Gist options
  • Save phbits/854343e658c4911bcbe6cec1b19a2f53 to your computer and use it in GitHub Desktop.
Save phbits/854343e658c4911bcbe6cec1b19a2f53 to your computer and use it in GitHub Desktop.
############################################[   DRAFT   ]#############################################

The following documentation describes the process of integrating a non-DSC PowerShell module called MyModule into the Sampler module framework. Benefits of this approach include an automated build pipeline that can easily leverage custom or community developed tasks.

For technical details, refer to:

Table of Contents

  1. Create Sampler Structure
  2. Integrate MyModule
  3. Git Integration
  4. Build
  5. Test
  6. Pipeline & Tasks
  7. Add Custom Task
  8. Configure Builtin Tasks

Create Sampler Structure

The following commands will create a Sampler SimpleModule.

# So the pipeline can automate versioning.
choco install GitVersion.Portable

Install-Module -Name 'Sampler' -Scope 'CurrentUser'

Import-Module -Name Sampler -Scope Local

$newSampleModuleParams = @{
	DestinationPath   = 'C:\source\'
	ModuleType        = 'SimpleModule'
	ModuleName        = 'MyModule'
	ModuleAuthor      = 'MyModuleAuthor'
	ModuleDescription = 'MyModule Description'
}

New-SampleModule @newSampleModuleParams

Integrate MyModule

With the Sampler framework in place, the next step is to integrate MyModule.

Private Functions

All private functions must reside in separate files within <ModuleRoot>\source\Private. The name of the file should be the name of the function with .ps1 extension.

Example

# Function: Get-PrivateFunction1
C:\source\MyModule\source\Private\Get-PrivateFunction1.ps1

# Function: Get-PrivateFunctionTwo
C:\source\MyModule\source\Private\Get-PrivateFunctionTwo.ps1

Public Functions

All public functions must reside in separate files within <ModuleRoot>\source\Public. The name of the file should be the name of the function with .ps1 extension.

Example

# Function: Get-PublicFunction
C:\source\MyModule\source\Public\Get-PublicFunction.ps1

# Function: Get-PublicFunctionTwo
C:\source\MyModule\source\Public\Get-PublicFunctionTwo.ps1

Module Manifest

The module manifest must be copied to the following location:

C:\source\MyModule\source\MyModule.psd1

Additional points regarding the module manifest include:

  • The RootModule should be set to MyModule.psm1
  • ReleaseNotes must exist at 'PrivateData.PSData.ReleaseNotes'.
  • ReleaseNotes can be emtpy string ('') - it is updated on build from CHANGELOG.md (the Unreleased section).
  • Prerelease is important to have in the PrivateData.PSData section as well. Otherwise preview version don't work (might fail deploy).

README.md

README.md should be copied to:

C:\source\MyModule\README.md

LICENSE

LICENSE should be copied to:

C:\source\MyModule\LICENSE

Update Help File

Update the localized help file at:

C:\source\MyModule\source\en-US\about_MyModule.help.txt

Folder Structure

At this point, the folder structure should look like the following:

C:\source
└───MyModule
    │   .gitattributes
    │   .gitignore
    │   azure-pipelines.yml
    │   build.ps1
    │   build.yaml
    │   CHANGELOG.md
    │   LICENSE                             <- LICENSE
    │   README.md                           <- README
    │   RequiredModules.psd1
    │   Resolve-Dependency.ps1
    │   Resolve-Dependency.psd1
    │
    ├───output
    │   └───RequiredModules
    ├───source
    │   │   MyModule.psd1                   <- Module Manifest
    │   │
    │   ├───en-US
    │   │       about_MyModule.help.txt     <- Update
    │   │
    │   ├───Private
    │   │       Get-PrivateFunction1.ps1    <- New File
    │   │       Get-PrivateFunctionTwo.ps1  <- New File
    │   │
    │   └───Public
    │           Get-PublicFunction.ps1      <- New File
    │           Get-PublicFunctionTwo.ps1   <- New File
    │
    └───tests
        └───QA
                module.tests.ps1

Git Integration

Initialize git and commit changes.

cd C:\source\MyModule
git init
git add *
git commit -m 'initial commit'

Build

When building a PowerShell script locally, the first step is to set the environment and download dependencies. The following command instructs the build script to complete the bootstrap process, download dependencies, and set the environment. The noop task stops the build script from running any tasks. For technical details, refer to the Sampler README and videos mentioned at the beginning.

.\build.ps1 -ResolveDependency -Tasks noop

Once the noop task has successfully completed, proceed with running a local build.

.\build.ps1 -Tasks build

A successful build will complete all defined build: tasks under BuildWorkFlow: in build.yaml. While this is configurable, the following are some tasks performed by default.

  • Clean output directory by removing everything except RequiredModules.
  • Create a versioned directory in output\MyModule\<version> as a destination for the newly built PowerShell module.
  • Consolidate Public & Private scripts into the root module. In this case MyModule.psm1.
  • Copy localized help file.
  • Create ReleaseNotes from changelog and update the Changelog for release.
  • Update module manifest with release notes.
  • Complete without errors.

At this point, the output folder should look similar to the following:

output
│   CHANGELOG.md
│   ReleaseNotes.md
│
├───RequiredModules
│   ├───ChangelogManagement
│   ├───Configuration
│   ├───DscResource.AnalyzerRules
│   ├───DscResource.Common
│   ├───DscResource.DocGenerator
│   ├───DscResource.Test
│   ├───InvokeBuild
│   ├───MarkdownLinkCheck
│   ├───ModuleBuilder
│   ├───Pester
│   ├───Plaster
│   ├───PowerShellForGitHub
│   ├───PSScriptAnalyzer
│   ├───Sampler
│   ├───Sampler.GitHubTasks
│   └───xDSCResourceDesigner
│
└───MyModule
    └───0.0.0.1
        │   MyModule.psd1
        │   MyModule.psm1
        │
        └───en-US
                about_MyModule.help.txt

Test

With a successful build, running builtin tests can be done with the following command. It instructs the build script to only run the test: tasks under BuildWorkFlow: in build.yaml.

.\build.ps1 -Tasks Test

A common error produced by this command will be regarding insufficient code coverage (error below). To fix, open build.yaml and under PESTER Configuration there's a setting: CodeCoverageThreshold Adjust this setting to an appropriate level or set to 0 to disable.

ERROR: The Code Coverage FAILURE: (0 %) is under the threshold of 85 %.

Adding QA Module Manifest Test

Once the default build and test tasks complete without errors, start adding tests specific to MyModule.

Suppose you want to add a test that verifies the newly built module manifest contains all required settings. Save the following code block to:

C:\source\MyModule\tests\QA\manifest.tests.ps1

Content of manifest.tests.ps1:

$here = Split-Path -Parent $MyInvocation.MyCommand.Path

# Convert-path required for PS7 or Join-Path fails
$ProjectPath = "$here\..\.." | Convert-Path
$ProjectName = (Get-ChildItem $ProjectPath\*\*.psd1 | Where-Object {
    ($_.Directory.Name -match 'source|src' -or $_.Directory.Name -eq $_.BaseName) -and
    $(try { Test-ModuleManifest $_.FullName -ErrorAction Stop }catch{$false}) }
).BaseName

$buildModuleFolder = Join-Path $ProjectPath -ChildPath "output\$ProjectName" -Resolve

$builtManifest = Get-ChildItem $buildModuleFolder -File `
                        -Filter "$ProjectName.psd1" -Recurse `
                        -ErrorAction SilentlyContinue

# Define test cases. The module manifest will be 
# checked for the following settings.
$requiredSettings = @(
	@{ 'Setting' = 'Author';            'Location' = 'root'; }
	@{ 'Setting' = 'CompanyName';       'Location' = 'root'; }
	@{ 'Setting' = 'Copyright';         'Location' = 'root'; }
	@{ 'Setting' = 'Description';       'Location' = 'root'; }
	@{ 'Setting' = 'FunctionsToExport'; 'Location' = 'root'; }
	@{ 'Setting' = 'GUID';              'Location' = 'root'; }
	@{ 'Setting' = 'ModuleVersion';     'Location' = 'root'; }
	@{ 'Setting' = 'PowerShellVersion'; 'Location' = 'root'; }
	@{ 'Setting' = 'LicenseUri';        'Location' = 'PrivateData.PSData'; }
	@{ 'Setting' = 'ProjectUri';        'Location' = 'PrivateData.PSData'; }
	@{ 'Setting' = 'ReleaseNotes';      'Location' = 'PrivateData.PSData'; }
	@{ 'Setting' = 'Tags';              'Location' = 'PrivateData.PSData'; }
)

    Describe 'Module Manifest' -Tag 'Manifest' {
        It 'Should exist.' {
            $builtManifest | Should -Not -BeNullOrEmpty
        }
        It 'Should be just one file.' {
            $builtManifest -is [Array] | Should -Be $false
        }
        It 'Should be a valid module manifest.' {
            Test-ModuleManifest -Path $builtManifest.FullName | Should -Be $true
        }
        Context 'Validate settings' {
            BeforeAll {
                $manifestData = Import-PowerShellDataFile $builtManifest.FullName
            }
            It "Should have setting: <Setting>." -TestCases $requiredSettings {
                param ($Setting, $Location)

                if ($Setting -eq 'ReleaseNotes')
                {
                    $manifestData['PrivateData']['PSData'].ContainsKey($Setting) | Should -Be $true

                } else {

                    if ($Location -eq 'root')
                    {
                        $manifestData[$Setting] | Should -Not -BeNullOrEmpty

                    } else {

                        $manifestData['PrivateData']['PSData'][$Setting] | Should -Not -BeNullOrEmpty
                    }
                }
            }
        }
    }

Now launch the build test task.

.\build.ps1 -Tasks Test

The new script file will be automatically picked up due to where it has been saved and it's name *.test.ps1. Output should look something like the following:

Executing script C:\source\MyModule\tests\QA\manifest.tests.ps1

  Describing Module Manifest
    [+] Should exist. 3ms
    [+] Should be just one file. 36ms
    [+] Should be a valid module manifest. 13ms

    Context Validate settings
      [+] Should have setting: Author. 4ms
      [+] Should have setting: CompanyName. 2ms
      [+] Should have setting: Copyright. 2ms
      [+] Should have setting: Description. 4ms
      [+] Should have setting: FunctionsToExport. 3ms
      [+] Should have setting: GUID. 2ms
      [+] Should have setting: ModuleVersion. 2ms
      [+] Should have setting: PowerShellVersion. 2ms
      [+] Should have setting: LicenseUri. 2ms
      [+] Should have setting: ProjectUri. 3ms
      [+] Should have setting: ReleaseNotes. 2ms
      [+] Should have setting: Tags. 3ms

Adding Unit Function Parameter Test

Now we'll add a Pester Unit test to verify function parameter properties haven't changed.

Save the following PowerShell code block to:

C:\source\MyModule\tests\Unit\MyModule.tests.ps1

Content of MyModule.tests.ps1:

$here = Split-Path -Parent $MyInvocation.MyCommand.Path

# Convert-path required for PS7 or Join-Path fails
$ProjectPath = "$here\..\.." | Convert-Path
$ProjectName = (Get-ChildItem $ProjectPath\*\*.psd1 | Where-Object {
    ($_.Directory.Name -match 'source|src' -or $_.Directory.Name -eq $_.BaseName) -and
    $(try { Test-ModuleManifest $_.FullName -ErrorAction Stop }catch{$false}) }
).BaseName

$buildModuleFolder = Join-Path $ProjectPath -ChildPath "output\$ProjectName" -Resolve

$builtManifest = Get-ChildItem $buildModuleFolder -File `
                        -Filter "$ProjectName.psd1" -Recurse `
                        -ErrorAction SilentlyContinue

$manifestData = Import-PowerShellDataFile $builtManifest.FullName

$rootModule = Join-Path $builtManifest.Directory -ChildPath $manifestData.RootModule -Resolve

# Read the built RootModule into memory to test private and public functions.
$script:moduleImport = Import-Module $rootModule -Scope Local -Force -PassThru

# Define test cases. Function parameters will be checked to 
# ensure they exist and are the right type.
$paramsGetPrivateFunction1 = @(
	@{ 'Name' = 'Input1'; 'Type' = [System.String]; }
	@{ 'Name' = 'Input2'; 'Type' = [System.Int32];  }
)

$paramsGetPublicFunction = @(
	@{ 'Name' = 'Input1';            'Type' = [System.String];                                }
	@{ 'Name' = 'Input2';            'Type' = [System.Int32];                                 }
	@{ 'Name' = 'SwitchedParameter'; 'Type' = [System.Management.Automation.SwitchParameter]; }
)

Describe 'Get-PrivateFunction1' {
    Context 'Test Function Parameters' {
        BeforeAll {
            $functionParams = Get-Command -Name Get-PrivateFunction1
        }
        It "Should have parameter: <Name>." -TestCases $paramsGetPrivateFunction1 {
            param ($Name,$Type)

            $functionParams.Parameters.ContainsKey($Name) | Should -Be $true
        }
        It "Parameter '<Name>' should be type <Type>." -TestCases $paramsGetPrivateFunction1 {
            param ($Name, $Type)

            $functionParams.Parameters[$Name].ParameterType -eq $Type | Should -Be $true
        }
    }
}

Describe 'Get-PublicFunction' {
    Context 'Test Function Parameters' {
        BeforeAll {
            $functionParams = Get-Command -Name Get-PublicFunction
        }
        It "Should have parameter: <Name>." -TestCases $paramsGetPublicFunction {
            param ($Name)

            $functionParams.Parameters.ContainsKey($Name) | Should -Be $true
        }
        It "Parameter '<Name>' should be type <Type>." -TestCases $paramsGetPublicFunction {
            param ($Name, $Type)

            $functionParams.Parameters[$Name].ParameterType -eq $Type | Should -Be $true
        }
    }
}

Now launch the build test task.

.\build.ps1 -Tasks Test

The new script file will be automatically picked up due to where it has been saved and it's name *.test.ps1. Output should look something like the following:

Executing script C:\source\MyModule\tests\Unit\MyModule.tests.ps1

  Describing Get-PrivateFunction1

    Context Test Function Parameters
      [+] Should have parameter: Input1. 6ms
      [+] Should have parameter: Input2. 6ms
      [+] Parameter 'Input1' should be type [string]. 4ms
      [+] Parameter 'Input2' should be type [int]. 3ms

  Describing Get-PublicFunction

    Context Test Function Parameters
      [+] Should have parameter: Input1. 16ms
      [+] Should have parameter: Input2. 3ms
      [+] Should have parameter: SwitchedParameter. 3ms
      [+] Parameter 'Input1' should be type [string]. 4ms
      [+] Parameter 'Input2' should be type [int]. 3ms
      [+] Parameter 'SwitchedParameter' should be type [switch]. 2ms

Troubleshooting

There are generally two techniques when troubleshooting tests and each is a build.ps1 parameter.

  • -PesterTag: Pester Tags offer the ability to "Tag" certain tests. Passing a tag name to this parameter will only run tests using the specified tag. One can tag tests actively being worked on to speed up the troubleshooting process. However, each test must be loaded into the environment in order to read the tags. Making this option faster than running a full test but slower than using -PesterScript. Related, there's a -PesterExcludeTag which will run all tests except for the specified tag.
  • -PesterScript: Specify which test file to run via its path from the project root. While all tests in the file will run, it is the only file loaded into the environment. Making it the fastest method for testing changes. Using a test script from above, here's an example:
    .\build.ps1 -Tasks build,test -PesterScript \tests\Unit\MyModule.tests.ps1

Pipeline & Tasks

Pipeline and Tasks are configured via build.yaml under the section BuildWorkFlow:. The default task, defined by '.', gets invoked when launching .\build.ps1 without specifying anything for -Tasks.

Tasks can be added/removed to satisfy pipeline requirements. Run the following command to list all available tasks:

.\build.ps1 -Tasks ?

Output should look similar to the following:

...
[build] Executing requested workflow: ?

Name                               Jobs   Synopsis
----                               ----   --------
Build_ModuleOutput_ModuleBuilder   {}     Build the Module based on its Build.psd1 definition
Clean                              {}     Deleting the content of the Build Output folder, except ./modules
Invoke_DscResource_Tests           {}     Making sure the Module meets some quality standard (help, tests)
...

If an error is returned (below), comment out the erroring task from build.yaml. Here the Publish_GitHub_Wiki_Content task will be commented out from build.yaml. If the task is desired, refer to Configure Builtin Tasks.

ERROR: Task 'publish': Missing task 'Publish_GitHub_Wiki_Content'.
At C:\source\MyModule\build.ps1:295 char:13
+             task $workflow $workflowItem
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\source\MyModule\output\RequiredModules\InvokeBuild\5.8.0\Invoke-Build.ps1 : Task 'publish': Missing task 'Publish_GitHub_Wiki_Content'.
At C:\source\MyModule\build.ps1:295 char:13
+             task $workflow $workflowItem
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Add Custom Task

Having the ability to create Custom Tasks is very helpful as it allows Sampler to adapt to any environment. There are two techniques for running custom tasks:

  1. build.yaml
  2. Script in .build Directory

Add Custom Task - build.yaml

When doing things on an adhoc basis, updating build.yaml with a custom task is likely the way to go. It involves editing the BuildWorkFlow: section of build.yaml with your custom task. Suppose we want to print the $PSVersionTable variable during a build. The following build.yaml snippet shows how this can be done. First by defining a new task Print_PS_Version_Table and then adding it to the build task.

####################################################
#       Sampler Pipeline Configuration             #
####################################################
BuildWorkflow:
  '.':
    - build
    - test
  # New Task
  Print_PS_Version_Table: |
    { Write-Host '[Print_PS_Version_Table] Starting.' -ForeGroundColor Green -BackgroundColor Black
      Write-Host "[Print_PS_Version_Table] `n$(($PSVersionTable | Out-String).Trim())" -ForeGroundColor Green -BackgroundColor Black
      Write-Host '[Print_PS_Version_Table] Finished.' -ForeGroundColor Green -BackgroundColor Black }

  build:
    - Clean
    - Build_Module_ModuleBuilder
    - Build_NestedModules_ModuleBuilder
    - Create_changelog_release_output
    - Print_PS_Version_Table             # <- added to Build Task

  pack:
    - build
    - package_module_nupkg

  hqrmtest:
    - DscResource_Tests_Stop_On_Fail

  test:
    - Pester_Tests_Stop_On_Fail
    - Pester_if_Code_Coverage_Under_Threshold
    - Invoke_DscResource_Tests

  publish:
    - publish_module_to_gallery
    - Publish_Release_To_GitHub
    - Create_ChangeLog_GitHub_PR

Save the file and launch a build.

.\build.ps1 -Tasks build

The last task of the build process should produce the following output. Notice it references the task (/build/Print_PS_Version_Table) and source (C:\source\MyModule\build.ps1:295).

===============================================================================
                        PRINT PS VERSION TABLE

-------------------------------------------------------------------------------
  /build/Print_PS_Version_Table
  C:\source\MyModule\build.ps1:295

[Print_PS_Version_Table] Starting.
[Print_PS_Version_Table]
Name                           Value
----                           -----
PSVersion                      5.1.17763.1852
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.1852
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
[Print_PS_Version_Table] Finished.
Done /build/Print_PS_Version_Table 00:00:00.0312510

Add Custom Task - Script in .build Directory

Using the prior example of printing $PSVersionTable, we'll convert it to a custom task as a script in the .build directory. The Sampler build script will automatically pick up scripts saved in this directory using the naming convention *.build.ps1.

First, remove the task from build.yaml but KEEP the reference under the build task. The following snippet of build.yaml has the adhoc task commented out.

####################################################
#       Sampler Pipeline Configuration             #
####################################################
BuildWorkflow:
  '.':
    - build
    - test
#  # New Task
#  Print_PS_Version_Table: |
#    { Write-Host '[Print_PS_Version_Table] Starting.' -ForeGroundColor Green -BackgroundColor Black
#      Write-Host "[Print_PS_Version_Table] `n$(($PSVersionTable | Out-String).Trim())" -ForeGroundColor Green -BackgroundColor Black
#      Write-Host '[Print_PS_Version_Table] Finished.' -ForeGroundColor Green -BackgroundColor Black }

  build:
    - Clean
    - Build_Module_ModuleBuilder
    - Build_NestedModules_ModuleBuilder
    - Create_changelog_release_output
    - Print_PS_Version_Table             # <- KEEP

  pack:
    - build
    - package_module_nupkg

  hqrmtest:
    - DscResource_Tests_Stop_On_Fail

  test:
    - Pester_Tests_Stop_On_Fail
    - Pester_if_Code_Coverage_Under_Threshold
    - Invoke_DscResource_Tests

  publish:
    - publish_module_to_gallery
    - Publish_Release_To_GitHub
    - Create_ChangeLog_GitHub_PR

Create the custom script by running the following PowerShell commands. Just copy/paste into a prompt

$fileContent = @'
task Print_PS_Version_Table {

    Write-Host '[Print_PS_Version_Table] Starting.' -ForeGroundColor Green -BackgroundColor Black
    Write-Host "[Print_PS_Version_Table] `n$(($PSVersionTable | Out-String).Trim())" -ForeGroundColor Green -BackgroundColor Black
    Write-Host '[Print_PS_Version_Table] Finished.' -ForeGroundColor Green -BackgroundColor Black
}
'@

New-Item -Path 'C:\source\MyModule\.build\' `
         -Name 'Print_PS_Version_Table.build.ps1' `
	 -ItemType File `
	 -Value $fileContent -Force

Now run the build task.

.\build.ps1 -Tasks build

Output should look like the following. Notice it references the task (/build/Print_PS_Version_Table) and source (C:\source\MyModule\.build\Print_PS_Version_Table.build.ps1:1).

===============================================================================
                        PRINT PS VERSION TABLE

-------------------------------------------------------------------------------
  /build/Print_PS_Version_Table
  C:\source\MyModule\.build\Print_PS_Version_Table.build.ps1:1

[Print_PS_Version_Table] Starting.
[Print_PS_Version_Table]
Name                           Value
----                           -----
PSVersion                      5.1.17763.1852
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17763.1852
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
[Print_PS_Version_Table] Finished.
Done /build/Print_PS_Version_Table 00:00:00.0376541

Configure Builtin Tasks

Sampler has many tasks provided by default which can be displayed by running .\build.ps1 -Tasks ?. Some tasks require additional configuration and new tasks can be added by including the appropriate module.

This section will walkthrough configuring MyModule to build and publish wiki content. Save the following to C:\source\MyModule\source\WikiSource\Home.md. The build process (specifically Generate_Wiki_Content task) will replace #.#.# with the build version.

# MyModule - #.#.# #

Welcome to `MyModule` v#.#.# wiki pages!

Update Required Modules

The first step is to update RequiredModules.psd1 to include the appropriate module. In this case it will be adding 'DscResource.DocGenerator'. Below is a snippet of this file which may be different from your own.

    InvokeBuild                 = 'latest'
    PSScriptAnalyzer            = 'latest'
    Pester                      = '4.10.1'
    Plaster                     = 'latest'
    ModuleBuilder               = 'latest'
    ChangelogManagement         = 'latest'
    Sampler                     = 'latest'
    'Sampler.GitHubTasks'       = 'latest'
    'DscResource.DocGenerator'  = 'latest'     # <- Add

Save the file and run the noop task to download the newly added dependency.

.\build.ps1 -ResolveDependency -Tasks noop

Update build.yaml

There are two areas to update:

  • ModuleBuildTasks: - Instructs the Sampler build script where to import tasks for use in BuildWorkFlow:.
  • BuildWorkFlow: - Defines where the InvokeBuild tasks should run in the pipeline.

Update build.yaml - ModuleBuildTasks:

To import Tasks provided by DscResource.DocGenerator, update ModuleBuildTasks: as follows:

ModuleBuildTasks:
  Sampler:
    - '*.build.Sampler.ib.tasks'
  Sampler.GitHubTasks:
    - '*.ib.tasks'
  DscResource.DocGenerator:       # <- Add
    - 'Task.*'                    # <- Add

Update build.yaml - BuildWorkFlow:

With tasks imported, they can now be assigned to the appropriate BuildWorkFlow:. Here we'll add two tasks. The first is adding Generate_Wiki_Content to the build workflow and then add Publish_GitHub_Wiki_Content to the publish workflow. The following is an example snippet.

BuildWorkflow:
  '.':
    - build

  build:
    - Clean
    - Build_Module_ModuleBuilder
    - Build_NestedModules_ModuleBuilder
    - Create_changelog_release_output
    - Generate_Wiki_Content                 # <- Add

  publish:
    - Publish_GitHub_Wiki_Content           # <- Add

Launch Build Task

Save the newly edited build.yaml and run the build task to generate the wiki page.

.\build.ps1 -Tasks build

The newly built file will be C:\source\MyModule\output\WikiContent\Home.md having the following or similar contents. Note the version number could be different.

# MyModule - 0.0.1 #

Welcome to `MyModule` v0.0.1 wiki pages!

Set $GitHubToken

Finally, the Publish_GitHub_Wiki_Content task will only run if the variable $GitHubToken is set to a GitHub Personal Access Token (PAT) either in parent scope, as an environment variable, or if passed to the build task.

The minimum permission for a PAT to publish wiki content is public_repo.

DscCommunity recommends creating a PAT with the following permissions as then it can manage packages (source).

[ ] repo			Full control of private repositories
    [ ] repo:status		Access commit status
    [ ] repo_deployment		Access deployment status
    [x] public_repo		Access public repositories
    [ ] repo:invite		Access repository invitations
    [ ] security_events		Read and write security events
[x] write:packages		Upload packages to GitHub Package Registry
    [x] read:packages		Download packages from GitHub Package Registry
[x] delete:packages		Delete packages from GitHub Package Registry 

Setting $GitHubToken

If testing locally, simply set the variable as follows. Note this is not secure so consider making a separate token just for this short term use.

$env:GitHubToken = 'some-token'

When using Azure Pipelines, refer to the DscCommunity instructions found here.

Security Considerations

There are security controls one must consider when using a PAT since they cannot be restricted to a specific repository. This means a compromised PAT will have equal access to all repos within an organization. For this reason, consider protecting the main/master branch using branch protection rules. It is a way to disable force pushing, prevent branches from being deleted, and optionally require status checks before merging.

While not applicable to this module, consider using Deploy Keys where possible. They are repo specific and can be configured as ReadOnly or with Write capability.

@johlju
Copy link

johlju commented May 2, 2021

To use the Publish_GitHub_Wiki_Content the module DscResource.DocGenerator need to be added and configured in build.yml. It can be use for regular modules too. I started a blog post for it a long time ago but haven't finished it yet. Though only the part that uploads the folder WIkiSource can be used. @gaelcolas was working on a PlatyPS task that might be used instead - not sure if it is ready.

@johlju
Copy link

johlju commented May 3, 2021

I threw together this module that you can use for reference if you like: https://github.com/viscalyx/PSXml

@phbits
Copy link
Author

phbits commented May 4, 2021

@johlju Thanks for the feedback! I've updated the gist and will take a closer look at PSXml. Also looking into adding custom build tasks.

@phbits
Copy link
Author

phbits commented May 5, 2021

To use the Publish_GitHub_Wiki_Content the module DscResource.DocGenerator need to be added and configured in build.yml. It can be use for regular modules too. I started a blog post for it a long time ago but haven't finished it yet. Though only the part that uploads the folder WIkiSource can be used. @gaelcolas was working on a PlatyPS task that might be used instead - not sure if it is ready.

Thanks for the input! I documented this task here.

To prevent this error, does it make sense to update build.yaml in the SimpleModule template to import tasks from DscResource.DocGenerator? The module is already listed in RequiredModules.psd1.

@johlju
Copy link

johlju commented May 6, 2021

I added gaelcolas/Sampler#277 (comment) to track something need to be done with that task for the template SimpleModule. I'm not sure it brings any value to configure it for a regular module today. What do you think?

The functionality of DscResource.DocGenerator:

  • Task Generate_Conceptual_Help - creates conceptual help for MOF- and class-based resources in output/MyModule/**/en-US (MOF-resources has individual en-US folders, e.g. output\SqlServerDsc\15.2.0\DSCResources\DSC_SqlAG\en-US\about_SqlAG.help.txt).
  • Task Generate_Wiki_Content - generates markdown documentation in output/WikiContent for MOF- and class-based resources. But also copies the markdown files from folder source/WikiSource to output/WikiContent. See example here: https://github.com/dsccommunity/SqlServerDsc/tree/main/source/WikiSource
    • Changes all placeholders (#.#.#) in a markdown file Home.md to the built module version number.
  • Task Publish_GitHub_Wiki_Content - Publishes the content in output/WikiContent (preferably in pipeline Deploy step) to the GitHub repository Wiki. Need to have a PAT available as secret environment variable in the pipeline to have permission to change GitHub repository Wiki. For more information, see Add access tokens to an Azure Pipeline.

So the only value Generate_Wiki_Content and Publish_GitHub_Wiki_Content would bring is being able to publish content from source/WikiSource.

What we need is a build task that generates markdown help for commands, for example based on the comment-based help in each individual script file (in Public folder). Build task should output markdown to output/WIkiContent. Maybe it could look like this, which is manually updated due to not having this build task yet: https://github.com/dsccommunity/DscResource.Common#cmdlets

@DrIOSX
Copy link

DrIOSX commented Apr 25, 2022

phbits

Firstly, thank you for the tutorial!
I'm getting an error when building.

No valid manifest found for project 'myproject'. Cannot update the Release Notes.

The manifest is in the source directory. There's an issue on sampler's GitHub about this:
gaelcolas/Sampler#358

With as much work as you've done, I would think it'd be likely that you have a workaround? /hoping

Any ideas or alternatives? I'm stumped.

I want to build a module and upload to the PWSH gallery automatically with some basic building so I can try and follow best practices. ^_^

@phbits
Copy link
Author

phbits commented Apr 28, 2022

Hi @DrIOSX! I don't recall a workaround for that off-hand nor it being an issue. It does appear the module manifest is hardcoded to look for modulename.psd1 (e.g. myproject.psd1) and it must reside in the module base path. Beyond this, I would need to see the build logs and do some testing as I've switched gears since working on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment