Skip to content

Instantly share code, notes, and snippets.

@dfinke
Created December 26, 2016 14:28
Show Gist options
  • Save dfinke/3ec097addcb83871e1d92c1ff74f981c to your computer and use it in GitHub Desktop.
Save dfinke/3ec097addcb83871e1d92c1ff74f981c to your computer and use it in GitHub Desktop.
PowerShell script to scrape US Government sites for members
function Get-Government {
param(
$name,
$state
)
$(
Get-USSenators | select @{n="name";e={$_.member_full}}, party, state, @{n="type";e={"Senate"}}
Get-USCongress | select @{n="name";e={$_.namelist}}, party, state, @{n="type";e={"House"}}
) |
Where {
$_.'name' -match $name -and
$_.'state' -match $state
}
}
function Get-USSenators {
param(
$name,
$state
)
$r=irm https://www.senate.gov/general/contact_information/senators_cfm.xml
$properties = echo member_full last_name first_name party state address phone email website class bioguide_id
$r.contact_information.member |
select $properties |
Where {
$_.'member-full' -match $name -or
$_.'last_name' -match $name -or
$_.'first_name' -match $name -and
$_.'state' -match $state
}
}
function Get-USCongress {
param(
$name,
$state
)
$r=irm http://clerk.house.gov/member_info/unofficial-115-member-elect-data.xml
$properties = echo namelist bioguideID lastname firstname middlename sort-name suffix courtesy prior-congress official-name formal-name party caucus state district townname office-building office-room office-zip office-zip-suffix phone
$r.MemberData.members.member.'member-info' |
foreach {
[PSCustomObject]@{
namelist=$_.namelist
bioguideID=$_.bioguideID
lastname=$_.lastname
firstname=$_.firstname
middlename=$_.middlename
'sort-name'=$_.'sort-name'
suffix=$_.suffix
courtesy=$_.courtesy
'prior-congress'=$_.'prior-congress'
'official-name'=$_.'official-name'
'formal-name'=$_.'formal-name'
party=$_.party
caucus=$_.caucus
state=$_.state.'postal-code'
district=$_.district
townname=$_.townname
'office-building'=$_.'office-building'
'office-room'=$_.'office-room'
'office-zip'=$_.'office-zip'
'office-zip-suffix'=$_.'office-zip-suffix'
phone=$_.phone
}
} |
Where {
$_.'namelist' -match $name -or
$_.'lastname' -match $name -or
$_.'firstname' -match $name -and
$_.'state' -match $state
}
}
@dfinke
Copy link
Author

dfinke commented Dec 26, 2016

Use the above PowerShell and Export-Excel to do data analysis on Congress.

$f="c:\temp\gov.xlsx"
rm $f -ErrorAction Ignore

Get-Government | sort state |
    Export-Excel $f `
        -Show `
        -AutoSize -AutoFilter `
        -IncludePivotTable -PivotRows State -PivotData @{Party='count'} -PivotColumns Type, Party        

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment