Skip to content

Instantly share code, notes, and snippets.

@gdsaxton

gdsaxton/IRS 990 e-File Data (8) -- Read in Electronic 990 Data to PANDAS, Wrangle Data, and Export to CSV and Stata.ipynb

Last active October 6, 2020 21:24
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save gdsaxton/448dd1b67e9c09535ff76bb700225930 to your computer and use it in GitHub Desktop.
Save gdsaxton/448dd1b67e9c09535ff76bb700225930 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
IRS 990 e-File Data (8) -- Read in Electronic 990 Data to PANDAS, Wrangle Data, and Export to CSV and Stata.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Overview\n",
"\n",
"This is the eighth and final in a series tutorials that illustrate how to download the <a href=\"https://aws.amazon.com/public-data-sets/irs-990/\">IRS 990 e-file data</a>. The goal of this tutorial will be to show you how to get a useable 990 dataset you can export to *Stata* or *R*, etc., to use for statistical analyses.\n",
"\n",
"<a href=\"http://social-metrics.org/irs-990-e-file-data-part-5/\">Previous tutorials</a> have shown how to download data from IRS 990, 990PF, and 990EZ filings into a MongoDB database. We also downloaded the *schemas* for the 990 filings in order to generate a codebook or data dictionary. \n",
"\n",
"To get a working dataset, in this tutorial we will do five things: 1) import the downloaded 990 data into PANDAS, 2) load the codebook, 3) use the codebook to identify relevant variables, 4) wrangle the data as necessary, and 5) export the data in *Stata* and CSV formats."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Load Packages"
]
},
{
"cell_type": "code",
"execution_count": 335,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import sys\n",
"import time\n",
"import json"
]
},
{
"cell_type": "code",
"execution_count": 336,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"from pandas import DataFrame\n",
"from pandas import Series"
]
},
{
"cell_type": "code",
"execution_count": 337,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"#http://pandas.pydata.org/pandas-docs/stable/options.html\n",
"pd.set_option('display.max_columns', None)\n",
"pd.set_option('max_colwidth', 250)"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"#### Set working directory"
]
},
{
"cell_type": "code",
"execution_count": 338,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"/Users/gsaxton/Dropbox/990 e-file data\n"
]
}
],
"source": [
"cd '/Users/gsaxton/Dropbox/990 e-file data'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Connect to MongoDB\n",
"Depending on the project, I will store the data in SQLite or MongoDB. This time I'll use MongoDB -- it's great for storing JSON data where each observation could have different variables. Before we get to the interesting part the following code blocks set up the MongoDB environment and the new database we'll be using. \n",
"\n",
"**_Note:_** In a terminal we'll have to start MongoDB by running the command *mongod* or *sudo mongod*. Then we run the following code block here to access MongoDB."
]
},
{
"cell_type": "code",
"execution_count": 339,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import pymongo\n",
"from pymongo import MongoClient\n",
"client = MongoClient()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Connect to the database where we stored our 990 data."
]
},
{
"cell_type": "code",
"execution_count": 340,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# DEFINE MY mongoDB DATABASE\n",
"db = client['irs_990_db']\n",
"\n",
"# DEFINE MY COLLECTION HOUSING 990 DATA\n",
"filings = db['filings_test']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Check how many observations in the database table.\n"
]
},
{
"cell_type": "code",
"execution_count": 341,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"33"
]
},
"execution_count": 341,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"filings.count()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Let's re-create our list of five EINs from previous tutorials."
]
},
{
"cell_type": "code",
"execution_count": 342,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"eins = ['010202467', '010211478', '010211513', '010211530', '010211543']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>All 33 filings should have one of these EINs. Let's check."
]
},
{
"cell_type": "code",
"execution_count": 343,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"33"
]
},
"execution_count": 343,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"filings.find({'EIN': { '$in': eins}}).count()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Read 990 DB into PANDAS DF\n",
"- I'm taking all 33 990 e-filings for our 5 EINs and importing them from MongoDB into a Python PANDAS dataframe for manipulation.\n",
"- As you can see if you scroll horizontally through this sample row, there are 607 columns. Lots of data!"
]
},
{
"cell_type": "code",
"execution_count": 502,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Number of columns: 607\n",
"Number of observations: 33\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>@documentCnt</th>\n",
" <th>@documentCount</th>\n",
" <th>@documentId</th>\n",
" <th>@referenceDocumentId</th>\n",
" <th>@returnVersion</th>\n",
" <th>@xmlns</th>\n",
" <th>@xmlns:xsi</th>\n",
" <th>@xsi:schemaLocation</th>\n",
" <th>AccountantCompileOrReview</th>\n",
" <th>AccountantCompileOrReviewBasis</th>\n",
" <th>AccountantCompileOrReviewInd</th>\n",
" <th>AccountsPayableAccrExpnssGrp</th>\n",
" <th>AccountsPayableAccruedExpenses</th>\n",
" <th>AccountsReceivable</th>\n",
" <th>AccountsReceivableGrp</th>\n",
" <th>ActivitiesConductedPartnership</th>\n",
" <th>ActivitiesConductedPrtshpInd</th>\n",
" <th>Activity2</th>\n",
" <th>Activity3</th>\n",
" <th>ActivityCode</th>\n",
" <th>ActivityOrMissionDesc</th>\n",
" <th>ActivityOrMissionDescription</th>\n",
" <th>ActivityOther</th>\n",
" <th>AddressPrincipalOfficerUS</th>\n",
" <th>Advertising</th>\n",
" <th>AdvertisingGrp</th>\n",
" <th>AllOtherContributions</th>\n",
" <th>AllOtherContributionsAmt</th>\n",
" <th>AllOtherExpenses</th>\n",
" <th>AllOtherExpensesGrp</th>\n",
" <th>AnnualDisclosureCoveredPersons</th>\n",
" <th>AnnualDisclosureCoveredPrsnInd</th>\n",
" <th>AuditCommittee</th>\n",
" <th>AuditCommitteeInd</th>\n",
" <th>BackupWthldComplianceInd</th>\n",
" <th>BalanceSheetAmountsReported</th>\n",
" <th>BenefitsPaidToMembersCY</th>\n",
" <th>BenefitsPaidToMembersPriorYear</th>\n",
" <th>BenefitsToMembers</th>\n",
" <th>BenefitsToMembersGrp</th>\n",
" <th>BooksInCareOfDetail</th>\n",
" <th>BsnssRltnshpThruFamilyMember</th>\n",
" <th>BsnssRltnshpWithOrganization</th>\n",
" <th>BusinessRlnWithFamMemInd</th>\n",
" <th>BusinessRlnWithOfficerEntInd</th>\n",
" <th>BusinessRlnWithOrgMemInd</th>\n",
" <th>CYBenefitsPaidToMembersAmt</th>\n",
" <th>CYContributionsGrantsAmt</th>\n",
" <th>CYGrantsAndSimilarPaidAmt</th>\n",
" <th>CYInvestmentIncomeAmt</th>\n",
" <th>CYOtherExpensesAmt</th>\n",
" <th>CYOtherRevenueAmt</th>\n",
" <th>CYProgramServiceRevenueAmt</th>\n",
" <th>CYRevenuesLessExpensesAmt</th>\n",
" <th>CYSalariesCompEmpBnftPaidAmt</th>\n",
" <th>CYTotalExpensesAmt</th>\n",
" <th>CYTotalFundraisingExpenseAmt</th>\n",
" <th>CYTotalProfFndrsngExpnsAmt</th>\n",
" <th>CYTotalRevenueAmt</th>\n",
" <th>CashNonInterestBearing</th>\n",
" <th>CashNonInterestBearingGrp</th>\n",
" <th>ChangeToOrgDocumentsInd</th>\n",
" <th>ChangesToOrganizingDocs</th>\n",
" <th>CntrbtnsRprtdFundraisingEvents</th>\n",
" <th>CntrctRcvdGreaterThan100KCnt</th>\n",
" <th>CollectionsOfArt</th>\n",
" <th>CollectionsOfArtInd</th>\n",
" <th>CompCurrentOfcrDirectorsGrp</th>\n",
" <th>CompCurrentOfficersDirectors</th>\n",
" <th>CompDisqualPersons</th>\n",
" <th>CompDisqualPersonsGrp</th>\n",
" <th>CompensationFromOtherSources</th>\n",
" <th>CompensationFromOtherSrcsInd</th>\n",
" <th>CompensationProcessCEO</th>\n",
" <th>CompensationProcessCEOInd</th>\n",
" <th>CompensationProcessOther</th>\n",
" <th>CompensationProcessOtherInd</th>\n",
" <th>ComplianceWithBackupWitholding</th>\n",
" <th>ConferencesMeetings</th>\n",
" <th>ConferencesMeetingsGrp</th>\n",
" <th>ConflictOfInterestPolicy</th>\n",
" <th>ConflictOfInterestPolicyInd</th>\n",
" <th>ConservationEasements</th>\n",
" <th>ConservationEasementsInd</th>\n",
" <th>ConsolidatedAuditFinancialStmt</th>\n",
" <th>ConsolidatedAuditFinclStmtInd</th>\n",
" <th>ContractorCompensation</th>\n",
" <th>ContractorCompensationGrp</th>\n",
" <th>ContriRptFundraisingEventAmt</th>\n",
" <th>ContributionsGrantsCurrentYear</th>\n",
" <th>ContributionsGrantsPriorYear</th>\n",
" <th>CostOfGoodsSold</th>\n",
" <th>CostOfGoodsSoldAmt</th>\n",
" <th>CreditCounseling</th>\n",
" <th>CreditCounselingInd</th>\n",
" <th>DLN</th>\n",
" <th>DecisionsSubjectToApprovaInd</th>\n",
" <th>DecisionsSubjectToApproval</th>\n",
" <th>DeductibleArtContributionInd</th>\n",
" <th>DeductibleContributionsOfArt</th>\n",
" <th>DeductibleNonCashContriInd</th>\n",
" <th>DeductibleNonCashContributions</th>\n",
" <th>DeferredRevenue</th>\n",
" <th>DeferredRevenueGrp</th>\n",
" <th>DelegationOfManagementDuties</th>\n",
" <th>DelegationOfMgmtDutiesInd</th>\n",
" <th>DepreciationDepletion</th>\n",
" <th>DepreciationDepletionGrp</th>\n",
" <th>Desc</th>\n",
" <th>DescribedIn501C3</th>\n",
" <th>DescribedInSection501c3Ind</th>\n",
" <th>Description</th>\n",
" <th>DisregardedEntity</th>\n",
" <th>DisregardedEntityInd</th>\n",
" <th>DistributionToDonor</th>\n",
" <th>DocumentRetentionPolicy</th>\n",
" <th>DocumentRetentionPolicyInd</th>\n",
" <th>DonatedServicesAndUseFcltsAmt</th>\n",
" <th>DonorAdvisedFundInd</th>\n",
" <th>DonorAdvisedFunds</th>\n",
" <th>EIN</th>\n",
" <th>ElectionOfBoardMembers</th>\n",
" <th>ElectionOfBoardMembersInd</th>\n",
" <th>EmployeeCnt</th>\n",
" <th>EmploymentTaxReturnsFiled</th>\n",
" <th>EmploymentTaxReturnsFiledInd</th>\n",
" <th>EngagedInExcessBenefitTransInd</th>\n",
" <th>EscrowAccount</th>\n",
" <th>EscrowAccountInd</th>\n",
" <th>EscrowAccountLiability</th>\n",
" <th>EscrowAccountLiabilityGrp</th>\n",
" <th>ExcessBenefitTransaction</th>\n",
" <th>Expense</th>\n",
" <th>ExpenseAmt</th>\n",
" <th>FSAudited</th>\n",
" <th>FSAuditedBasis</th>\n",
" <th>FSAuditedBasisGrp</th>\n",
" <th>FSAuditedInd</th>\n",
" <th>FamilyOrBusinessRelationship</th>\n",
" <th>FamilyOrBusinessRlnInd</th>\n",
" <th>FederalGrantAuditPerformed</th>\n",
" <th>FederalGrantAuditPerformedInd</th>\n",
" <th>FederalGrantAuditRequired</th>\n",
" <th>FederalGrantAuditRequiredInd</th>\n",
" <th>FederatedCampaigns</th>\n",
" <th>FederatedCampaignsAmt</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" <th>FeesForServicesInvstMgmntFees</th>\n",
" <th>FeesForServicesLegal</th>\n",
" <th>FeesForServicesLegalGrp</th>\n",
" <th>FeesForServicesLobbying</th>\n",
" <th>FeesForServicesLobbyingGrp</th>\n",
" <th>FeesForServicesManagement</th>\n",
" <th>FeesForServicesManagementGrp</th>\n",
" <th>FeesForServicesOther</th>\n",
" <th>FeesForServicesOtherGrp</th>\n",
" <th>FeesForServicesProfFundraising</th>\n",
" <th>FeesForSrvcInvstMgmntFeesGrp</th>\n",
" <th>FinancialStatementConsolidated</th>\n",
" <th>FinancialStatementSeparate</th>\n",
" <th>FollowSFAS117</th>\n",
" <th>ForeignActivities</th>\n",
" <th>ForeignActivitiesInd</th>\n",
" <th>ForeignFinancialAccount</th>\n",
" <th>ForeignFinancialAccountInd</th>\n",
" <th>ForeignGrants</th>\n",
" <th>ForeignGrantsGrp</th>\n",
" <th>ForeignOffice</th>\n",
" <th>ForeignOfficeInd</th>\n",
" <th>Form8282PropertyDisposedOf</th>\n",
" <th>Form8282PropertyDisposedOfInd</th>\n",
" <th>Form990-TFiled</th>\n",
" <th>Form990PartVIISectionA</th>\n",
" <th>Form990PartVIISectionAGrp</th>\n",
" <th>Form990ProvidedToGoverningBody</th>\n",
" <th>Form990ProvidedToGvrnBodyInd</th>\n",
" <th>Form990TFiledInd</th>\n",
" <th>FormType</th>\n",
" <th>FormationYr</th>\n",
" <th>FormerOfcrEmployeesListedInd</th>\n",
" <th>FormersListed</th>\n",
" <th>FundraisingActivities</th>\n",
" <th>FundraisingActivitiesInd</th>\n",
" <th>FundraisingAmt</th>\n",
" <th>FundraisingDirectExpenses</th>\n",
" <th>FundraisingDirectExpensesAmt</th>\n",
" <th>FundraisingEvents</th>\n",
" <th>FundraisingGrossIncomeAmt</th>\n",
" <th>FundsToPayPremiums</th>\n",
" <th>GainOrLoss</th>\n",
" <th>GainOrLossGrp</th>\n",
" <th>Gaming</th>\n",
" <th>GamingActivitiesInd</th>\n",
" <th>GamingDirectExpensesAmt</th>\n",
" <th>GamingGrossIncomeAmt</th>\n",
" <th>GoverningBodyVotingMembersCnt</th>\n",
" <th>GovernmentGrants</th>\n",
" <th>GovernmentGrantsAmt</th>\n",
" <th>GrantAmt</th>\n",
" <th>GrantToRelatedPerson</th>\n",
" <th>GrantToRelatedPersonInd</th>\n",
" <th>Grants</th>\n",
" <th>GrantsAndSimilarAmntsCY</th>\n",
" <th>GrantsAndSimilarAmntsPriorYear</th>\n",
" <th>GrantsPayable</th>\n",
" <th>GrantsPayableGrp</th>\n",
" <th>GrantsToDomesticIndividuals</th>\n",
" <th>GrantsToDomesticIndividualsGrp</th>\n",
" <th>GrantsToDomesticOrgs</th>\n",
" <th>GrantsToDomesticOrgsGrp</th>\n",
" <th>GrantsToIndividuals</th>\n",
" <th>GrantsToIndividualsInd</th>\n",
" <th>GrantsToOrganizations</th>\n",
" <th>GrantsToOrganizationsInd</th>\n",
" <th>GrossAmountSalesAssets</th>\n",
" <th>GrossAmountSalesAssetsGrp</th>\n",
" <th>GrossIncomeFundraisingEvents</th>\n",
" <th>GrossIncomeGaming</th>\n",
" <th>GrossReceipts</th>\n",
" <th>GrossReceiptsAmt</th>\n",
" <th>GrossRents</th>\n",
" <th>GrossRentsGrp</th>\n",
" <th>GrossSalesOfInventory</th>\n",
" <th>GrossSalesOfInventoryAmt</th>\n",
" <th>GroupReturnForAffiliates</th>\n",
" <th>GroupReturnForAffiliatesInd</th>\n",
" <th>Hospital</th>\n",
" <th>IRPDocumentCnt</th>\n",
" <th>IRPDocumentW2GCnt</th>\n",
" <th>IRS990ScheduleA</th>\n",
" <th>IRS990ScheduleB</th>\n",
" <th>IRS990ScheduleC</th>\n",
" <th>IRS990ScheduleD</th>\n",
" <th>IRS990ScheduleF</th>\n",
" <th>IRS990ScheduleG</th>\n",
" <th>IRS990ScheduleI</th>\n",
" <th>IRS990ScheduleJ</th>\n",
" <th>IRS990ScheduleK</th>\n",
" <th>IRS990ScheduleL</th>\n",
" <th>IRS990ScheduleM</th>\n",
" <th>IRS990ScheduleO</th>\n",
" <th>IRS990ScheduleR</th>\n",
" <th>IncludeFIN48FootnoteInd</th>\n",
" <th>IncmFromInvestBondProceedsGrp</th>\n",
" <th>IncomeFromInvestBondProceeds</th>\n",
" <th>IndependentAuditFinancialStmt</th>\n",
" <th>IndependentAuditFinclStmtInd</th>\n",
" <th>IndependentVotingMemberCnt</th>\n",
" <th>IndivRcvdGreaterThan100KCnt</th>\n",
" <th>IndoorTanningServices</th>\n",
" <th>IndoorTanningServicesInd</th>\n",
" <th>InfoInScheduleOPartIII</th>\n",
" <th>InfoInScheduleOPartIIIInd</th>\n",
" <th>InfoInScheduleOPartIX</th>\n",
" <th>InfoInScheduleOPartIXInd</th>\n",
" <th>InfoInScheduleOPartVI</th>\n",
" <th>InfoInScheduleOPartVIIInd</th>\n",
" <th>InfoInScheduleOPartVIInd</th>\n",
" <th>InfoInScheduleOPartX</th>\n",
" <th>InfoInScheduleOPartXI</th>\n",
" <th>InfoInScheduleOPartXII</th>\n",
" <th>InfoInScheduleOPartXIIInd</th>\n",
" <th>InfoInScheduleOPartXIInd</th>\n",
" <th>InformationTechnology</th>\n",
" <th>InformationTechnologyGrp</th>\n",
" <th>Insurance</th>\n",
" <th>InsuranceGrp</th>\n",
" <th>IntangibleAssets</th>\n",
" <th>IntangibleAssetsGrp</th>\n",
" <th>Interest</th>\n",
" <th>InterestGrp</th>\n",
" <th>InventoriesForSaleOrUse</th>\n",
" <th>InventoriesForSaleOrUseGrp</th>\n",
" <th>InvestTaxExemptBonds</th>\n",
" <th>InvestTaxExemptBondsInd</th>\n",
" <th>InvestmentInJointVenture</th>\n",
" <th>InvestmentInJointVentureInd</th>\n",
" <th>InvestmentIncome</th>\n",
" <th>InvestmentIncomeCurrentYear</th>\n",
" <th>InvestmentIncomeGrp</th>\n",
" <th>InvestmentIncomePriorYear</th>\n",
" <th>InvestmentsOtherSecurities</th>\n",
" <th>InvestmentsOtherSecuritiesGrp</th>\n",
" <th>InvestmentsProgramRelated</th>\n",
" <th>InvestmentsProgramRelatedGrp</th>\n",
" <th>InvestmentsPubTradedSecGrp</th>\n",
" <th>InvestmentsPubTradedSecurities</th>\n",
" <th>LandBldgEquipAccumDeprecAmt</th>\n",
" <th>LandBldgEquipBasisNetGrp</th>\n",
" <th>LandBldgEquipCostOrOtherBssAmt</th>\n",
" <th>LandBldgEquipmentAccumDeprec</th>\n",
" <th>LandBuildingsEquipmentBasis</th>\n",
" <th>LandBuildingsEquipmentBasisNet</th>\n",
" <th>LastUpdated</th>\n",
" <th>LegalDomicileStateCd</th>\n",
" <th>LessCostOthBasisSalesExpenses</th>\n",
" <th>LessCostOthBasisSalesExpnssGrp</th>\n",
" <th>LessRentalExpenses</th>\n",
" <th>LessRentalExpensesGrp</th>\n",
" <th>LoanOutstandingInd</th>\n",
" <th>LoanToOfficerOrDQP</th>\n",
" <th>LoansFromOfficersDirectors</th>\n",
" <th>LoansFromOfficersDirectorsGrp</th>\n",
" <th>LobbyingActivities</th>\n",
" <th>LobbyingActivitiesInd</th>\n",
" <th>LocalChapters</th>\n",
" <th>LocalChaptersInd</th>\n",
" <th>MaterialDiversionOrMisuse</th>\n",
" <th>MaterialDiversionOrMisuseInd</th>\n",
" <th>MembersOrStockholders</th>\n",
" <th>MembersOrStockholdersInd</th>\n",
" <th>MembershipDues</th>\n",
" <th>MembershipDuesAmt</th>\n",
" <th>MethodOfAccountingAccrual</th>\n",
" <th>MethodOfAccountingAccrualInd</th>\n",
" <th>MinutesOfCommittees</th>\n",
" <th>MinutesOfCommitteesInd</th>\n",
" <th>MinutesOfGoverningBody</th>\n",
" <th>MinutesOfGoverningBodyInd</th>\n",
" <th>MissionDesc</th>\n",
" <th>MissionDescription</th>\n",
" <th>MoreThan5000KToIndividuals</th>\n",
" <th>MoreThan5000KToIndividualsInd</th>\n",
" <th>MoreThan5000KToOrgInd</th>\n",
" <th>MoreThan5000KToOrganizations</th>\n",
" <th>MortNotesPyblSecuredInvestProp</th>\n",
" <th>MortgNotesPyblScrdInvstPropGrp</th>\n",
" <th>NameOfForeignCountry</th>\n",
" <th>NameOfPrincipalOfficerPerson</th>\n",
" <th>NbrIndependentVotingMembers</th>\n",
" <th>NbrVotingGoverningBodyMembers</th>\n",
" <th>NbrVotingMembersGoverningBody</th>\n",
" <th>NetAssetsOrFundBalancesBOY</th>\n",
" <th>NetAssetsOrFundBalancesBOYAmt</th>\n",
" <th>NetAssetsOrFundBalancesEOY</th>\n",
" <th>NetAssetsOrFundBalancesEOYAmt</th>\n",
" <th>NetGainOrLossInvestments</th>\n",
" <th>NetGainOrLossInvestmentsGrp</th>\n",
" <th>NetIncmFromFundraisingEvtGrp</th>\n",
" <th>NetIncomeFromFundraisingEvents</th>\n",
" <th>NetIncomeFromGaming</th>\n",
" <th>NetIncomeFromGamingGrp</th>\n",
" <th>NetIncomeOrLoss</th>\n",
" <th>NetIncomeOrLossGrp</th>\n",
" <th>NetRentalIncomeOrLoss</th>\n",
" <th>NetRentalIncomeOrLossGrp</th>\n",
" <th>NetUnrelatedBusTxblIncmAmt</th>\n",
" <th>NetUnrelatedBusinessTxblIncome</th>\n",
" <th>NetUnrlzdGainsLossesInvstAmt</th>\n",
" <th>NoListedPersonsCompensatedInd</th>\n",
" <th>NonDeductibleContributions</th>\n",
" <th>NoncashContributions</th>\n",
" <th>NoncashContributionsAmt</th>\n",
" <th>NondeductibleContributionsInd</th>\n",
" <th>NumberFormsTransmittedWith1096</th>\n",
" <th>NumberIndependentVotingMembers</th>\n",
" <th>NumberIndividualsGT100K</th>\n",
" <th>NumberOfContractorsGT100K</th>\n",
" <th>NumberOfEmployees</th>\n",
" <th>NumberW2GIncluded</th>\n",
" <th>ObjectId</th>\n",
" <th>Occupancy</th>\n",
" <th>OccupancyGrp</th>\n",
" <th>OfficeExpenses</th>\n",
" <th>OfficeExpensesGrp</th>\n",
" <th>OfficerEntityWithBsnssRltnshp</th>\n",
" <th>OfficerMailingAddress</th>\n",
" <th>OfficerMailingAddressInd</th>\n",
" <th>OnBehalfOfIssuer</th>\n",
" <th>OnBehalfOfIssuerInd</th>\n",
" <th>OperateHospitalInd</th>\n",
" <th>Organization501c</th>\n",
" <th>Organization501c3</th>\n",
" <th>Organization501c3Ind</th>\n",
" <th>OrganizationFollowsSFAS117Ind</th>\n",
" <th>OrganizationName</th>\n",
" <th>OthNotesLoansReceivableNetGrp</th>\n",
" <th>OtherAssetsTotal</th>\n",
" <th>OtherAssetsTotalGrp</th>\n",
" <th>OtherChangesInNetAssetsAmt</th>\n",
" <th>OtherEmployeeBenefits</th>\n",
" <th>OtherEmployeeBenefitsGrp</th>\n",
" <th>OtherExpensePriorYear</th>\n",
" <th>OtherExpenses</th>\n",
" <th>OtherExpensesCurrentYear</th>\n",
" <th>OtherExpensesGrp</th>\n",
" <th>OtherLiabilities</th>\n",
" <th>OtherLiabilitiesGrp</th>\n",
" <th>OtherNotesLoansReceivableNet</th>\n",
" <th>OtherRevenueCurrentYear</th>\n",
" <th>OtherRevenueMisc</th>\n",
" <th>OtherRevenueMiscGrp</th>\n",
" <th>OtherRevenuePriorYear</th>\n",
" <th>OtherRevenueTotalAmt</th>\n",
" <th>OtherSalariesAndWages</th>\n",
" <th>OtherSalariesAndWagesGrp</th>\n",
" <th>OtherWebsite</th>\n",
" <th>OtherWebsiteInd</th>\n",
" <th>OwnWebsite</th>\n",
" <th>OwnWebsiteInd</th>\n",
" <th>PYBenefitsPaidToMembersAmt</th>\n",
" <th>PYContributionsGrantsAmt</th>\n",
" <th>PYExcessBenefitTransInd</th>\n",
" <th>PYGrantsAndSimilarPaidAmt</th>\n",
" <th>PYInvestmentIncomeAmt</th>\n",
" <th>PYOtherExpensesAmt</th>\n",
" <th>PYOtherRevenueAmt</th>\n",
" <th>PYProgramServiceRevenueAmt</th>\n",
" <th>PYRevenuesLessExpensesAmt</th>\n",
" <th>PYSalariesCompEmpBnftPaidAmt</th>\n",
" <th>PYTotalExpensesAmt</th>\n",
" <th>PYTotalProfFndrsngExpnsAmt</th>\n",
" <th>PYTotalRevenueAmt</th>\n",
" <th>PartialLiquidation</th>\n",
" <th>PartialLiquidationInd</th>\n",
" <th>PayPremiumsPrsnlBnftCntrctInd</th>\n",
" <th>PaymentsToAffiliates</th>\n",
" <th>PaymentsToAffiliatesGrp</th>\n",
" <th>PayrollTaxes</th>\n",
" <th>PayrollTaxesGrp</th>\n",
" <th>PensionPlanContributions</th>\n",
" <th>PensionPlanContributionsGrp</th>\n",
" <th>PermanentlyRestrictedNetAssets</th>\n",
" <th>PermanentlyRstrNetAssetsGrp</th>\n",
" <th>PledgesAndGrantsReceivable</th>\n",
" <th>PledgesAndGrantsReceivableGrp</th>\n",
" <th>PoliticalActivities</th>\n",
" <th>PoliticalCampaignActyInd</th>\n",
" <th>PremiumsPaid</th>\n",
" <th>PrepaidExpensesDeferredCharges</th>\n",
" <th>PrepaidExpensesDefrdChargesGrp</th>\n",
" <th>PrincipalOfficerNm</th>\n",
" <th>PriorExcessBenefitTransaction</th>\n",
" <th>PriorPeriodAdjustmentsAmt</th>\n",
" <th>ProfessionalFundraising</th>\n",
" <th>ProfessionalFundraisingInd</th>\n",
" <th>ProgSrvcAccomActy2Grp</th>\n",
" <th>ProgSrvcAccomActy3Grp</th>\n",
" <th>ProgSrvcAccomActyOtherGrp</th>\n",
" <th>ProgramServiceRevenue</th>\n",
" <th>ProgramServiceRevenueCY</th>\n",
" <th>ProgramServiceRevenueGrp</th>\n",
" <th>ProgramServiceRevenuePriorYear</th>\n",
" <th>ProhibitedTaxShelterTrans</th>\n",
" <th>ProhibitedTaxShelterTransInd</th>\n",
" <th>PymtTravelEntrtnmntPubOfclGrp</th>\n",
" <th>QuidProQuoContriDisclInd</th>\n",
" <th>QuidProQuoContributions</th>\n",
" <th>QuidProQuoContributionsInd</th>\n",
" <th>QuidProQuoDisclosure</th>\n",
" <th>RcvFndsToPayPrsnlBnftCntrctInd</th>\n",
" <th>RcvblFromDisqualifiedPrsnGrp</th>\n",
" <th>ReceivablesFromDisqualPersons</th>\n",
" <th>ReceivablesFromOfficersEtc</th>\n",
" <th>ReceivablesFromOfficersEtcGrp</th>\n",
" <th>ReconcilationOtherChanges</th>\n",
" <th>ReconcilationRevenueExpenses</th>\n",
" <th>ReconcilationRevenueExpnssAmt</th>\n",
" <th>ReconciliationUnrealizedInvest</th>\n",
" <th>RegularMonitoringEnforcement</th>\n",
" <th>RegularMonitoringEnfrcInd</th>\n",
" <th>RelatedEntity</th>\n",
" <th>RelatedEntityInd</th>\n",
" <th>RelatedOrgControlledEntity</th>\n",
" <th>RelatedOrganizationCtrlEntInd</th>\n",
" <th>RentalIncomeOrLoss</th>\n",
" <th>RentalIncomeOrLossGrp</th>\n",
" <th>ReportFin48Footnote</th>\n",
" <th>ReportInvestOthSecurities</th>\n",
" <th>ReportInvestmentsOtherSecInd</th>\n",
" <th>ReportLandBldgEquip</th>\n",
" <th>ReportLandBuildingEquipmentInd</th>\n",
" <th>ReportOtherAssets</th>\n",
" <th>ReportOtherAssetsInd</th>\n",
" <th>ReportOtherLiabilities</th>\n",
" <th>ReportOtherLiabilitiesInd</th>\n",
" <th>ReportProgRelInvest</th>\n",
" <th>ReportProgramRelatedInvstInd</th>\n",
" <th>ReturnHeader</th>\n",
" <th>Revenue</th>\n",
" <th>RevenueAmt</th>\n",
" <th>RevenuesLessExpensesCY</th>\n",
" <th>RevenuesLessExpensesPriorYear</th>\n",
" <th>Royalties</th>\n",
" <th>RoyaltiesGrp</th>\n",
" <th>RoyaltiesRevenue</th>\n",
" <th>RoyaltiesRevenueGrp</th>\n",
" <th>SalariesEtcCurrentYear</th>\n",
" <th>SalariesEtcPriorYear</th>\n",
" <th>SavingsAndTempCashInvestments</th>\n",
" <th>SavingsAndTempCashInvstGrp</th>\n",
" <th>ScheduleBRequired</th>\n",
" <th>ScheduleBRequiredInd</th>\n",
" <th>ScheduleJRequired</th>\n",
" <th>ScheduleJRequiredInd</th>\n",
" <th>ScheduleORequired</th>\n",
" <th>ScheduleORequiredInd</th>\n",
" <th>School</th>\n",
" <th>SchoolOperatingInd</th>\n",
" <th>SignificantChange</th>\n",
" <th>SignificantChangeInd</th>\n",
" <th>SignificantNewProgramServices</th>\n",
" <th>SignificantNewProgramSrvcInd</th>\n",
" <th>StateLegalDomicile</th>\n",
" <th>StatesWhereCopyOfReturnIsFiled</th>\n",
" <th>StatesWhereCopyOfReturnIsFldCd</th>\n",
" <th>SubjectToProxyTax</th>\n",
" <th>SubjectToProxyTaxInd</th>\n",
" <th>SubmittedOn</th>\n",
" <th>TaxExemptBondLiabilities</th>\n",
" <th>TaxExemptBondLiabilitiesGrp</th>\n",
" <th>TaxExemptBonds</th>\n",
" <th>TaxExemptBondsInd</th>\n",
" <th>TaxPeriod</th>\n",
" <th>TaxableDistributions</th>\n",
" <th>TaxablePartyNotification</th>\n",
" <th>TaxablePartyNotificationInd</th>\n",
" <th>TempOrPermanentEndowmentsInd</th>\n",
" <th>TemporarilyRestrictedNetAssets</th>\n",
" <th>TemporarilyRstrNetAssetsGrp</th>\n",
" <th>TermOrPermanentEndowments</th>\n",
" <th>TerminateOperationsInd</th>\n",
" <th>Terminated</th>\n",
" <th>TheBooksAreInCareOf</th>\n",
" <th>TotLiabNetAssetsFundBalanceGrp</th>\n",
" <th>TotReportableCompRltdOrgAmt</th>\n",
" <th>TotalAssets</th>\n",
" <th>TotalAssetsBOY</th>\n",
" <th>TotalAssetsBOYAmt</th>\n",
" <th>TotalAssetsEOY</th>\n",
" <th>TotalAssetsEOYAmt</th>\n",
" <th>TotalAssetsGrp</th>\n",
" <th>TotalCompGT150K</th>\n",
" <th>TotalCompGreaterThan150KInd</th>\n",
" <th>TotalContributions</th>\n",
" <th>TotalContributionsAmt</th>\n",
" <th>TotalEmployeeCnt</th>\n",
" <th>TotalExpensesCurrentYear</th>\n",
" <th>TotalExpensesPriorYear</th>\n",
" <th>TotalFunctionalExpenses</th>\n",
" <th>TotalFunctionalExpensesGrp</th>\n",
" <th>TotalFundrsngExpCurrentYear</th>\n",
" <th>TotalGrossUBI</th>\n",
" <th>TotalGrossUBIAmt</th>\n",
" <th>TotalLiabNetAssetsFundBalances</th>\n",
" <th>TotalLiabilities</th>\n",
" <th>TotalLiabilitiesBOY</th>\n",
" <th>TotalLiabilitiesBOYAmt</th>\n",
" <th>TotalLiabilitiesEOY</th>\n",
" <th>TotalLiabilitiesEOYAmt</th>\n",
" <th>TotalLiabilitiesGrp</th>\n",
" <th>TotalNbrEmployees</th>\n",
" <th>TotalNbrVolunteers</th>\n",
" <th>TotalNetAssetsFundBalanceGrp</th>\n",
" <th>TotalNetAssetsFundBalances</th>\n",
" <th>TotalOfOtherProgramServiceExp</th>\n",
" <th>TotalOfOtherProgramServiceGrnt</th>\n",
" <th>TotalOfOtherProgramServiceRev</th>\n",
" <th>TotalOthProgramServiceRevGrp</th>\n",
" <th>TotalOthProgramServiceRevenue</th>\n",
" <th>TotalOtherCompensation</th>\n",
" <th>TotalOtherCompensationAmt</th>\n",
" <th>TotalOtherProgSrvcExpenseAmt</th>\n",
" <th>TotalOtherProgSrvcGrantAmt</th>\n",
" <th>TotalOtherProgSrvcRevenueAmt</th>\n",
" <th>TotalOtherRevenue</th>\n",
" <th>TotalProfFundrsngExpCY</th>\n",
" <th>TotalProfFundrsngExpPriorYear</th>\n",
" <th>TotalProgramServiceExpense</th>\n",
" <th>TotalProgramServiceExpensesAmt</th>\n",
" <th>TotalProgramServiceRevenue</th>\n",
" <th>TotalProgramServiceRevenueAmt</th>\n",
" <th>TotalReportableCompFrmRltdOrgs</th>\n",
" <th>TotalReportableCompFromOrg</th>\n",
" <th>TotalReportableCompFromOrgAmt</th>\n",
" <th>TotalRevenue</th>\n",
" <th>TotalRevenueCurrentYear</th>\n",
" <th>TotalRevenueGrp</th>\n",
" <th>TotalRevenuePriorYear</th>\n",
" <th>TotalVolunteersCnt</th>\n",
" <th>TransactionRelatedEntity</th>\n",
" <th>TransfersToExemptNonChrtblOrg</th>\n",
" <th>Travel</th>\n",
" <th>TravelEntrtnmntPublicOfficials</th>\n",
" <th>TravelGrp</th>\n",
" <th>TrnsfrExmptNonChrtblRltdOrgInd</th>\n",
" <th>TypeOfOrganizationCorpInd</th>\n",
" <th>TypeOfOrganizationCorporation</th>\n",
" <th>URL</th>\n",
" <th>USAddress</th>\n",
" <th>UnrelatedBusIncmOverLimitInd</th>\n",
" <th>UnrelatedBusinessIncome</th>\n",
" <th>UnrestrictedNetAssets</th>\n",
" <th>UnrestrictedNetAssetsGrp</th>\n",
" <th>UnsecuredNotesLoansPayable</th>\n",
" <th>UnsecuredNotesLoansPayableGrp</th>\n",
" <th>UponRequest</th>\n",
" <th>UponRequestInd</th>\n",
" <th>VotingMembersGoverningBodyCnt</th>\n",
" <th>VotingMembersIndependentCnt</th>\n",
" <th>WebSite</th>\n",
" <th>WebsiteAddressTxt</th>\n",
" <th>WhistleblowerPolicy</th>\n",
" <th>WhistleblowerPolicyInd</th>\n",
" <th>YearFormation</th>\n",
" <th>_id</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>NaN</td>\n",
" <td>7</td>\n",
" <td>RetDoc1038000001</td>\n",
" <td>RetDoc1044400001</td>\n",
" <td>2009v1.7</td>\n",
" <td>http://www.irs.gov/efile</td>\n",
" <td>http://www.w3.org/2001/XMLSchema-instance</td>\n",
" <td>http://www.irs.gov/efile</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'80072', u'EOY': u'62455'}</td>\n",
" <td>{u'BOY': u'19547', u'EOY': u'72662'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'Expense': u'296564', u'Description': u'OPERATION AND MAINTENANCE OF THE WADSWORTH LONGFELLOW HOUSE (A HISTORIC STRUCTURE) AND MUSEUM11,487 ON-SITE VISITORS', u'Revenue': u'58195'}</td>\n",
" <td>{u'Expense': u'380450', u'Description': u'PROGRAMS AND EDUCATION3,680 ON-SITE VISITORS', u'Revenue': u'4035'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>PRESERVE AND TEACH THE HISTORY OF MAINE</td>\n",
" <td>{u'Expense': u'274266', u'Description': u'REMAINING PROGRAM SERVICES INCLUDE MUSEUM GIFT SHOP, MEMBERSHIP DEVELOPMENT, AND DIGITAL SERVICES.', u'Revenue': u'195220'}</td>\n",
" <td>{u'City': u'PORTLAND', u'State': u'ME', u'ZIPCode': u'04101', u'AddressLine1': u'485 CONGRESS STREET'}</td>\n",
" <td>{u'ProgramServices': u'18533', u'Fundraising': u'631', u'Total': u'19164'}</td>\n",
" <td>NaN</td>\n",
" <td>1496363</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'50514', u'ProgramServices': u'145152', u'Fundraising': u'45042', u'Total': u'240708'}</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'821778', u'EOY': u'736801'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2069610</td>\n",
" <td>2678197</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>93493070005101</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'381985', u'Total': u'381985'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1039100001'}</td>\n",
" <td>NaN</td>\n",
" <td>OPERATION AND MAINTENANCE OF HISTORICAL RESEARCH LIBRARY5,178 ON-SITE VISITORSTHE MAINE HISTORICAL SOCIETY WEB SITE IS PART OF ALL MAJOR PROGRAMS.547,782 UNIQUE WEB SITE VISITORS FOR YEAR OVER ALL PROGRAMS.</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>010211530</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>241105</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'24250', u'Total': u'24250'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'22257', u'Total': u'22257'}</td>\n",
" <td>{u'ManagementAndGeneral': u'291', u'Total': u'291'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'3280', u'ProgramServices': u'144389', u'Fundraising': u'748', u'Total': u'148417'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>X</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[{u'NamePerson': u'E CHRISTOPHER LIVESAY', u'Title': u'PRESIDENT', u'ReportableCompFromOrganization': u'0', u'AverageHoursPerWeek': u'6.00', u'OtherCompensation': u'0', u'ReportableCompFromRelatedOrgs': u'0', u'Officer': u'X', u'IndividualTrustee...</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>990</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'Securities': u'213312', u'Other': u'-14681'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>460054</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>{u'Securities': u'1839586'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>4400965</td>\n",
" <td>NaN</td>\n",
" <td>{u'Real': u'106357'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'GiftsGrantsContribReceived170': {u'CurrentTaxYear': u'2069610', u'CurrentTaxYearMinus4Years': u'1640889', u'Total': u'12679080', u'CurrentTaxYearMinus3Years': u'4699796', u'CurrentTaxYearMinus2Years': u'1590588', u'CurrentTaxYearMinus1Year': u...</td>\n",
" <td>{u'ContributorInfo': {u'ContributorAddressUS': {u'City': u'RESTRICTED', u'State': u'RESTRICTED', u'AddressLine2': u'RESTRICTED', u'ZIPCode': u'RESTRICTED', u'AddressLine1': u'RESTRICTED'}, u'AggregateContributions': u'RESTRICTED', u'ContributorNu...</td>\n",
" <td>NaN</td>\n",
" <td>{u'CollectUsedForLoanOrExchPrgrms': u'X', u'OtherRevenues': u'47320', u'PermanentEndowmentEOYBalance': u'0.11000', u'OtherLiabilities': [{u'Amount': u'4083', u'Description': u'LEASE DEPOSITS'}, {u'Amount': u'20346', u'Description': u'ANNUITY PAYM...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'ReviewProcessUnusualNCGifts': u'0', u'ThirdPartiesUsed': u'1', u'OtherNonCashContributionsTable': {u'NumberOfContributions': u'234', u'MethodOfDeterminingRevenues': u'COLLECTIONS NOT RECOGNIZED', u'NonCashCheckbox': u'X', u'Description': u'HIS...</td>\n",
" <td>{u'GeneralExplanation': [{u'Explanation': u'THE SOCIETY OFFERS MEMBERSHIPS, FOR AN ANNUAL MEMBERSHIP FEE, TO THE GENERAL PUBLIC WHICH COME WITH CERTAIN BENEFITS AND RIGHTS.', u'Identifier': u'Form 990, Part VI, Section A, line 6'}, {u'Explanation...</td>\n",
" <td>{u'ReceiptOfIntAnnRentsRoyalties': u'0', u'Form990ScheduleRPartIV': {u'NameOfRelatedOrg': {u'BusinessNameLine1': u'SEA LIGHT CORPORATION'}, u'PrimaryActivity': u'REAL ESTATE', u'LegalDomicileState': u'ME', u'AddressUS': {u'City': u'PORTLAND', u'S...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'832', u'ProgramServices': u'34439', u'Fundraising': u'5009', u'Total': u'40280'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'11758', u'ProgramServices': u'3260', u'Total': u'15018'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'36939', u'Total': u'36939'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'118786', u'EOY': u'119768'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumn': u'118952', u'ExclusionAmount': u'118952'}</td>\n",
" <td>317583</td>\n",
" <td>NaN</td>\n",
" <td>-19309</td>\n",
" <td>{u'BOY': u'13125'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'5271636', u'EOY': u'5266062'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1319343</td>\n",
" <td>8776406</td>\n",
" <td>{u'BOY': u'7815938', u'EOY': u'7457063'}</td>\n",
" <td>2016-03-21T17:23:53</td>\n",
" <td>NaN</td>\n",
" <td>{u'Securities': u'1626274', u'Other': u'14681'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'Real': u'47320'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>113193</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>THE MAINE HISTORICAL SOCIETY PRESERVES THE HERITAGE AND HISTORY OF MAINE: THE STORIES OF MAINE PEOPLE, THE TRADITIONS OF MAINE COMMUNITIES, AND THE RECORD OF MAINE'S PLACE IN A CHANGING WORLD. BECAUSE AN UNDERSTANDING OF THE PAST IS VITAL TO A HE...</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>{u'BOY': u'2666552', u'EOY': u'1466552'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>RICHARD D'ABATE</td>\n",
" <td>29</td>\n",
" <td>29</td>\n",
" <td>29</td>\n",
" <td>12488334</td>\n",
" <td>NaN</td>\n",
" <td>13095117</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumn': u'198631', u'ExclusionAmount': u'198631'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumn': u'59037', u'ExclusionAmount': u'59037'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>196702</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>21</td>\n",
" <td>29</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>48</td>\n",
" <td>0</td>\n",
" <td>201100709349300510</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'X', u'@typeOf501cOrganization': u'3'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>MAINE HISTORICAL SOCIETY</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'21478', u'ProgramServices': u'44089', u'Fundraising': u'13300', u'Total': u'78867'}</td>\n",
" <td>NaN</td>\n",
" <td>1346596</td>\n",
" <td>[{u'ProgramServices': u'141272', u'Description': u'ACQUISITIONS', u'Total': u'141272'}, {u'ProgramServices': u'53270', u'Description': u'GIFT SHOP/RESALE', u'Total': u'53270'}, {u'ManagementAndGeneral': u'13475', u'ProgramServices': u'26686', u'D...</td>\n",
" <td>1213005</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'40857', u'EOY': u'24429'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>59037</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>65827</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'206383', u'ProgramServices': u'522899', u'Fundraising': u'124372', u'Total': u'853654'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'14331', u'ProgramServices': u'41355', u'Fundraising': u'8966', u'Total': u'64652'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'ManagementAndGeneral': u'5078', u'ProgramServices': u'6668', u'Fundraising': u'2803', u'Total': u'14549'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'702762', u'EOY': u'185732'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'1186752', u'EOY': u'962844'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'28253', u'EOY': u'33353'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[{u'BusinessCode': u'453220', u'TotalRevenueColumn': u'98964', u'RelatedOrExemptFunctionIncome': u'98964', u'Description': u'GIFT SHOP SALES'}, {u'BusinessCode': u'900099', u'TotalRevenueColumn': u'58195', u'RelatedOrExemptFunctionIncome': u'5819...</td>\n",
" <td>266460</td>\n",
" <td>NaN</td>\n",
" <td>267171</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1043400001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'Real': u'59037'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'Filer': {u'Phone': u'2077741822', u'Name': {u'BusinessNameLine1': u'MAINE HISTORICAL SOCIETY'}, u'EIN': u'010211530', u'USAddress': {u'City': u'PORTLAND', u'State': u'ME', u'ZIPCode': u'04101', u'AddressLine1': u'485 CONGRESS STREET'}, u'NameC...</td>\n",
" <td>9010</td>\n",
" <td>NaN</td>\n",
" <td>487963</td>\n",
" <td>483881</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1011722</td>\n",
" <td>1161409</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1234500001'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>ME</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2011-03-22</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>201009</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'1618354', u'EOY': u'1959137'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>{u'TelephoneNumber': u'2077741822', u'NameBusiness': {u'BusinessNameLine1': u'JACKIE FENLASON DIR OF FINANCE'}, u'AddressUS': {u'City': u'PORTLAND', u'State': u'ME', u'ZIPCode': u'04101', u'AddressLine1': u'485 CONGRESS STREEt'}}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'15275815', u'EOY': u'14648553'}</td>\n",
" <td>15275815</td>\n",
" <td>NaN</td>\n",
" <td>14648553</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>2069610</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2224727</td>\n",
" <td>2508005</td>\n",
" <td>{u'ManagementAndGeneral': u'810283', u'ProgramServices': u'1192385', u'Fundraising': u'222059', u'Total': u'2224727'}</td>\n",
" <td>NaN</td>\n",
" <td>222059</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'15275815', u'EOY': u'14648553'}</td>\n",
" <td>{u'BOY': u'2787481', u'EOY': u'1553436'}</td>\n",
" <td>2787481</td>\n",
" <td>NaN</td>\n",
" <td>1553436</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>48</td>\n",
" <td>165</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOY': u'12488334', u'EOY': u'13095117'}</td>\n",
" <td>274266</td>\n",
" <td>NaN</td>\n",
" <td>195220</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumn': u'79476', u'RelatedOrExemptFunctionIncome': u'79476'}</td>\n",
" <td>3203</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>1192385</td>\n",
" <td>NaN</td>\n",
" <td>266460</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>91402</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumn': u'2712690', u'RelatedOrExemptFunctionIncome': u'266460', u'UnrelatedBusinessRevenue': u'0', u'ExclusionAmount': u'376620'}</td>\n",
" <td>2712690</td>\n",
" <td>NaN</td>\n",
" <td>2991886</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201100709349300510_public.xml</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>{u'BOY': u'10167218', u'EOY': u'10950248'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>WWW.MAINEHISTORY.ORG</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1822</td>\n",
" <td>5adf754335fd3fd83d06d437</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" @documentCnt @documentCount @documentId @referenceDocumentId \\\n",
"0 NaN 7 RetDoc1038000001 RetDoc1044400001 \n",
"\n",
" @returnVersion @xmlns \\\n",
"0 2009v1.7 http://www.irs.gov/efile \n",
"\n",
" @xmlns:xsi @xsi:schemaLocation \\\n",
"0 http://www.w3.org/2001/XMLSchema-instance http://www.irs.gov/efile \n",
"\n",
" AccountantCompileOrReview AccountantCompileOrReviewBasis \\\n",
"0 0 NaN \n",
"\n",
" AccountantCompileOrReviewInd AccountsPayableAccrExpnssGrp \\\n",
"0 NaN NaN \n",
"\n",
" AccountsPayableAccruedExpenses AccountsReceivable \\\n",
"0 {u'BOY': u'80072', u'EOY': u'62455'} {u'BOY': u'19547', u'EOY': u'72662'} \n",
"\n",
" AccountsReceivableGrp \\\n",
"0 NaN \n",
"\n",
" ActivitiesConductedPartnership \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'} \n",
"\n",
" ActivitiesConductedPrtshpInd \\\n",
"0 NaN \n",
"\n",
" Activity2 \\\n",
"0 {u'Expense': u'296564', u'Description': u'OPERATION AND MAINTENANCE OF THE WADSWORTH LONGFELLOW HOUSE (A HISTORIC STRUCTURE) AND MUSEUM11,487 ON-SITE VISITORS', u'Revenue': u'58195'} \n",
"\n",
" Activity3 \\\n",
"0 {u'Expense': u'380450', u'Description': u'PROGRAMS AND EDUCATION3,680 ON-SITE VISITORS', u'Revenue': u'4035'} \n",
"\n",
" ActivityCode ActivityOrMissionDesc ActivityOrMissionDescription \\\n",
"0 NaN NaN PRESERVE AND TEACH THE HISTORY OF MAINE \n",
"\n",
" ActivityOther \\\n",
"0 {u'Expense': u'274266', u'Description': u'REMAINING PROGRAM SERVICES INCLUDE MUSEUM GIFT SHOP, MEMBERSHIP DEVELOPMENT, AND DIGITAL SERVICES.', u'Revenue': u'195220'} \n",
"\n",
" AddressPrincipalOfficerUS \\\n",
"0 {u'City': u'PORTLAND', u'State': u'ME', u'ZIPCode': u'04101', u'AddressLine1': u'485 CONGRESS STREET'} \n",
"\n",
" Advertising \\\n",
"0 {u'ProgramServices': u'18533', u'Fundraising': u'631', u'Total': u'19164'} \n",
"\n",
" AdvertisingGrp AllOtherContributions AllOtherContributionsAmt \\\n",
"0 NaN 1496363 NaN \n",
"\n",
" AllOtherExpenses \\\n",
"0 {u'ManagementAndGeneral': u'50514', u'ProgramServices': u'145152', u'Fundraising': u'45042', u'Total': u'240708'} \n",
"\n",
" AllOtherExpensesGrp AnnualDisclosureCoveredPersons \\\n",
"0 NaN 1 \n",
"\n",
" AnnualDisclosureCoveredPrsnInd AuditCommittee AuditCommitteeInd \\\n",
"0 NaN 1 NaN \n",
"\n",
" BackupWthldComplianceInd \\\n",
"0 NaN \n",
"\n",
" BalanceSheetAmountsReported \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" BenefitsPaidToMembersCY BenefitsPaidToMembersPriorYear BenefitsToMembers \\\n",
"0 0 NaN NaN \n",
"\n",
" BenefitsToMembersGrp BooksInCareOfDetail BsnssRltnshpThruFamilyMember \\\n",
"0 NaN NaN 0 \n",
"\n",
" BsnssRltnshpWithOrganization BusinessRlnWithFamMemInd \\\n",
"0 0 NaN \n",
"\n",
" BusinessRlnWithOfficerEntInd BusinessRlnWithOrgMemInd \\\n",
"0 NaN NaN \n",
"\n",
" CYBenefitsPaidToMembersAmt CYContributionsGrantsAmt \\\n",
"0 NaN NaN \n",
"\n",
" CYGrantsAndSimilarPaidAmt CYInvestmentIncomeAmt CYOtherExpensesAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" CYOtherRevenueAmt CYProgramServiceRevenueAmt CYRevenuesLessExpensesAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" CYSalariesCompEmpBnftPaidAmt CYTotalExpensesAmt \\\n",
"0 NaN NaN \n",
"\n",
" CYTotalFundraisingExpenseAmt CYTotalProfFndrsngExpnsAmt CYTotalRevenueAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" CashNonInterestBearing CashNonInterestBearingGrp \\\n",
"0 {u'BOY': u'821778', u'EOY': u'736801'} NaN \n",
"\n",
" ChangeToOrgDocumentsInd ChangesToOrganizingDocs \\\n",
"0 NaN 0 \n",
"\n",
" CntrbtnsRprtdFundraisingEvents CntrctRcvdGreaterThan100KCnt \\\n",
"0 NaN NaN \n",
"\n",
" CollectionsOfArt \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" CollectionsOfArtInd CompCurrentOfcrDirectorsGrp \\\n",
"0 NaN NaN \n",
"\n",
" CompCurrentOfficersDirectors CompDisqualPersons CompDisqualPersonsGrp \\\n",
"0 NaN NaN NaN \n",
"\n",
" CompensationFromOtherSources CompensationFromOtherSrcsInd \\\n",
"0 0 NaN \n",
"\n",
" CompensationProcessCEO CompensationProcessCEOInd CompensationProcessOther \\\n",
"0 0 NaN 0 \n",
"\n",
" CompensationProcessOtherInd ComplianceWithBackupWitholding \\\n",
"0 NaN 1 \n",
"\n",
" ConferencesMeetings ConferencesMeetingsGrp ConflictOfInterestPolicy \\\n",
"0 NaN NaN 1 \n",
"\n",
" ConflictOfInterestPolicyInd \\\n",
"0 NaN \n",
"\n",
" ConservationEasements \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ConservationEasementsInd \\\n",
"0 NaN \n",
"\n",
" ConsolidatedAuditFinancialStmt \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ConsolidatedAuditFinclStmtInd ContractorCompensation \\\n",
"0 NaN NaN \n",
"\n",
" ContractorCompensationGrp ContriRptFundraisingEventAmt \\\n",
"0 NaN NaN \n",
"\n",
" ContributionsGrantsCurrentYear ContributionsGrantsPriorYear CostOfGoodsSold \\\n",
"0 2069610 2678197 NaN \n",
"\n",
" CostOfGoodsSoldAmt \\\n",
"0 NaN \n",
"\n",
" CreditCounseling \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" CreditCounselingInd DLN DecisionsSubjectToApprovaInd \\\n",
"0 NaN 93493070005101 NaN \n",
"\n",
" DecisionsSubjectToApproval DeductibleArtContributionInd \\\n",
"0 0 NaN \n",
"\n",
" DeductibleContributionsOfArt \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'} \n",
"\n",
" DeductibleNonCashContriInd \\\n",
"0 NaN \n",
"\n",
" DeductibleNonCashContributions \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'} \n",
"\n",
" DeferredRevenue DeferredRevenueGrp DelegationOfManagementDuties \\\n",
"0 NaN NaN 0 \n",
"\n",
" DelegationOfMgmtDutiesInd \\\n",
"0 NaN \n",
"\n",
" DepreciationDepletion \\\n",
"0 {u'ManagementAndGeneral': u'381985', u'Total': u'381985'} \n",
"\n",
" DepreciationDepletionGrp Desc \\\n",
"0 NaN NaN \n",
"\n",
" DescribedIn501C3 \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1039100001'} \n",
"\n",
" DescribedInSection501c3Ind \\\n",
"0 NaN \n",
"\n",
" Description \\\n",
"0 OPERATION AND MAINTENANCE OF HISTORICAL RESEARCH LIBRARY5,178 ON-SITE VISITORSTHE MAINE HISTORICAL SOCIETY WEB SITE IS PART OF ALL MAJOR PROGRAMS.547,782 UNIQUE WEB SITE VISITORS FOR YEAR OVER ALL PROGRAMS. \n",
"\n",
" DisregardedEntity \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'} \n",
"\n",
" DisregardedEntityInd DistributionToDonor DocumentRetentionPolicy \\\n",
"0 NaN NaN 0 \n",
"\n",
" DocumentRetentionPolicyInd DonatedServicesAndUseFcltsAmt \\\n",
"0 NaN NaN \n",
"\n",
" DonorAdvisedFundInd \\\n",
"0 NaN \n",
"\n",
" DonorAdvisedFunds EIN \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} 010211530 \n",
"\n",
" ElectionOfBoardMembers ElectionOfBoardMembersInd EmployeeCnt \\\n",
"0 1 NaN NaN \n",
"\n",
" EmploymentTaxReturnsFiled EmploymentTaxReturnsFiledInd \\\n",
"0 1 NaN \n",
"\n",
" EngagedInExcessBenefitTransInd EscrowAccount EscrowAccountInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" EscrowAccountLiability EscrowAccountLiabilityGrp ExcessBenefitTransaction \\\n",
"0 NaN NaN 0 \n",
"\n",
" Expense ExpenseAmt FSAudited FSAuditedBasis FSAuditedBasisGrp FSAuditedInd \\\n",
"0 241105 NaN 1 NaN NaN NaN \n",
"\n",
" FamilyOrBusinessRelationship FamilyOrBusinessRlnInd \\\n",
"0 0 NaN \n",
"\n",
" FederalGrantAuditPerformed FederalGrantAuditPerformedInd \\\n",
"0 NaN NaN \n",
"\n",
" FederalGrantAuditRequired FederalGrantAuditRequiredInd FederatedCampaigns \\\n",
"0 0 NaN NaN \n",
"\n",
" FederatedCampaignsAmt \\\n",
"0 NaN \n",
"\n",
" FeesForServicesAccounting \\\n",
"0 {u'ManagementAndGeneral': u'24250', u'Total': u'24250'} \n",
"\n",
" FeesForServicesAccountingGrp \\\n",
"0 NaN \n",
"\n",
" FeesForServicesInvstMgmntFees \\\n",
"0 {u'ManagementAndGeneral': u'22257', u'Total': u'22257'} \n",
"\n",
" FeesForServicesLegal \\\n",
"0 {u'ManagementAndGeneral': u'291', u'Total': u'291'} \n",
"\n",
" FeesForServicesLegalGrp FeesForServicesLobbying FeesForServicesLobbyingGrp \\\n",
"0 NaN NaN NaN \n",
"\n",
" FeesForServicesManagement FeesForServicesManagementGrp \\\n",
"0 NaN NaN \n",
"\n",
" FeesForServicesOther \\\n",
"0 {u'ManagementAndGeneral': u'3280', u'ProgramServices': u'144389', u'Fundraising': u'748', u'Total': u'148417'} \n",
"\n",
" FeesForServicesOtherGrp FeesForServicesProfFundraising \\\n",
"0 NaN NaN \n",
"\n",
" FeesForSrvcInvstMgmntFeesGrp FinancialStatementConsolidated \\\n",
"0 NaN NaN \n",
"\n",
" FinancialStatementSeparate FollowSFAS117 ForeignActivities \\\n",
"0 X X 0 \n",
"\n",
" ForeignActivitiesInd ForeignFinancialAccount ForeignFinancialAccountInd \\\n",
"0 NaN 0 NaN \n",
"\n",
" ForeignGrants ForeignGrantsGrp ForeignOffice ForeignOfficeInd \\\n",
"0 NaN NaN 0 NaN \n",
"\n",
" Form8282PropertyDisposedOf Form8282PropertyDisposedOfInd Form990-TFiled \\\n",
"0 0 NaN NaN \n",
"\n",
" Form990PartVIISectionA \\\n",
"0 [{u'NamePerson': u'E CHRISTOPHER LIVESAY', u'Title': u'PRESIDENT', u'ReportableCompFromOrganization': u'0', u'AverageHoursPerWeek': u'6.00', u'OtherCompensation': u'0', u'ReportableCompFromRelatedOrgs': u'0', u'Officer': u'X', u'IndividualTrustee... \n",
"\n",
" Form990PartVIISectionAGrp Form990ProvidedToGoverningBody \\\n",
"0 NaN 0 \n",
"\n",
" Form990ProvidedToGvrnBodyInd Form990TFiledInd FormType FormationYr \\\n",
"0 NaN NaN 990 NaN \n",
"\n",
" FormerOfcrEmployeesListedInd FormersListed FundraisingActivities \\\n",
"0 NaN 0 0 \n",
"\n",
" FundraisingActivitiesInd FundraisingAmt FundraisingDirectExpenses \\\n",
"0 NaN NaN NaN \n",
"\n",
" FundraisingDirectExpensesAmt FundraisingEvents FundraisingGrossIncomeAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" FundsToPayPremiums GainOrLoss \\\n",
"0 NaN {u'Securities': u'213312', u'Other': u'-14681'} \n",
"\n",
" GainOrLossGrp Gaming GamingActivitiesInd GamingDirectExpensesAmt \\\n",
"0 NaN 0 NaN NaN \n",
"\n",
" GamingGrossIncomeAmt GoverningBodyVotingMembersCnt GovernmentGrants \\\n",
"0 NaN NaN 460054 \n",
"\n",
" GovernmentGrantsAmt GrantAmt GrantToRelatedPerson GrantToRelatedPersonInd \\\n",
"0 NaN NaN 0 NaN \n",
"\n",
" Grants GrantsAndSimilarAmntsCY GrantsAndSimilarAmntsPriorYear GrantsPayable \\\n",
"0 NaN 0 NaN NaN \n",
"\n",
" GrantsPayableGrp GrantsToDomesticIndividuals GrantsToDomesticIndividualsGrp \\\n",
"0 NaN NaN NaN \n",
"\n",
" GrantsToDomesticOrgs GrantsToDomesticOrgsGrp GrantsToIndividuals \\\n",
"0 NaN NaN 0 \n",
"\n",
" GrantsToIndividualsInd GrantsToOrganizations GrantsToOrganizationsInd \\\n",
"0 NaN 0 NaN \n",
"\n",
" GrossAmountSalesAssets GrossAmountSalesAssetsGrp \\\n",
"0 {u'Securities': u'1839586'} NaN \n",
"\n",
" GrossIncomeFundraisingEvents GrossIncomeGaming GrossReceipts \\\n",
"0 NaN NaN 4400965 \n",
"\n",
" GrossReceiptsAmt GrossRents GrossRentsGrp GrossSalesOfInventory \\\n",
"0 NaN {u'Real': u'106357'} NaN NaN \n",
"\n",
" GrossSalesOfInventoryAmt GroupReturnForAffiliates \\\n",
"0 NaN 0 \n",
"\n",
" GroupReturnForAffiliatesInd Hospital IRPDocumentCnt IRPDocumentW2GCnt \\\n",
"0 NaN 0 NaN NaN \n",
"\n",
" IRS990ScheduleA \\\n",
"0 {u'GiftsGrantsContribReceived170': {u'CurrentTaxYear': u'2069610', u'CurrentTaxYearMinus4Years': u'1640889', u'Total': u'12679080', u'CurrentTaxYearMinus3Years': u'4699796', u'CurrentTaxYearMinus2Years': u'1590588', u'CurrentTaxYearMinus1Year': u... \n",
"\n",
" IRS990ScheduleB \\\n",
"0 {u'ContributorInfo': {u'ContributorAddressUS': {u'City': u'RESTRICTED', u'State': u'RESTRICTED', u'AddressLine2': u'RESTRICTED', u'ZIPCode': u'RESTRICTED', u'AddressLine1': u'RESTRICTED'}, u'AggregateContributions': u'RESTRICTED', u'ContributorNu... \n",
"\n",
" IRS990ScheduleC \\\n",
"0 NaN \n",
"\n",
" IRS990ScheduleD \\\n",
"0 {u'CollectUsedForLoanOrExchPrgrms': u'X', u'OtherRevenues': u'47320', u'PermanentEndowmentEOYBalance': u'0.11000', u'OtherLiabilities': [{u'Amount': u'4083', u'Description': u'LEASE DEPOSITS'}, {u'Amount': u'20346', u'Description': u'ANNUITY PAYM... \n",
"\n",
" IRS990ScheduleF IRS990ScheduleG IRS990ScheduleI IRS990ScheduleJ \\\n",
"0 NaN NaN NaN NaN \n",
"\n",
" IRS990ScheduleK IRS990ScheduleL \\\n",
"0 NaN NaN \n",
"\n",
" IRS990ScheduleM \\\n",
"0 {u'ReviewProcessUnusualNCGifts': u'0', u'ThirdPartiesUsed': u'1', u'OtherNonCashContributionsTable': {u'NumberOfContributions': u'234', u'MethodOfDeterminingRevenues': u'COLLECTIONS NOT RECOGNIZED', u'NonCashCheckbox': u'X', u'Description': u'HIS... \n",
"\n",
" IRS990ScheduleO \\\n",
"0 {u'GeneralExplanation': [{u'Explanation': u'THE SOCIETY OFFERS MEMBERSHIPS, FOR AN ANNUAL MEMBERSHIP FEE, TO THE GENERAL PUBLIC WHICH COME WITH CERTAIN BENEFITS AND RIGHTS.', u'Identifier': u'Form 990, Part VI, Section A, line 6'}, {u'Explanation... \n",
"\n",
" IRS990ScheduleR \\\n",
"0 {u'ReceiptOfIntAnnRentsRoyalties': u'0', u'Form990ScheduleRPartIV': {u'NameOfRelatedOrg': {u'BusinessNameLine1': u'SEA LIGHT CORPORATION'}, u'PrimaryActivity': u'REAL ESTATE', u'LegalDomicileState': u'ME', u'AddressUS': {u'City': u'PORTLAND', u'S... \n",
"\n",
" IncludeFIN48FootnoteInd IncmFromInvestBondProceedsGrp \\\n",
"0 NaN NaN \n",
"\n",
" IncomeFromInvestBondProceeds \\\n",
"0 NaN \n",
"\n",
" IndependentAuditFinancialStmt \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" IndependentAuditFinclStmtInd IndependentVotingMemberCnt \\\n",
"0 NaN NaN \n",
"\n",
" IndivRcvdGreaterThan100KCnt IndoorTanningServices IndoorTanningServicesInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" InfoInScheduleOPartIII InfoInScheduleOPartIIIInd InfoInScheduleOPartIX \\\n",
"0 NaN NaN NaN \n",
"\n",
" InfoInScheduleOPartIXInd InfoInScheduleOPartVI InfoInScheduleOPartVIIInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" InfoInScheduleOPartVIInd InfoInScheduleOPartX InfoInScheduleOPartXI \\\n",
"0 NaN NaN NaN \n",
"\n",
" InfoInScheduleOPartXII InfoInScheduleOPartXIIInd InfoInScheduleOPartXIInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" InformationTechnology \\\n",
"0 {u'ManagementAndGeneral': u'832', u'ProgramServices': u'34439', u'Fundraising': u'5009', u'Total': u'40280'} \n",
"\n",
" InformationTechnologyGrp \\\n",
"0 NaN \n",
"\n",
" Insurance \\\n",
"0 {u'ManagementAndGeneral': u'11758', u'ProgramServices': u'3260', u'Total': u'15018'} \n",
"\n",
" InsuranceGrp IntangibleAssets IntangibleAssetsGrp \\\n",
"0 NaN NaN NaN \n",
"\n",
" Interest InterestGrp \\\n",
"0 {u'ManagementAndGeneral': u'36939', u'Total': u'36939'} NaN \n",
"\n",
" InventoriesForSaleOrUse InventoriesForSaleOrUseGrp \\\n",
"0 {u'BOY': u'118786', u'EOY': u'119768'} NaN \n",
"\n",
" InvestTaxExemptBonds InvestTaxExemptBondsInd InvestmentInJointVenture \\\n",
"0 NaN NaN 0 \n",
"\n",
" InvestmentInJointVentureInd \\\n",
"0 NaN \n",
"\n",
" InvestmentIncome \\\n",
"0 {u'TotalRevenueColumn': u'118952', u'ExclusionAmount': u'118952'} \n",
"\n",
" InvestmentIncomeCurrentYear InvestmentIncomeGrp InvestmentIncomePriorYear \\\n",
"0 317583 NaN -19309 \n",
"\n",
" InvestmentsOtherSecurities InvestmentsOtherSecuritiesGrp \\\n",
"0 {u'BOY': u'13125'} NaN \n",
"\n",
" InvestmentsProgramRelated InvestmentsProgramRelatedGrp \\\n",
"0 NaN NaN \n",
"\n",
" InvestmentsPubTradedSecGrp InvestmentsPubTradedSecurities \\\n",
"0 NaN {u'BOY': u'5271636', u'EOY': u'5266062'} \n",
"\n",
" LandBldgEquipAccumDeprecAmt LandBldgEquipBasisNetGrp \\\n",
"0 NaN NaN \n",
"\n",
" LandBldgEquipCostOrOtherBssAmt LandBldgEquipmentAccumDeprec \\\n",
"0 NaN 1319343 \n",
"\n",
" LandBuildingsEquipmentBasis LandBuildingsEquipmentBasisNet \\\n",
"0 8776406 {u'BOY': u'7815938', u'EOY': u'7457063'} \n",
"\n",
" LastUpdated LegalDomicileStateCd \\\n",
"0 2016-03-21T17:23:53 NaN \n",
"\n",
" LessCostOthBasisSalesExpenses \\\n",
"0 {u'Securities': u'1626274', u'Other': u'14681'} \n",
"\n",
" LessCostOthBasisSalesExpnssGrp LessRentalExpenses LessRentalExpensesGrp \\\n",
"0 NaN {u'Real': u'47320'} NaN \n",
"\n",
" LoanOutstandingInd LoanToOfficerOrDQP LoansFromOfficersDirectors \\\n",
"0 NaN 0 NaN \n",
"\n",
" LoansFromOfficersDirectorsGrp LobbyingActivities LobbyingActivitiesInd \\\n",
"0 NaN 0 NaN \n",
"\n",
" LocalChapters LocalChaptersInd MaterialDiversionOrMisuse \\\n",
"0 0 NaN 0 \n",
"\n",
" MaterialDiversionOrMisuseInd MembersOrStockholders MembersOrStockholdersInd \\\n",
"0 NaN 1 NaN \n",
"\n",
" MembershipDues MembershipDuesAmt MethodOfAccountingAccrual \\\n",
"0 113193 NaN X \n",
"\n",
" MethodOfAccountingAccrualInd MinutesOfCommittees MinutesOfCommitteesInd \\\n",
"0 NaN 1 NaN \n",
"\n",
" MinutesOfGoverningBody MinutesOfGoverningBodyInd MissionDesc \\\n",
"0 1 NaN NaN \n",
"\n",
" MissionDescription \\\n",
"0 THE MAINE HISTORICAL SOCIETY PRESERVES THE HERITAGE AND HISTORY OF MAINE: THE STORIES OF MAINE PEOPLE, THE TRADITIONS OF MAINE COMMUNITIES, AND THE RECORD OF MAINE'S PLACE IN A CHANGING WORLD. BECAUSE AN UNDERSTANDING OF THE PAST IS VITAL TO A HE... \n",
"\n",
" MoreThan5000KToIndividuals MoreThan5000KToIndividualsInd \\\n",
"0 0 NaN \n",
"\n",
" MoreThan5000KToOrgInd MoreThan5000KToOrganizations \\\n",
"0 NaN 0 \n",
"\n",
" MortNotesPyblSecuredInvestProp MortgNotesPyblScrdInvstPropGrp \\\n",
"0 {u'BOY': u'2666552', u'EOY': u'1466552'} NaN \n",
"\n",
" NameOfForeignCountry NameOfPrincipalOfficerPerson \\\n",
"0 NaN RICHARD D'ABATE \n",
"\n",
" NbrIndependentVotingMembers NbrVotingGoverningBodyMembers \\\n",
"0 29 29 \n",
"\n",
" NbrVotingMembersGoverningBody NetAssetsOrFundBalancesBOY \\\n",
"0 29 12488334 \n",
"\n",
" NetAssetsOrFundBalancesBOYAmt NetAssetsOrFundBalancesEOY \\\n",
"0 NaN 13095117 \n",
"\n",
" NetAssetsOrFundBalancesEOYAmt \\\n",
"0 NaN \n",
"\n",
" NetGainOrLossInvestments \\\n",
"0 {u'TotalRevenueColumn': u'198631', u'ExclusionAmount': u'198631'} \n",
"\n",
" NetGainOrLossInvestmentsGrp NetIncmFromFundraisingEvtGrp \\\n",
"0 NaN NaN \n",
"\n",
" NetIncomeFromFundraisingEvents NetIncomeFromGaming NetIncomeFromGamingGrp \\\n",
"0 NaN NaN NaN \n",
"\n",
" NetIncomeOrLoss NetIncomeOrLossGrp \\\n",
"0 NaN NaN \n",
"\n",
" NetRentalIncomeOrLoss \\\n",
"0 {u'TotalRevenueColumn': u'59037', u'ExclusionAmount': u'59037'} \n",
"\n",
" NetRentalIncomeOrLossGrp NetUnrelatedBusTxblIncmAmt \\\n",
"0 NaN NaN \n",
"\n",
" NetUnrelatedBusinessTxblIncome NetUnrlzdGainsLossesInvstAmt \\\n",
"0 0 NaN \n",
"\n",
" NoListedPersonsCompensatedInd NonDeductibleContributions \\\n",
"0 NaN 0 \n",
"\n",
" NoncashContributions NoncashContributionsAmt NondeductibleContributionsInd \\\n",
"0 196702 NaN NaN \n",
"\n",
" NumberFormsTransmittedWith1096 NumberIndependentVotingMembers \\\n",
"0 21 29 \n",
"\n",
" NumberIndividualsGT100K NumberOfContractorsGT100K NumberOfEmployees \\\n",
"0 0 0 48 \n",
"\n",
" NumberW2GIncluded ObjectId Occupancy OccupancyGrp OfficeExpenses \\\n",
"0 0 201100709349300510 NaN NaN NaN \n",
"\n",
" OfficeExpensesGrp OfficerEntityWithBsnssRltnshp OfficerMailingAddress \\\n",
"0 NaN 0 0 \n",
"\n",
" OfficerMailingAddressInd OnBehalfOfIssuer OnBehalfOfIssuerInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" OperateHospitalInd Organization501c \\\n",
"0 NaN {u'#text': u'X', u'@typeOf501cOrganization': u'3'} \n",
"\n",
" Organization501c3 Organization501c3Ind OrganizationFollowsSFAS117Ind \\\n",
"0 NaN NaN NaN \n",
"\n",
" OrganizationName OthNotesLoansReceivableNetGrp OtherAssetsTotal \\\n",
"0 MAINE HISTORICAL SOCIETY NaN NaN \n",
"\n",
" OtherAssetsTotalGrp OtherChangesInNetAssetsAmt \\\n",
"0 NaN NaN \n",
"\n",
" OtherEmployeeBenefits \\\n",
"0 {u'ManagementAndGeneral': u'21478', u'ProgramServices': u'44089', u'Fundraising': u'13300', u'Total': u'78867'} \n",
"\n",
" OtherEmployeeBenefitsGrp OtherExpensePriorYear \\\n",
"0 NaN 1346596 \n",
"\n",
" OtherExpenses \\\n",
"0 [{u'ProgramServices': u'141272', u'Description': u'ACQUISITIONS', u'Total': u'141272'}, {u'ProgramServices': u'53270', u'Description': u'GIFT SHOP/RESALE', u'Total': u'53270'}, {u'ManagementAndGeneral': u'13475', u'ProgramServices': u'26686', u'D... \n",
"\n",
" OtherExpensesCurrentYear OtherExpensesGrp \\\n",
"0 1213005 NaN \n",
"\n",
" OtherLiabilities OtherLiabilitiesGrp \\\n",
"0 {u'BOY': u'40857', u'EOY': u'24429'} NaN \n",
"\n",
" OtherNotesLoansReceivableNet OtherRevenueCurrentYear OtherRevenueMisc \\\n",
"0 NaN 59037 NaN \n",
"\n",
" OtherRevenueMiscGrp OtherRevenuePriorYear OtherRevenueTotalAmt \\\n",
"0 NaN 65827 NaN \n",
"\n",
" OtherSalariesAndWages \\\n",
"0 {u'ManagementAndGeneral': u'206383', u'ProgramServices': u'522899', u'Fundraising': u'124372', u'Total': u'853654'} \n",
"\n",
" OtherSalariesAndWagesGrp OtherWebsite OtherWebsiteInd OwnWebsite \\\n",
"0 NaN NaN NaN NaN \n",
"\n",
" OwnWebsiteInd PYBenefitsPaidToMembersAmt PYContributionsGrantsAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" PYExcessBenefitTransInd PYGrantsAndSimilarPaidAmt PYInvestmentIncomeAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" PYOtherExpensesAmt PYOtherRevenueAmt PYProgramServiceRevenueAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" PYRevenuesLessExpensesAmt PYSalariesCompEmpBnftPaidAmt PYTotalExpensesAmt \\\n",
"0 NaN NaN NaN \n",
"\n",
" PYTotalProfFndrsngExpnsAmt PYTotalRevenueAmt PartialLiquidation \\\n",
"0 NaN NaN 0 \n",
"\n",
" PartialLiquidationInd PayPremiumsPrsnlBnftCntrctInd PaymentsToAffiliates \\\n",
"0 NaN NaN NaN \n",
"\n",
" PaymentsToAffiliatesGrp \\\n",
"0 NaN \n",
"\n",
" PayrollTaxes \\\n",
"0 {u'ManagementAndGeneral': u'14331', u'ProgramServices': u'41355', u'Fundraising': u'8966', u'Total': u'64652'} \n",
"\n",
" PayrollTaxesGrp \\\n",
"0 NaN \n",
"\n",
" PensionPlanContributions \\\n",
"0 {u'ManagementAndGeneral': u'5078', u'ProgramServices': u'6668', u'Fundraising': u'2803', u'Total': u'14549'} \n",
"\n",
" PensionPlanContributionsGrp PermanentlyRestrictedNetAssets \\\n",
"0 NaN {u'BOY': u'702762', u'EOY': u'185732'} \n",
"\n",
" PermanentlyRstrNetAssetsGrp PledgesAndGrantsReceivable \\\n",
"0 NaN {u'BOY': u'1186752', u'EOY': u'962844'} \n",
"\n",
" PledgesAndGrantsReceivableGrp PoliticalActivities PoliticalCampaignActyInd \\\n",
"0 NaN 0 NaN \n",
"\n",
" PremiumsPaid PrepaidExpensesDeferredCharges \\\n",
"0 NaN {u'BOY': u'28253', u'EOY': u'33353'} \n",
"\n",
" PrepaidExpensesDefrdChargesGrp PrincipalOfficerNm \\\n",
"0 NaN NaN \n",
"\n",
" PriorExcessBenefitTransaction PriorPeriodAdjustmentsAmt \\\n",
"0 0 NaN \n",
"\n",
" ProfessionalFundraising ProfessionalFundraisingInd ProgSrvcAccomActy2Grp \\\n",
"0 0 NaN NaN \n",
"\n",
" ProgSrvcAccomActy3Grp ProgSrvcAccomActyOtherGrp \\\n",
"0 NaN NaN \n",
"\n",
" ProgramServiceRevenue \\\n",
"0 [{u'BusinessCode': u'453220', u'TotalRevenueColumn': u'98964', u'RelatedOrExemptFunctionIncome': u'98964', u'Description': u'GIFT SHOP SALES'}, {u'BusinessCode': u'900099', u'TotalRevenueColumn': u'58195', u'RelatedOrExemptFunctionIncome': u'5819... \n",
"\n",
" ProgramServiceRevenueCY ProgramServiceRevenueGrp \\\n",
"0 266460 NaN \n",
"\n",
" ProgramServiceRevenuePriorYear ProhibitedTaxShelterTrans \\\n",
"0 267171 0 \n",
"\n",
" ProhibitedTaxShelterTransInd PymtTravelEntrtnmntPubOfclGrp \\\n",
"0 NaN NaN \n",
"\n",
" QuidProQuoContriDisclInd QuidProQuoContributions QuidProQuoContributionsInd \\\n",
"0 NaN 1 NaN \n",
"\n",
" QuidProQuoDisclosure RcvFndsToPayPrsnlBnftCntrctInd \\\n",
"0 1 NaN \n",
"\n",
" RcvblFromDisqualifiedPrsnGrp ReceivablesFromDisqualPersons \\\n",
"0 NaN NaN \n",
"\n",
" ReceivablesFromOfficersEtc ReceivablesFromOfficersEtcGrp \\\n",
"0 NaN NaN \n",
"\n",
" ReconcilationOtherChanges ReconcilationRevenueExpenses \\\n",
"0 NaN NaN \n",
"\n",
" ReconcilationRevenueExpnssAmt ReconciliationUnrealizedInvest \\\n",
"0 NaN NaN \n",
"\n",
" RegularMonitoringEnforcement RegularMonitoringEnfrcInd \\\n",
"0 1 NaN \n",
"\n",
" RelatedEntity \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1043400001'} \n",
"\n",
" RelatedEntityInd \\\n",
"0 NaN \n",
"\n",
" RelatedOrgControlledEntity \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'} \n",
"\n",
" RelatedOrganizationCtrlEntInd RentalIncomeOrLoss RentalIncomeOrLossGrp \\\n",
"0 NaN {u'Real': u'59037'} NaN \n",
"\n",
" ReportFin48Footnote ReportInvestOthSecurities ReportInvestmentsOtherSecInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" ReportLandBldgEquip ReportLandBuildingEquipmentInd ReportOtherAssets \\\n",
"0 NaN NaN NaN \n",
"\n",
" ReportOtherAssetsInd ReportOtherLiabilities ReportOtherLiabilitiesInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" ReportProgRelInvest ReportProgramRelatedInvstInd \\\n",
"0 NaN NaN \n",
"\n",
" ReturnHeader \\\n",
"0 {u'Filer': {u'Phone': u'2077741822', u'Name': {u'BusinessNameLine1': u'MAINE HISTORICAL SOCIETY'}, u'EIN': u'010211530', u'USAddress': {u'City': u'PORTLAND', u'State': u'ME', u'ZIPCode': u'04101', u'AddressLine1': u'485 CONGRESS STREET'}, u'NameC... \n",
"\n",
" Revenue RevenueAmt RevenuesLessExpensesCY RevenuesLessExpensesPriorYear \\\n",
"0 9010 NaN 487963 483881 \n",
"\n",
" Royalties RoyaltiesGrp RoyaltiesRevenue RoyaltiesRevenueGrp \\\n",
"0 NaN NaN NaN NaN \n",
"\n",
" SalariesEtcCurrentYear SalariesEtcPriorYear SavingsAndTempCashInvestments \\\n",
"0 1011722 1161409 NaN \n",
"\n",
" SavingsAndTempCashInvstGrp \\\n",
"0 NaN \n",
"\n",
" ScheduleBRequired \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1234500001'} \n",
"\n",
" ScheduleBRequiredInd ScheduleJRequired ScheduleJRequiredInd \\\n",
"0 NaN 0 NaN \n",
"\n",
" ScheduleORequired ScheduleORequiredInd School SchoolOperatingInd \\\n",
"0 1 NaN 0 NaN \n",
"\n",
" SignificantChange SignificantChangeInd SignificantNewProgramServices \\\n",
"0 0 NaN 0 \n",
"\n",
" SignificantNewProgramSrvcInd StateLegalDomicile \\\n",
"0 NaN ME \n",
"\n",
" StatesWhereCopyOfReturnIsFiled StatesWhereCopyOfReturnIsFldCd \\\n",
"0 NaN NaN \n",
"\n",
" SubjectToProxyTax SubjectToProxyTaxInd SubmittedOn TaxExemptBondLiabilities \\\n",
"0 NaN NaN 2011-03-22 NaN \n",
"\n",
" TaxExemptBondLiabilitiesGrp TaxExemptBonds TaxExemptBondsInd TaxPeriod \\\n",
"0 NaN 0 NaN 201009 \n",
"\n",
" TaxableDistributions TaxablePartyNotification TaxablePartyNotificationInd \\\n",
"0 NaN 0 NaN \n",
"\n",
" TempOrPermanentEndowmentsInd TemporarilyRestrictedNetAssets \\\n",
"0 NaN {u'BOY': u'1618354', u'EOY': u'1959137'} \n",
"\n",
" TemporarilyRstrNetAssetsGrp \\\n",
"0 NaN \n",
"\n",
" TermOrPermanentEndowments \\\n",
"0 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" TerminateOperationsInd Terminated \\\n",
"0 NaN 0 \n",
"\n",
" TheBooksAreInCareOf \\\n",
"0 {u'TelephoneNumber': u'2077741822', u'NameBusiness': {u'BusinessNameLine1': u'JACKIE FENLASON DIR OF FINANCE'}, u'AddressUS': {u'City': u'PORTLAND', u'State': u'ME', u'ZIPCode': u'04101', u'AddressLine1': u'485 CONGRESS STREEt'}} \n",
"\n",
" TotLiabNetAssetsFundBalanceGrp TotReportableCompRltdOrgAmt \\\n",
"0 NaN NaN \n",
"\n",
" TotalAssets TotalAssetsBOY \\\n",
"0 {u'BOY': u'15275815', u'EOY': u'14648553'} 15275815 \n",
"\n",
" TotalAssetsBOYAmt TotalAssetsEOY TotalAssetsEOYAmt TotalAssetsGrp \\\n",
"0 NaN 14648553 NaN NaN \n",
"\n",
" TotalCompGT150K TotalCompGreaterThan150KInd TotalContributions \\\n",
"0 0 NaN 2069610 \n",
"\n",
" TotalContributionsAmt TotalEmployeeCnt TotalExpensesCurrentYear \\\n",
"0 NaN NaN 2224727 \n",
"\n",
" TotalExpensesPriorYear \\\n",
"0 2508005 \n",
"\n",
" TotalFunctionalExpenses \\\n",
"0 {u'ManagementAndGeneral': u'810283', u'ProgramServices': u'1192385', u'Fundraising': u'222059', u'Total': u'2224727'} \n",
"\n",
" TotalFunctionalExpensesGrp TotalFundrsngExpCurrentYear TotalGrossUBI \\\n",
"0 NaN 222059 0 \n",
"\n",
" TotalGrossUBIAmt TotalLiabNetAssetsFundBalances \\\n",
"0 NaN {u'BOY': u'15275815', u'EOY': u'14648553'} \n",
"\n",
" TotalLiabilities TotalLiabilitiesBOY \\\n",
"0 {u'BOY': u'2787481', u'EOY': u'1553436'} 2787481 \n",
"\n",
" TotalLiabilitiesBOYAmt TotalLiabilitiesEOY TotalLiabilitiesEOYAmt \\\n",
"0 NaN 1553436 NaN \n",
"\n",
" TotalLiabilitiesGrp TotalNbrEmployees TotalNbrVolunteers \\\n",
"0 NaN 48 165 \n",
"\n",
" TotalNetAssetsFundBalanceGrp TotalNetAssetsFundBalances \\\n",
"0 NaN {u'BOY': u'12488334', u'EOY': u'13095117'} \n",
"\n",
" TotalOfOtherProgramServiceExp TotalOfOtherProgramServiceGrnt \\\n",
"0 274266 NaN \n",
"\n",
" TotalOfOtherProgramServiceRev TotalOthProgramServiceRevGrp \\\n",
"0 195220 NaN \n",
"\n",
" TotalOthProgramServiceRevenue \\\n",
"0 {u'TotalRevenueColumn': u'79476', u'RelatedOrExemptFunctionIncome': u'79476'} \n",
"\n",
" TotalOtherCompensation TotalOtherCompensationAmt \\\n",
"0 3203 NaN \n",
"\n",
" TotalOtherProgSrvcExpenseAmt TotalOtherProgSrvcGrantAmt \\\n",
"0 NaN NaN \n",
"\n",
" TotalOtherProgSrvcRevenueAmt TotalOtherRevenue TotalProfFundrsngExpCY \\\n",
"0 NaN NaN 0 \n",
"\n",
" TotalProfFundrsngExpPriorYear TotalProgramServiceExpense \\\n",
"0 NaN 1192385 \n",
"\n",
" TotalProgramServiceExpensesAmt TotalProgramServiceRevenue \\\n",
"0 NaN 266460 \n",
"\n",
" TotalProgramServiceRevenueAmt TotalReportableCompFrmRltdOrgs \\\n",
"0 NaN 0 \n",
"\n",
" TotalReportableCompFromOrg TotalReportableCompFromOrgAmt \\\n",
"0 91402 NaN \n",
"\n",
" TotalRevenue \\\n",
"0 {u'TotalRevenueColumn': u'2712690', u'RelatedOrExemptFunctionIncome': u'266460', u'UnrelatedBusinessRevenue': u'0', u'ExclusionAmount': u'376620'} \n",
"\n",
" TotalRevenueCurrentYear TotalRevenueGrp TotalRevenuePriorYear \\\n",
"0 2712690 NaN 2991886 \n",
"\n",
" TotalVolunteersCnt TransactionRelatedEntity \\\n",
"0 NaN NaN \n",
"\n",
" TransfersToExemptNonChrtblOrg Travel \\\n",
"0 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1043400001'} NaN \n",
"\n",
" TravelEntrtnmntPublicOfficials TravelGrp TrnsfrExmptNonChrtblRltdOrgInd \\\n",
"0 NaN NaN NaN \n",
"\n",
" TypeOfOrganizationCorpInd TypeOfOrganizationCorporation \\\n",
"0 NaN X \n",
"\n",
" URL \\\n",
"0 https://s3.amazonaws.com/irs-form-990/201100709349300510_public.xml \n",
"\n",
" USAddress UnrelatedBusIncmOverLimitInd UnrelatedBusinessIncome \\\n",
"0 NaN NaN 0 \n",
"\n",
" UnrestrictedNetAssets UnrestrictedNetAssetsGrp \\\n",
"0 {u'BOY': u'10167218', u'EOY': u'10950248'} NaN \n",
"\n",
" UnsecuredNotesLoansPayable UnsecuredNotesLoansPayableGrp UponRequest \\\n",
"0 NaN NaN X \n",
"\n",
" UponRequestInd VotingMembersGoverningBodyCnt VotingMembersIndependentCnt \\\n",
"0 NaN NaN NaN \n",
"\n",
" WebSite WebsiteAddressTxt WhistleblowerPolicy \\\n",
"0 WWW.MAINEHISTORY.ORG NaN 1 \n",
"\n",
" WhistleblowerPolicyInd YearFormation _id \n",
"0 NaN 1822 5adf754335fd3fd83d06d437 "
]
},
"execution_count": 502,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.DataFrame(list(filings.find()))\n",
"print \"Number of columns:\", len(df.columns)\n",
"print \"Number of observations:\", len(df)\n",
"df[:1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Inspect a few basic variables"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>All are 990 filings (as opposed to, for instance, 990EZ or 990PF)"
]
},
{
"cell_type": "code",
"execution_count": 503,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"990 33\n",
"Name: FormType, dtype: int64"
]
},
"execution_count": 503,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['FormType'].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Create FYE (fiscal year ending) variable form *TaxPeriod*. We only need the first four characters from the variable."
]
},
{
"cell_type": "code",
"execution_count": 504,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"201506 2\n",
"201606 2\n",
"201706 2\n",
"201412 2\n",
"201312 2\n",
"201306 2\n",
"201212 2\n",
"201206 2\n",
"201406 2\n",
"201612 2\n",
"201512 2\n",
"201005 1\n",
"201309 1\n",
"201112 1\n",
"201009 1\n",
"201409 1\n",
"201109 1\n",
"201106 1\n",
"201012 1\n",
"201105 1\n",
"201209 1\n",
"201509 1\n",
"Name: TaxPeriod, dtype: int64"
]
},
"execution_count": 504,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['TaxPeriod'].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Create the new variable *FYE* by adding 'FY' to the first four characters of *TaxPeriod*. We now thus have a variable where we can select filings by fiscal year (as opposed to, say, the year the filing was submitted)."
]
},
{
"cell_type": "code",
"execution_count": 505,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"FY2015 5\n",
"FY2014 5\n",
"FY2013 5\n",
"FY2012 5\n",
"FY2016 4\n",
"FY2011 4\n",
"FY2010 3\n",
"FY2017 2\n",
"Name: FYE, dtype: int64"
]
},
"execution_count": 505,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['FYE'] = 'FY' + df['TaxPeriod'].str[:4]\n",
"df['FYE'].value_counts()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Let's now sort our data by *EIN* and *FYE* and then check the first 10 rows with a few key columns"
]
},
{
"cell_type": "code",
"execution_count": 506,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>EIN</th>\n",
" <th>OrganizationName</th>\n",
" <th>FormType</th>\n",
" <th>SubmittedOn</th>\n",
" <th>TaxPeriod</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>010202467</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>990</td>\n",
" <td>2018-01-05</td>\n",
" <td>201612</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>010202467</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>990</td>\n",
" <td>2017-04-11</td>\n",
" <td>201512</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>010202467</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>990</td>\n",
" <td>2016-02-16</td>\n",
" <td>201412</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>010202467</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>990</td>\n",
" <td>2014-12-10</td>\n",
" <td>201312</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>010202467</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>990</td>\n",
" <td>2013-12-31</td>\n",
" <td>201212</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>010202467</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>990</td>\n",
" <td>2012-12-21</td>\n",
" <td>201112</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>010202467</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>990</td>\n",
" <td>2011-12-12</td>\n",
" <td>201012</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>010211478</td>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>990</td>\n",
" <td>2018-03-02</td>\n",
" <td>201706</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>010211478</td>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>990</td>\n",
" <td>2017-04-17</td>\n",
" <td>201606</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>010211478</td>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>990</td>\n",
" <td>2016-03-29</td>\n",
" <td>201506</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" EIN OrganizationName FormType SubmittedOn \\\n",
"29 010202467 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY 990 2018-01-05 \n",
"27 010202467 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY 990 2017-04-11 \n",
"21 010202467 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY 990 2016-02-16 \n",
"15 010202467 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY 990 2014-12-10 \n",
"8 010202467 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY 990 2013-12-31 \n",
"7 010202467 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY 990 2012-12-21 \n",
"3 010202467 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY 990 2011-12-12 \n",
"31 010211478 UNITED WAY OF EASTERN MAINE 990 2018-03-02 \n",
"26 010211478 UNITED WAY OF EASTERN MAINE 990 2017-04-17 \n",
"22 010211478 UNITED WAY OF EASTERN MAINE 990 2016-03-29 \n",
"\n",
" TaxPeriod FYE \n",
"29 201612 FY2016 \n",
"27 201512 FY2015 \n",
"21 201412 FY2014 \n",
"15 201312 FY2013 \n",
"8 201212 FY2012 \n",
"7 201112 FY2011 \n",
"3 201012 FY2010 \n",
"31 201706 FY2017 \n",
"26 201606 FY2016 \n",
"22 201506 FY2015 "
]
},
"execution_count": 506,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = df.sort_values(by=['EIN', 'FYE'], ascending=[1,0])\n",
"cols = ['EIN', 'OrganizationName', 'FormType', 'SubmittedOn', 'TaxPeriod', 'FYE']\n",
"df[cols][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Read in Codebook\n",
"Read in the codebook we saved from the last tutorial."
]
},
{
"cell_type": "code",
"execution_count": 376,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# of columns: 6\n",
"# of observations: 652\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>variable</th>\n",
" <th>description</th>\n",
" <th>line_number</th>\n",
" <th>first_year</th>\n",
" <th>last_year</th>\n",
" <th>all_years</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>AccountantCompileOrReview</td>\n",
" <td>Accountant provide compilation or review?</td>\n",
" <td>Part XII Line 2a</td>\n",
" <td>2010</td>\n",
" <td>2010</td>\n",
" <td>[2010]</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>AccountantCompileOrReviewInd</td>\n",
" <td>Accountant provide compilation or review?</td>\n",
" <td>Part XII Line 2a</td>\n",
" <td>2013</td>\n",
" <td>2015</td>\n",
" <td>[2013, 2014, 2015]</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" variable description \\\n",
"0 AccountantCompileOrReview Accountant provide compilation or review? \n",
"1 AccountantCompileOrReviewInd Accountant provide compilation or review? \n",
"\n",
" line_number first_year last_year all_years \n",
"0 Part XII Line 2a 2010 2010 [2010] \n",
"1 Part XII Line 2a 2013 2015 [2013, 2014, 2015] "
]
},
"execution_count": 376,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"variables_df = pd.read_pickle('variable descriptions 2010-2015 990 e-file data (collapsed).pkl')\n",
"print '# of columns:', len(variables_df.columns)\n",
"print '# of observations:', len(variables_df)\n",
"variables_df[:2]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use the Codebook to Identify Relevant Variables "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>I'm going to take you through a few of the common issues you will have in wrangling with the data. As a first example, let's look for all the variables that relate to Part XII, Line 2b in the 990 form. This part of the 990 is associated with 3 different variable names in the 990 e-filings XML files."
]
},
{
"cell_type": "code",
"execution_count": 377,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>variable</th>\n",
" <th>description</th>\n",
" <th>line_number</th>\n",
" <th>first_year</th>\n",
" <th>last_year</th>\n",
" <th>all_years</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>142</th>\n",
" <td>FSAudited</td>\n",
" <td>Financial sheets audited?</td>\n",
" <td>Part XII Line 2b</td>\n",
" <td>2010</td>\n",
" <td>2010</td>\n",
" <td>[2010]</td>\n",
" </tr>\n",
" <tr>\n",
" <th>143</th>\n",
" <td>FSAuditedBasisGrp</td>\n",
" <td>Basis in which the financial statements were audited by an independent accountant</td>\n",
" <td>Part XII Line 2b</td>\n",
" <td>2013</td>\n",
" <td>2015</td>\n",
" <td>[2013, 2014, 2015]</td>\n",
" </tr>\n",
" <tr>\n",
" <th>144</th>\n",
" <td>FSAuditedInd</td>\n",
" <td>Financial sheets audited?</td>\n",
" <td>Part XII Line 2b</td>\n",
" <td>2013</td>\n",
" <td>2015</td>\n",
" <td>[2013, 2014, 2015]</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" variable \\\n",
"142 FSAudited \n",
"143 FSAuditedBasisGrp \n",
"144 FSAuditedInd \n",
"\n",
" description \\\n",
"142 Financial sheets audited? \n",
"143 Basis in which the financial statements were audited by an independent accountant \n",
"144 Financial sheets audited? \n",
"\n",
" line_number first_year last_year all_years \n",
"142 Part XII Line 2b 2010 2010 [2010] \n",
"143 Part XII Line 2b 2013 2015 [2013, 2014, 2015] \n",
"144 Part XII Line 2b 2013 2015 [2013, 2014, 2015] "
]
},
"execution_count": 377,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"variables_df[variables_df['line_number']=='Part XII Line 2b']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Let's create a list of the three variable names."
]
},
{
"cell_type": "code",
"execution_count": 378,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['FSAudited', 'FSAuditedBasisGrp', 'FSAuditedInd']"
]
},
"execution_count": 378,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fs_audited_cols = variables_df[variables_df['line_number']=='Part XII Line 2b']['variable'].tolist()\n",
"fs_audited_cols"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Then inspect these three variables in the first three rows of data "
]
},
{
"cell_type": "code",
"execution_count": 379,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>FSAudited</th>\n",
" <th>FSAuditedBasisGrp</th>\n",
" <th>FSAuditedInd</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>true</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>true</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>true</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" FSAudited FSAuditedBasisGrp FSAuditedInd\n",
"29 NaN {u'SeparateBasisFinclStmtInd': u'X'} 1\n",
"27 NaN {u'SeparateBasisFinclStmtInd': u'X'} 1\n",
"21 NaN {u'SeparateBasisFinclStmtInd': u'X'} 1\n",
"15 NaN {u'SeparateBasisFinclStmtInd': u'X'} 1\n",
"8 1 NaN NaN\n",
"7 1 NaN NaN\n",
"3 1 NaN NaN\n",
"31 NaN {u'SeparateBasisFinclStmtInd': u'X'} true\n",
"26 NaN {u'SeparateBasisFinclStmtInd': u'X'} true\n",
"22 NaN {u'SeparateBasisFinclStmtInd': u'X'} true"
]
},
"execution_count": 379,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df[fs_audited_cols][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Audit Committee\n",
"Now let's try another from a different angle. We'll look for whether the organization has an audit committee. So, we'll search for variables that have \"committee\" in the description."
]
},
{
"cell_type": "code",
"execution_count": 380,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>variable</th>\n",
" <th>description</th>\n",
" <th>line_number</th>\n",
" <th>first_year</th>\n",
" <th>last_year</th>\n",
" <th>all_years</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>32</th>\n",
" <td>AuditCommittee</td>\n",
" <td>Does the organization have an audit committee?</td>\n",
" <td>Part XII Line 2c</td>\n",
" <td>2010</td>\n",
" <td>2010</td>\n",
" <td>[2010]</td>\n",
" </tr>\n",
" <tr>\n",
" <th>33</th>\n",
" <td>AuditCommitteeInd</td>\n",
" <td>Does the organization have an audit committee?</td>\n",
" <td>Part XII Line 2c</td>\n",
" <td>2013</td>\n",
" <td>2015</td>\n",
" <td>[2013, 2014, 2015]</td>\n",
" </tr>\n",
" <tr>\n",
" <th>336</th>\n",
" <td>MinutesOfCommittees</td>\n",
" <td>Minutes of committees?</td>\n",
" <td>Part VI Section A Line 8b</td>\n",
" <td>2010</td>\n",
" <td>2010</td>\n",
" <td>[2010]</td>\n",
" </tr>\n",
" <tr>\n",
" <th>337</th>\n",
" <td>MinutesOfCommitteesInd</td>\n",
" <td>Minutes of committees?</td>\n",
" <td>Part VI Section A Line 8b</td>\n",
" <td>2013</td>\n",
" <td>2015</td>\n",
" <td>[2013, 2014, 2015]</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" variable description \\\n",
"32 AuditCommittee Does the organization have an audit committee? \n",
"33 AuditCommitteeInd Does the organization have an audit committee? \n",
"336 MinutesOfCommittees Minutes of committees? \n",
"337 MinutesOfCommitteesInd Minutes of committees? \n",
"\n",
" line_number first_year last_year all_years \n",
"32 Part XII Line 2c 2010 2010 [2010] \n",
"33 Part XII Line 2c 2013 2015 [2013, 2014, 2015] \n",
"336 Part VI Section A Line 8b 2010 2010 [2010] \n",
"337 Part VI Section A Line 8b 2013 2015 [2013, 2014, 2015] "
]
},
"execution_count": 380,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"variables_df[variables_df['description'].str.contains('committee')]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>We want the first two."
]
},
{
"cell_type": "code",
"execution_count": 381,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>AuditCommitteeInd</th>\n",
" <th>AuditCommittee</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" AuditCommitteeInd AuditCommittee FYE\n",
"29 1 NaN FY2016\n",
"27 1 NaN FY2015\n",
"21 1 NaN FY2014\n",
"15 1 NaN FY2013\n",
"8 NaN 1 FY2012\n",
"7 NaN 1 FY2011\n",
"3 NaN 1 FY2010\n",
"31 true NaN FY2017\n",
"26 true NaN FY2016\n",
"22 true NaN FY2015"
]
},
"execution_count": 381,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df[['AuditCommitteeInd', 'AuditCommittee','FYE']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Wrangle the Data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>As you can see, there are two issues above. One, there is a mix of '1' and 'true' in the first variable. Two, there are two different variables, with *AuditCommittee* being used in FY2010-FY2012 and *AuditCommitteeInd* being used in FY2013-FY2017.\n",
"\n",
"So, what we will do first is create a combined variable. We can do this with the *npwhere* command. In the following code block we are creating a new variable called *audit_committee* and saying, \"If the variable *AuditCommittee* is not empty, assign that value to *audit_committee*, otherwise use the value of *AuditCommitteeInd*\". We then print out the frequencies for our new variable and the first 10 rows of our data for four chosen columns."
]
},
{
"cell_type": "code",
"execution_count": 507,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1 19\n",
"true 14\n",
"Name: audit_committee, dtype: int64 \n",
"\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>audit_committee</th>\n",
" <th>AuditCommittee</th>\n",
" <th>AuditCommitteeInd</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" audit_committee AuditCommittee AuditCommitteeInd FYE\n",
"29 1 NaN 1 FY2016\n",
"27 1 NaN 1 FY2015\n",
"21 1 NaN 1 FY2014\n",
"15 1 NaN 1 FY2013\n",
"8 1 1 NaN FY2012\n",
"7 1 1 NaN FY2011\n",
"3 1 1 NaN FY2010\n",
"31 true NaN true FY2017\n",
"26 true NaN true FY2016\n",
"22 true NaN true FY2015"
]
},
"execution_count": 507,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['audit_committee'] = np.where(df['AuditCommittee'].notnull(), df['AuditCommittee'], df['AuditCommitteeInd'])\n",
"print df['audit_committee'].value_counts(), '\\n'\n",
"df[['audit_committee', 'AuditCommittee', 'AuditCommitteeInd', 'FYE']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>We still need to change the all the 'true's to '1's. Let's do another series of *np.where* statements. This time, we are changing values of 'true' and 'false' to 1 and 0, respectively, and leaving the value alone otherwise. We then convert string versions of '1' and '0' to integers."
]
},
{
"cell_type": "code",
"execution_count": 508,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1 33\n",
"Name: audit_committee, dtype: int64 \n",
"\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>audit_committee</th>\n",
" <th>AuditCommittee</th>\n",
" <th>AuditCommitteeInd</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" audit_committee AuditCommittee AuditCommitteeInd FYE\n",
"29 1 NaN 1 FY2016\n",
"27 1 NaN 1 FY2015\n",
"21 1 NaN 1 FY2014\n",
"15 1 NaN 1 FY2013\n",
"8 1 1 NaN FY2012\n",
"7 1 1 NaN FY2011\n",
"3 1 1 NaN FY2010\n",
"31 1 NaN true FY2017\n",
"26 1 NaN true FY2016\n",
"22 1 NaN true FY2015"
]
},
"execution_count": 508,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['audit_committee'] = np.where(df['audit_committee']=='true', 1, df['audit_committee'])\n",
"df['audit_committee'] = np.where( df['audit_committee']=='false', 0, df['audit_committee'] )\n",
"df['audit_committee'] = np.where( df['audit_committee']=='1', 1, df['audit_committee'] )\n",
"df['audit_committee'] = np.where( df['audit_committee']=='0', 0, df['audit_committee'] )\n",
"#df['audit_committee'] = df['audit_committee'].astype('int') #SHORT-CUT FOR THE ABOVE TWO LINES (CONVERT VARIABLE TO INTEGER)\n",
"print df['audit_committee'].value_counts(), '\\n'\n",
"df[['audit_committee', 'AuditCommittee', 'AuditCommitteeInd', 'FYE']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Creating shortcuts\n",
"If you are using a lot of different variables you are going to run into the above issue repeatedly. In Python we can thus create a series of functions that can be used as shortcuts. First we'll create a function called 'combine' that will combine two variables. It takes as *inputs* four things: our dataset/dataframe (*df*), the name we'd like for our new variable (*newvar*), the name of the first variable to combine (*var1*), and the name of the second variable to combine (*var2*)."
]
},
{
"cell_type": "code",
"execution_count": 509,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def combine(df, newvar, var1, var2):\n",
" df[newvar] = np.where(df[var1].notnull(), df[var1], df[var2])\n",
" print df[newvar].value_counts(), '\\n'\n",
" return df[[newvar, var1, var2, 'FYE']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Let's now implement it. "
]
},
{
"cell_type": "code",
"execution_count": 510,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1 19\n",
"true 14\n",
"Name: audit_committee, dtype: int64 \n",
"\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>audit_committee</th>\n",
" <th>AuditCommittee</th>\n",
" <th>AuditCommitteeInd</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>true</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" audit_committee AuditCommittee AuditCommitteeInd FYE\n",
"29 1 NaN 1 FY2016\n",
"27 1 NaN 1 FY2015\n",
"21 1 NaN 1 FY2014\n",
"15 1 NaN 1 FY2013\n",
"8 1 1 NaN FY2012\n",
"7 1 1 NaN FY2011\n",
"3 1 1 NaN FY2010\n",
"31 true NaN true FY2017\n",
"26 true NaN true FY2016\n",
"22 true NaN true FY2015"
]
},
"execution_count": 510,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"combine(df, 'audit_committee', 'AuditCommittee', 'AuditCommitteeInd')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>We get the same output. The benefit is that each time we need to combine two variables we just need to use the one-liner. Now let's create a function for normalizing the scores and run it."
]
},
{
"cell_type": "code",
"execution_count": 511,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def binarize(df, variable, var1, var2):\n",
" df[variable] = np.where(df[variable]=='true', 1, df[variable])\n",
" df[variable] = np.where(df[variable]=='false', 0, df[variable])\n",
" df[variable] = np.where(df[variable]=='1', 1, df[variable])\n",
" df[variable] = np.where(df[variable]=='0', 0, df[variable])\n",
" print df[variable].value_counts(), '\\n'\n",
" return df[[variable, var1, var2, 'FYE']][:10]"
]
},
{
"cell_type": "code",
"execution_count": 512,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1 33\n",
"Name: audit_committee, dtype: int64 \n",
"\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>audit_committee</th>\n",
" <th>AuditCommittee</th>\n",
" <th>AuditCommitteeInd</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>true</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" audit_committee AuditCommittee AuditCommitteeInd FYE\n",
"29 1 NaN 1 FY2016\n",
"27 1 NaN 1 FY2015\n",
"21 1 NaN 1 FY2014\n",
"15 1 NaN 1 FY2013\n",
"8 1 1 NaN FY2012\n",
"7 1 1 NaN FY2011\n",
"3 1 1 NaN FY2010\n",
"31 1 NaN true FY2017\n",
"26 1 NaN true FY2016\n",
"22 1 NaN true FY2015"
]
},
"execution_count": 512,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"binarize(df, 'audit_committee', 'AuditCommittee', 'AuditCommitteeInd')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Wrangling with 'nested' columns\n",
"Another issue you will encounter is with columns that contain nested data. Let's take a look at a variable that is both spread across two columns and nested: *Fees for Accounting Services*. "
]
},
{
"cell_type": "code",
"execution_count": 388,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['FeesForServicesAccounting', 'FeesForServicesAccountingGrp']"
]
},
"execution_count": 388,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"accounting_fees_columns = variables_df[variables_df['line_number']=='Part IX Line 11c']['variable'].tolist()\n",
"accounting_fees_columns"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Inspecting the data, the data are 'nested' in that each cell contains a *dictionary* with two key-value pairs. We also see an additional issue: the data we want has different names in each column: it's *Total* in one column and *TotalAmt* in the other. "
]
},
{
"cell_type": "code",
"execution_count": 389,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>FYE</th>\n",
" <th>OrganizationName</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>FY2016</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>FY2015</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>FY2014</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>FY2013</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>FY2012</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" FYE OrganizationName \\\n",
"29 FY2016 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"27 FY2015 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"21 FY2014 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"15 FY2013 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"8 FY2012 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"\n",
" FeesForServicesAccounting \\\n",
"29 NaN \n",
"27 NaN \n",
"21 NaN \n",
"15 NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"\n",
" FeesForServicesAccountingGrp \n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 NaN "
]
},
"execution_count": 389,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df[['FYE', 'OrganizationName'] + accounting_fees_columns][:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>\n",
"#### Solution 1: Combine columns, then transform\n",
"There are several approaches to dealing with this issue. One is to combine the columns first -- the caveat is that we need to also fill empty rows or else our code will break. We will also slightly modify our *combine* function because *value_counts( )* will not work on a dictionary column."
]
},
{
"cell_type": "code",
"execution_count": 408,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def combine_dict(df, newvar, var1, var2):\n",
" df[newvar] = np.where(df[var1].notnull(), df[var1], df[var2])\n",
" #print df[newvar].value_counts(), '\\n'\n",
" return df[[newvar, var1, var2, 'FYE']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>First we combine the columns into our new variable *fees_for_services_accounting*"
]
},
{
"cell_type": "code",
"execution_count": 480,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>NaN</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>NaN</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" fees_for_services_accounting \\\n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccounting \\\n",
"29 NaN \n",
"27 NaN \n",
"21 NaN \n",
"15 NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccountingGrp FYE \n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} FY2016 \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} FY2015 \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} FY2014 \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} FY2013 \n",
"8 NaN FY2012 \n",
"7 NaN FY2011 \n",
"3 NaN FY2010 \n",
"31 NaN FY2017 \n",
"26 NaN FY2016 \n",
"22 NaN FY2015 "
]
},
"execution_count": 480,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"combine_dict(df, 'fees_for_services_accounting', 'FeesForServicesAccounting', 'FeesForServicesAccountingGrp')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Then fill in the empty rows with a dictionary that is empty: *{'TotalAmt': np.nan}*. We're using *np.where* again."
]
},
{
"cell_type": "code",
"execution_count": 481,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>FYE</th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>FY2016</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>FY2015</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>FY2014</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>FY2013</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>FY2012</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>FY2011</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>FY2010</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>FY2017</td>\n",
" <td>{u'TotalAmt': nan}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>FY2016</td>\n",
" <td>{u'TotalAmt': nan}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>FY2015</td>\n",
" <td>{u'TotalAmt': nan}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" FYE fees_for_services_accounting \\\n",
"29 FY2016 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 FY2015 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 FY2014 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 FY2013 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 FY2012 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 FY2011 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 FY2010 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 FY2017 {u'TotalAmt': nan} \n",
"26 FY2016 {u'TotalAmt': nan} \n",
"22 FY2015 {u'TotalAmt': nan} \n",
"\n",
" FeesForServicesAccounting \\\n",
"29 NaN \n",
"27 NaN \n",
"21 NaN \n",
"15 NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccountingGrp \n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 NaN \n",
"7 NaN \n",
"3 NaN \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN "
]
},
"execution_count": 481,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'] = np.where(df['fees_for_services_accounting'].isnull(), {'TotalAmt': np.nan}, df['fees_for_services_accounting'])\n",
"df[['FYE', 'fees_for_services_accounting', 'FeesForServicesAccounting', 'FeesForServicesAccountingGrp']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>We now can go ahead and grab the total dollar values from the nested dictionaries in this column. Depending on your discipline, you might call this *data wrangling* or *data munging* (data science), *feature engineering* (machine learning), or *data transformation* (social scientist), or some other term. Any time we want to transform a column, there are two general approaches in PANDAS. One is to loop over each row, applying the transformation one at a time. While this works for small datasets, it is inefficient and can really slow you down. You want to avoid it in favor of the second approach: one-liners that will transformations all rows in one shot. PANDAS can a wide array of built-in functions for this. In our case, however, we will use a custom function &mdash; what is known as a *lambda* function. With this function we are applying a transformation to each row &mdash; if the cell has a 'TotalAmt' key, get the value of that key (in 'float' format) and make it the new value of *fees_for_services_accounting*, otherwise leave the value as is."
]
},
{
"cell_type": "code",
"execution_count": 482,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>FYE</th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>FY2016</td>\n",
" <td>24600</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>FY2015</td>\n",
" <td>23835</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>FY2014</td>\n",
" <td>23700</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>FY2013</td>\n",
" <td>21725</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>FY2012</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>FY2011</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>FY2010</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>FY2017</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>FY2016</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>FY2015</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" FYE fees_for_services_accounting \\\n",
"29 FY2016 24600 \n",
"27 FY2015 23835 \n",
"21 FY2014 23700 \n",
"15 FY2013 21725 \n",
"8 FY2012 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 FY2011 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 FY2010 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 FY2017 NaN \n",
"26 FY2016 NaN \n",
"22 FY2015 NaN \n",
"\n",
" FeesForServicesAccounting \\\n",
"29 NaN \n",
"27 NaN \n",
"21 NaN \n",
"15 NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccountingGrp \n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 NaN \n",
"7 NaN \n",
"3 NaN \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN "
]
},
"execution_count": 482,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'] = df['fees_for_services_accounting'].apply(lambda x: x if not x.get('TotalAmt') else float(x.get('TotalAmt')))\n",
"df[['FYE', 'fees_for_services_accounting', 'FeesForServicesAccounting', 'FeesForServicesAccountingGrp']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Inspecting the first ten instances, we see that the first four are 'float' type (numeric with decimals), the next three are dictionaries, and the final three are of type 'float'."
]
},
{
"cell_type": "code",
"execution_count": 485,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"29 <type 'float'>\n",
"27 <type 'float'>\n",
"21 <type 'float'>\n",
"15 <type 'float'>\n",
"8 <type 'dict'>\n",
"7 <type 'dict'>\n",
"3 <type 'dict'>\n",
"31 <type 'float'>\n",
"26 <type 'float'>\n",
"22 <type 'float'>\n",
"Name: fees_for_services_accounting, dtype: object"
]
},
"execution_count": 485,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'].map(lambda x: type(x))[:10]"
]
},
{
"cell_type": "code",
"execution_count": 486,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>FYE</th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>FY2016</td>\n",
" <td>24600.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>FY2015</td>\n",
" <td>23835.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>FY2014</td>\n",
" <td>23700.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>FY2013</td>\n",
" <td>21725.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>FY2012</td>\n",
" <td>20875.0</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>FY2011</td>\n",
" <td>19650.0</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>FY2010</td>\n",
" <td>18400.0</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>FY2017</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>FY2016</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>FY2015</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" FYE fees_for_services_accounting \\\n",
"29 FY2016 24600.0 \n",
"27 FY2015 23835.0 \n",
"21 FY2014 23700.0 \n",
"15 FY2013 21725.0 \n",
"8 FY2012 20875.0 \n",
"7 FY2011 19650.0 \n",
"3 FY2010 18400.0 \n",
"31 FY2017 NaN \n",
"26 FY2016 NaN \n",
"22 FY2015 NaN \n",
"\n",
" FeesForServicesAccounting \\\n",
"29 NaN \n",
"27 NaN \n",
"21 NaN \n",
"15 NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccountingGrp \n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 NaN \n",
"7 NaN \n",
"3 NaN \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN "
]
},
"execution_count": 486,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'] = df['fees_for_services_accounting'].apply(lambda x: x if type(x)==float else float(x.get('Total')))\n",
"df[['FYE', 'fees_for_services_accounting', 'FeesForServicesAccounting', 'FeesForServicesAccountingGrp']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Inspect the first ten instances again. Now they are all in *float* format."
]
},
{
"cell_type": "code",
"execution_count": 488,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"29 <type 'float'>\n",
"27 <type 'float'>\n",
"21 <type 'float'>\n",
"15 <type 'float'>\n",
"8 <type 'float'>\n",
"7 <type 'float'>\n",
"3 <type 'float'>\n",
"31 <type 'float'>\n",
"26 <type 'float'>\n",
"22 <type 'float'>\n",
"Name: fees_for_services_accounting, dtype: object"
]
},
"execution_count": 488,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'].map(lambda x: type(x))[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Generate the descriptive statistics for our combined variable."
]
},
{
"cell_type": "code",
"execution_count": 489,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"count 25.000000\n",
"mean 99924.520000\n",
"std 140836.720325\n",
"min 12750.000000\n",
"25% 15750.000000\n",
"50% 21000.000000\n",
"75% 183156.000000\n",
"max 436577.000000\n",
"Name: fees_for_services_accounting, dtype: float64"
]
},
"execution_count": 489,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'].describe().T"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>\n",
"#### Solution 2: Transform columns, then combine\n",
"Alternatively, you could make the transformations directly on the *FeesForServicesAccountingGrp* and *FeesForServicesAccounting* columns."
]
},
{
"cell_type": "code",
"execution_count": 494,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"29 24600\n",
"27 23835\n",
"21 23700\n",
"15 21725\n",
"8 NaN\n",
"7 NaN\n",
"3 NaN\n",
"31 NaN\n",
"26 NaN\n",
"22 NaN\n",
"Name: FeesForServicesAccountingGrp, dtype: object"
]
},
"execution_count": 494,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['FeesForServicesAccountingGrp'].apply(lambda x: np.nan if pd.isnull(x) else x.get('TotalAmt'))[:10]"
]
},
{
"cell_type": "code",
"execution_count": 493,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"29 NaN\n",
"27 NaN\n",
"21 NaN\n",
"15 NaN\n",
"8 20875\n",
"7 19650\n",
"3 18400\n",
"31 NaN\n",
"26 NaN\n",
"22 NaN\n",
"Name: FeesForServicesAccounting, dtype: object"
]
},
"execution_count": 493,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['FeesForServicesAccounting'].apply(lambda x: np.nan if pd.isnull(x) else x.get('Total'))[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Above I have not applied the changes to the two columns (as you can see in the following code block) &mdash; if you wanted to implement this approach you would simply add &nbsp; <code>df['FeesForServicesAccountingGrp'] = </code> &nbsp; and &nbsp; <code>df['FeesForServicesAccounting'] = </code>, &nbsp; respectively to the beginning of the two above code blocks, then apply our *combine* function to generate the combined *fees_for_services_accounting* variable. This all depends on your preferred workflow and, among other things, your comfort level with changing the original columns. In my *Solution 1* above I have kept the two original columns unchanged &mdash; applying all the transformations to the new column after combining."
]
},
{
"cell_type": "code",
"execution_count": 495,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>FYE</th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>FY2016</td>\n",
" <td>24600.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>FY2015</td>\n",
" <td>23835.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>FY2014</td>\n",
" <td>23700.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>FY2013</td>\n",
" <td>21725.0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>FY2012</td>\n",
" <td>20875.0</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>FY2011</td>\n",
" <td>19650.0</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>FY2010</td>\n",
" <td>18400.0</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>FY2017</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>FY2016</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>FY2015</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" FYE fees_for_services_accounting \\\n",
"29 FY2016 24600.0 \n",
"27 FY2015 23835.0 \n",
"21 FY2014 23700.0 \n",
"15 FY2013 21725.0 \n",
"8 FY2012 20875.0 \n",
"7 FY2011 19650.0 \n",
"3 FY2010 18400.0 \n",
"31 FY2017 NaN \n",
"26 FY2016 NaN \n",
"22 FY2015 NaN \n",
"\n",
" FeesForServicesAccounting \\\n",
"29 NaN \n",
"27 NaN \n",
"21 NaN \n",
"15 NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccountingGrp \n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 NaN \n",
"7 NaN \n",
"3 NaN \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN "
]
},
"execution_count": 495,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df[['FYE', 'fees_for_services_accounting', 'FeesForServicesAccounting', 'FeesForServicesAccountingGrp']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>\n",
"#### Solution 3: Combine columns then 'flatten' the nested column\n",
"Sometimes you will want to 'flatten' a nested column first before applying further transformations. Let's do this after first combining the two columns. "
]
},
{
"cell_type": "code",
"execution_count": 513,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" <th>FYE</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" <td>FY2014</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" <td>FY2013</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" <td>FY2012</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>NaN</td>\n",
" <td>FY2011</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>NaN</td>\n",
" <td>FY2010</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>FY2017</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>FY2016</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>FY2015</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" fees_for_services_accounting \\\n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccounting \\\n",
"29 NaN \n",
"27 NaN \n",
"21 NaN \n",
"15 NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN \n",
"\n",
" FeesForServicesAccountingGrp FYE \n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} FY2016 \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} FY2015 \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} FY2014 \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} FY2013 \n",
"8 NaN FY2012 \n",
"7 NaN FY2011 \n",
"3 NaN FY2010 \n",
"31 NaN FY2017 \n",
"26 NaN FY2016 \n",
"22 NaN FY2015 "
]
},
"execution_count": 513,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"combine_dict(df, 'fees_for_services_accounting', 'FeesForServicesAccounting', 'FeesForServicesAccountingGrp')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>The following code will flatten the nested data in our combined variable *fees_for_services_accounting*. It will create as many new columns as there are *keys* in our nested data &mdash; and it will prepend each new column with the prefix *fsfa*. In this case eight columns are created. "
]
},
{
"cell_type": "code",
"execution_count": 516,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>fsfa_0</th>\n",
" <th>fsfa_Fundraising</th>\n",
" <th>fsfa_ManagementAndGeneral</th>\n",
" <th>fsfa_ManagementAndGeneralAmt</th>\n",
" <th>fsfa_ProgramServices</th>\n",
" <th>fsfa_ProgramServicesAmt</th>\n",
" <th>fsfa_Total</th>\n",
" <th>fsfa_TotalAmt</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23835</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23835</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23700</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23700</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>21725</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>21725</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>20875</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>20875</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>19650</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>19650</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>18400</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>18400</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" fsfa_0 fsfa_Fundraising fsfa_ManagementAndGeneral \\\n",
"29 NaN NaN NaN \n",
"27 NaN NaN NaN \n",
"21 NaN NaN NaN \n",
"15 NaN NaN NaN \n",
"8 NaN NaN 20875 \n",
"7 NaN NaN 19650 \n",
"3 NaN NaN 18400 \n",
"31 NaN NaN NaN \n",
"26 NaN NaN NaN \n",
"22 NaN NaN NaN \n",
"\n",
" fsfa_ManagementAndGeneralAmt fsfa_ProgramServices fsfa_ProgramServicesAmt \\\n",
"29 24600 NaN NaN \n",
"27 23835 NaN NaN \n",
"21 23700 NaN NaN \n",
"15 21725 NaN NaN \n",
"8 NaN NaN NaN \n",
"7 NaN NaN NaN \n",
"3 NaN NaN NaN \n",
"31 NaN NaN NaN \n",
"26 NaN NaN NaN \n",
"22 NaN NaN NaN \n",
"\n",
" fsfa_Total fsfa_TotalAmt \n",
"29 NaN 24600 \n",
"27 NaN 23835 \n",
"21 NaN 23700 \n",
"15 NaN 21725 \n",
"8 20875 NaN \n",
"7 19650 NaN \n",
"3 18400 NaN \n",
"31 NaN NaN \n",
"26 NaN NaN \n",
"22 NaN NaN "
]
},
"execution_count": 516,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'].apply(pd.Series).add_prefix('fsfa_')[:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>We have not yet actually added the new columns to our dataframe. To add them modify the code like this:"
]
},
{
"cell_type": "code",
"execution_count": 519,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>@documentCnt</th>\n",
" <th>@documentCount</th>\n",
" <th>@documentId</th>\n",
" <th>@referenceDocumentId</th>\n",
" <th>@returnVersion</th>\n",
" <th>@xmlns</th>\n",
" <th>@xmlns:xsi</th>\n",
" <th>@xsi:schemaLocation</th>\n",
" <th>AccountantCompileOrReview</th>\n",
" <th>AccountantCompileOrReviewBasis</th>\n",
" <th>AccountantCompileOrReviewInd</th>\n",
" <th>AccountsPayableAccrExpnssGrp</th>\n",
" <th>AccountsPayableAccruedExpenses</th>\n",
" <th>AccountsReceivable</th>\n",
" <th>AccountsReceivableGrp</th>\n",
" <th>ActivitiesConductedPartnership</th>\n",
" <th>ActivitiesConductedPrtshpInd</th>\n",
" <th>Activity2</th>\n",
" <th>Activity3</th>\n",
" <th>ActivityCode</th>\n",
" <th>ActivityOrMissionDesc</th>\n",
" <th>ActivityOrMissionDescription</th>\n",
" <th>ActivityOther</th>\n",
" <th>AddressPrincipalOfficerUS</th>\n",
" <th>Advertising</th>\n",
" <th>AdvertisingGrp</th>\n",
" <th>AllOtherContributions</th>\n",
" <th>AllOtherContributionsAmt</th>\n",
" <th>AllOtherExpenses</th>\n",
" <th>AllOtherExpensesGrp</th>\n",
" <th>AnnualDisclosureCoveredPersons</th>\n",
" <th>AnnualDisclosureCoveredPrsnInd</th>\n",
" <th>AuditCommittee</th>\n",
" <th>AuditCommitteeInd</th>\n",
" <th>BackupWthldComplianceInd</th>\n",
" <th>BalanceSheetAmountsReported</th>\n",
" <th>BenefitsPaidToMembersCY</th>\n",
" <th>BenefitsPaidToMembersPriorYear</th>\n",
" <th>BenefitsToMembers</th>\n",
" <th>BenefitsToMembersGrp</th>\n",
" <th>BooksInCareOfDetail</th>\n",
" <th>BsnssRltnshpThruFamilyMember</th>\n",
" <th>BsnssRltnshpWithOrganization</th>\n",
" <th>BusinessRlnWithFamMemInd</th>\n",
" <th>BusinessRlnWithOfficerEntInd</th>\n",
" <th>BusinessRlnWithOrgMemInd</th>\n",
" <th>CYBenefitsPaidToMembersAmt</th>\n",
" <th>CYContributionsGrantsAmt</th>\n",
" <th>CYGrantsAndSimilarPaidAmt</th>\n",
" <th>CYInvestmentIncomeAmt</th>\n",
" <th>CYOtherExpensesAmt</th>\n",
" <th>CYOtherRevenueAmt</th>\n",
" <th>CYProgramServiceRevenueAmt</th>\n",
" <th>CYRevenuesLessExpensesAmt</th>\n",
" <th>CYSalariesCompEmpBnftPaidAmt</th>\n",
" <th>CYTotalExpensesAmt</th>\n",
" <th>CYTotalFundraisingExpenseAmt</th>\n",
" <th>CYTotalProfFndrsngExpnsAmt</th>\n",
" <th>CYTotalRevenueAmt</th>\n",
" <th>CashNonInterestBearing</th>\n",
" <th>CashNonInterestBearingGrp</th>\n",
" <th>ChangeToOrgDocumentsInd</th>\n",
" <th>ChangesToOrganizingDocs</th>\n",
" <th>CntrbtnsRprtdFundraisingEvents</th>\n",
" <th>CntrctRcvdGreaterThan100KCnt</th>\n",
" <th>CollectionsOfArt</th>\n",
" <th>CollectionsOfArtInd</th>\n",
" <th>CompCurrentOfcrDirectorsGrp</th>\n",
" <th>CompCurrentOfficersDirectors</th>\n",
" <th>CompDisqualPersons</th>\n",
" <th>CompDisqualPersonsGrp</th>\n",
" <th>CompensationFromOtherSources</th>\n",
" <th>CompensationFromOtherSrcsInd</th>\n",
" <th>CompensationProcessCEO</th>\n",
" <th>CompensationProcessCEOInd</th>\n",
" <th>CompensationProcessOther</th>\n",
" <th>CompensationProcessOtherInd</th>\n",
" <th>ComplianceWithBackupWitholding</th>\n",
" <th>ConferencesMeetings</th>\n",
" <th>ConferencesMeetingsGrp</th>\n",
" <th>ConflictOfInterestPolicy</th>\n",
" <th>ConflictOfInterestPolicyInd</th>\n",
" <th>ConservationEasements</th>\n",
" <th>ConservationEasementsInd</th>\n",
" <th>ConsolidatedAuditFinancialStmt</th>\n",
" <th>ConsolidatedAuditFinclStmtInd</th>\n",
" <th>ContractorCompensation</th>\n",
" <th>ContractorCompensationGrp</th>\n",
" <th>ContriRptFundraisingEventAmt</th>\n",
" <th>ContributionsGrantsCurrentYear</th>\n",
" <th>ContributionsGrantsPriorYear</th>\n",
" <th>CostOfGoodsSold</th>\n",
" <th>CostOfGoodsSoldAmt</th>\n",
" <th>CreditCounseling</th>\n",
" <th>CreditCounselingInd</th>\n",
" <th>DLN</th>\n",
" <th>DecisionsSubjectToApprovaInd</th>\n",
" <th>DecisionsSubjectToApproval</th>\n",
" <th>DeductibleArtContributionInd</th>\n",
" <th>DeductibleContributionsOfArt</th>\n",
" <th>DeductibleNonCashContriInd</th>\n",
" <th>DeductibleNonCashContributions</th>\n",
" <th>DeferredRevenue</th>\n",
" <th>DeferredRevenueGrp</th>\n",
" <th>DelegationOfManagementDuties</th>\n",
" <th>DelegationOfMgmtDutiesInd</th>\n",
" <th>DepreciationDepletion</th>\n",
" <th>DepreciationDepletionGrp</th>\n",
" <th>Desc</th>\n",
" <th>DescribedIn501C3</th>\n",
" <th>DescribedInSection501c3Ind</th>\n",
" <th>Description</th>\n",
" <th>DisregardedEntity</th>\n",
" <th>DisregardedEntityInd</th>\n",
" <th>DistributionToDonor</th>\n",
" <th>DocumentRetentionPolicy</th>\n",
" <th>DocumentRetentionPolicyInd</th>\n",
" <th>DonatedServicesAndUseFcltsAmt</th>\n",
" <th>DonorAdvisedFundInd</th>\n",
" <th>DonorAdvisedFunds</th>\n",
" <th>EIN</th>\n",
" <th>ElectionOfBoardMembers</th>\n",
" <th>ElectionOfBoardMembersInd</th>\n",
" <th>EmployeeCnt</th>\n",
" <th>EmploymentTaxReturnsFiled</th>\n",
" <th>EmploymentTaxReturnsFiledInd</th>\n",
" <th>EngagedInExcessBenefitTransInd</th>\n",
" <th>EscrowAccount</th>\n",
" <th>EscrowAccountInd</th>\n",
" <th>EscrowAccountLiability</th>\n",
" <th>EscrowAccountLiabilityGrp</th>\n",
" <th>ExcessBenefitTransaction</th>\n",
" <th>Expense</th>\n",
" <th>ExpenseAmt</th>\n",
" <th>FSAudited</th>\n",
" <th>FSAuditedBasis</th>\n",
" <th>FSAuditedBasisGrp</th>\n",
" <th>FSAuditedInd</th>\n",
" <th>FamilyOrBusinessRelationship</th>\n",
" <th>FamilyOrBusinessRlnInd</th>\n",
" <th>FederalGrantAuditPerformed</th>\n",
" <th>FederalGrantAuditPerformedInd</th>\n",
" <th>FederalGrantAuditRequired</th>\n",
" <th>FederalGrantAuditRequiredInd</th>\n",
" <th>FederatedCampaigns</th>\n",
" <th>FederatedCampaignsAmt</th>\n",
" <th>FeesForServicesAccounting</th>\n",
" <th>FeesForServicesAccountingGrp</th>\n",
" <th>FeesForServicesInvstMgmntFees</th>\n",
" <th>FeesForServicesLegal</th>\n",
" <th>FeesForServicesLegalGrp</th>\n",
" <th>FeesForServicesLobbying</th>\n",
" <th>FeesForServicesLobbyingGrp</th>\n",
" <th>FeesForServicesManagement</th>\n",
" <th>FeesForServicesManagementGrp</th>\n",
" <th>FeesForServicesOther</th>\n",
" <th>FeesForServicesOtherGrp</th>\n",
" <th>FeesForServicesProfFundraising</th>\n",
" <th>FeesForSrvcInvstMgmntFeesGrp</th>\n",
" <th>FinancialStatementConsolidated</th>\n",
" <th>FinancialStatementSeparate</th>\n",
" <th>FollowSFAS117</th>\n",
" <th>ForeignActivities</th>\n",
" <th>ForeignActivitiesInd</th>\n",
" <th>ForeignFinancialAccount</th>\n",
" <th>ForeignFinancialAccountInd</th>\n",
" <th>ForeignGrants</th>\n",
" <th>ForeignGrantsGrp</th>\n",
" <th>ForeignOffice</th>\n",
" <th>ForeignOfficeInd</th>\n",
" <th>Form8282PropertyDisposedOf</th>\n",
" <th>Form8282PropertyDisposedOfInd</th>\n",
" <th>Form990-TFiled</th>\n",
" <th>Form990PartVIISectionA</th>\n",
" <th>Form990PartVIISectionAGrp</th>\n",
" <th>Form990ProvidedToGoverningBody</th>\n",
" <th>Form990ProvidedToGvrnBodyInd</th>\n",
" <th>Form990TFiledInd</th>\n",
" <th>FormType</th>\n",
" <th>FormationYr</th>\n",
" <th>FormerOfcrEmployeesListedInd</th>\n",
" <th>FormersListed</th>\n",
" <th>FundraisingActivities</th>\n",
" <th>FundraisingActivitiesInd</th>\n",
" <th>FundraisingAmt</th>\n",
" <th>FundraisingDirectExpenses</th>\n",
" <th>FundraisingDirectExpensesAmt</th>\n",
" <th>FundraisingEvents</th>\n",
" <th>FundraisingGrossIncomeAmt</th>\n",
" <th>FundsToPayPremiums</th>\n",
" <th>GainOrLoss</th>\n",
" <th>GainOrLossGrp</th>\n",
" <th>Gaming</th>\n",
" <th>GamingActivitiesInd</th>\n",
" <th>GamingDirectExpensesAmt</th>\n",
" <th>GamingGrossIncomeAmt</th>\n",
" <th>GoverningBodyVotingMembersCnt</th>\n",
" <th>GovernmentGrants</th>\n",
" <th>GovernmentGrantsAmt</th>\n",
" <th>GrantAmt</th>\n",
" <th>GrantToRelatedPerson</th>\n",
" <th>GrantToRelatedPersonInd</th>\n",
" <th>Grants</th>\n",
" <th>GrantsAndSimilarAmntsCY</th>\n",
" <th>GrantsAndSimilarAmntsPriorYear</th>\n",
" <th>GrantsPayable</th>\n",
" <th>GrantsPayableGrp</th>\n",
" <th>GrantsToDomesticIndividuals</th>\n",
" <th>GrantsToDomesticIndividualsGrp</th>\n",
" <th>GrantsToDomesticOrgs</th>\n",
" <th>GrantsToDomesticOrgsGrp</th>\n",
" <th>GrantsToIndividuals</th>\n",
" <th>GrantsToIndividualsInd</th>\n",
" <th>GrantsToOrganizations</th>\n",
" <th>GrantsToOrganizationsInd</th>\n",
" <th>GrossAmountSalesAssets</th>\n",
" <th>GrossAmountSalesAssetsGrp</th>\n",
" <th>GrossIncomeFundraisingEvents</th>\n",
" <th>GrossIncomeGaming</th>\n",
" <th>GrossReceipts</th>\n",
" <th>GrossReceiptsAmt</th>\n",
" <th>GrossRents</th>\n",
" <th>GrossRentsGrp</th>\n",
" <th>GrossSalesOfInventory</th>\n",
" <th>GrossSalesOfInventoryAmt</th>\n",
" <th>GroupReturnForAffiliates</th>\n",
" <th>GroupReturnForAffiliatesInd</th>\n",
" <th>Hospital</th>\n",
" <th>IRPDocumentCnt</th>\n",
" <th>IRPDocumentW2GCnt</th>\n",
" <th>IRS990ScheduleA</th>\n",
" <th>IRS990ScheduleB</th>\n",
" <th>IRS990ScheduleC</th>\n",
" <th>IRS990ScheduleD</th>\n",
" <th>IRS990ScheduleF</th>\n",
" <th>IRS990ScheduleG</th>\n",
" <th>IRS990ScheduleI</th>\n",
" <th>IRS990ScheduleJ</th>\n",
" <th>IRS990ScheduleK</th>\n",
" <th>IRS990ScheduleL</th>\n",
" <th>IRS990ScheduleM</th>\n",
" <th>IRS990ScheduleO</th>\n",
" <th>IRS990ScheduleR</th>\n",
" <th>IncludeFIN48FootnoteInd</th>\n",
" <th>IncmFromInvestBondProceedsGrp</th>\n",
" <th>IncomeFromInvestBondProceeds</th>\n",
" <th>IndependentAuditFinancialStmt</th>\n",
" <th>IndependentAuditFinclStmtInd</th>\n",
" <th>IndependentVotingMemberCnt</th>\n",
" <th>IndivRcvdGreaterThan100KCnt</th>\n",
" <th>IndoorTanningServices</th>\n",
" <th>IndoorTanningServicesInd</th>\n",
" <th>InfoInScheduleOPartIII</th>\n",
" <th>InfoInScheduleOPartIIIInd</th>\n",
" <th>InfoInScheduleOPartIX</th>\n",
" <th>InfoInScheduleOPartIXInd</th>\n",
" <th>InfoInScheduleOPartVI</th>\n",
" <th>InfoInScheduleOPartVIIInd</th>\n",
" <th>InfoInScheduleOPartVIInd</th>\n",
" <th>InfoInScheduleOPartX</th>\n",
" <th>InfoInScheduleOPartXI</th>\n",
" <th>InfoInScheduleOPartXII</th>\n",
" <th>InfoInScheduleOPartXIIInd</th>\n",
" <th>InfoInScheduleOPartXIInd</th>\n",
" <th>InformationTechnology</th>\n",
" <th>InformationTechnologyGrp</th>\n",
" <th>Insurance</th>\n",
" <th>InsuranceGrp</th>\n",
" <th>IntangibleAssets</th>\n",
" <th>IntangibleAssetsGrp</th>\n",
" <th>Interest</th>\n",
" <th>InterestGrp</th>\n",
" <th>InventoriesForSaleOrUse</th>\n",
" <th>InventoriesForSaleOrUseGrp</th>\n",
" <th>InvestTaxExemptBonds</th>\n",
" <th>InvestTaxExemptBondsInd</th>\n",
" <th>InvestmentInJointVenture</th>\n",
" <th>InvestmentInJointVentureInd</th>\n",
" <th>InvestmentIncome</th>\n",
" <th>InvestmentIncomeCurrentYear</th>\n",
" <th>InvestmentIncomeGrp</th>\n",
" <th>InvestmentIncomePriorYear</th>\n",
" <th>InvestmentsOtherSecurities</th>\n",
" <th>InvestmentsOtherSecuritiesGrp</th>\n",
" <th>InvestmentsProgramRelated</th>\n",
" <th>InvestmentsProgramRelatedGrp</th>\n",
" <th>InvestmentsPubTradedSecGrp</th>\n",
" <th>InvestmentsPubTradedSecurities</th>\n",
" <th>LandBldgEquipAccumDeprecAmt</th>\n",
" <th>LandBldgEquipBasisNetGrp</th>\n",
" <th>LandBldgEquipCostOrOtherBssAmt</th>\n",
" <th>LandBldgEquipmentAccumDeprec</th>\n",
" <th>LandBuildingsEquipmentBasis</th>\n",
" <th>LandBuildingsEquipmentBasisNet</th>\n",
" <th>LastUpdated</th>\n",
" <th>LegalDomicileStateCd</th>\n",
" <th>LessCostOthBasisSalesExpenses</th>\n",
" <th>LessCostOthBasisSalesExpnssGrp</th>\n",
" <th>LessRentalExpenses</th>\n",
" <th>LessRentalExpensesGrp</th>\n",
" <th>LoanOutstandingInd</th>\n",
" <th>LoanToOfficerOrDQP</th>\n",
" <th>LoansFromOfficersDirectors</th>\n",
" <th>LoansFromOfficersDirectorsGrp</th>\n",
" <th>LobbyingActivities</th>\n",
" <th>LobbyingActivitiesInd</th>\n",
" <th>LocalChapters</th>\n",
" <th>LocalChaptersInd</th>\n",
" <th>MaterialDiversionOrMisuse</th>\n",
" <th>MaterialDiversionOrMisuseInd</th>\n",
" <th>MembersOrStockholders</th>\n",
" <th>MembersOrStockholdersInd</th>\n",
" <th>MembershipDues</th>\n",
" <th>MembershipDuesAmt</th>\n",
" <th>MethodOfAccountingAccrual</th>\n",
" <th>MethodOfAccountingAccrualInd</th>\n",
" <th>MinutesOfCommittees</th>\n",
" <th>MinutesOfCommitteesInd</th>\n",
" <th>MinutesOfGoverningBody</th>\n",
" <th>MinutesOfGoverningBodyInd</th>\n",
" <th>MissionDesc</th>\n",
" <th>MissionDescription</th>\n",
" <th>MoreThan5000KToIndividuals</th>\n",
" <th>MoreThan5000KToIndividualsInd</th>\n",
" <th>MoreThan5000KToOrgInd</th>\n",
" <th>MoreThan5000KToOrganizations</th>\n",
" <th>MortNotesPyblSecuredInvestProp</th>\n",
" <th>MortgNotesPyblScrdInvstPropGrp</th>\n",
" <th>NameOfForeignCountry</th>\n",
" <th>NameOfPrincipalOfficerPerson</th>\n",
" <th>NbrIndependentVotingMembers</th>\n",
" <th>NbrVotingGoverningBodyMembers</th>\n",
" <th>NbrVotingMembersGoverningBody</th>\n",
" <th>NetAssetsOrFundBalancesBOY</th>\n",
" <th>NetAssetsOrFundBalancesBOYAmt</th>\n",
" <th>NetAssetsOrFundBalancesEOY</th>\n",
" <th>NetAssetsOrFundBalancesEOYAmt</th>\n",
" <th>NetGainOrLossInvestments</th>\n",
" <th>NetGainOrLossInvestmentsGrp</th>\n",
" <th>NetIncmFromFundraisingEvtGrp</th>\n",
" <th>NetIncomeFromFundraisingEvents</th>\n",
" <th>NetIncomeFromGaming</th>\n",
" <th>NetIncomeFromGamingGrp</th>\n",
" <th>NetIncomeOrLoss</th>\n",
" <th>NetIncomeOrLossGrp</th>\n",
" <th>NetRentalIncomeOrLoss</th>\n",
" <th>NetRentalIncomeOrLossGrp</th>\n",
" <th>NetUnrelatedBusTxblIncmAmt</th>\n",
" <th>NetUnrelatedBusinessTxblIncome</th>\n",
" <th>NetUnrlzdGainsLossesInvstAmt</th>\n",
" <th>NoListedPersonsCompensatedInd</th>\n",
" <th>NonDeductibleContributions</th>\n",
" <th>NoncashContributions</th>\n",
" <th>NoncashContributionsAmt</th>\n",
" <th>NondeductibleContributionsInd</th>\n",
" <th>NumberFormsTransmittedWith1096</th>\n",
" <th>NumberIndependentVotingMembers</th>\n",
" <th>NumberIndividualsGT100K</th>\n",
" <th>NumberOfContractorsGT100K</th>\n",
" <th>NumberOfEmployees</th>\n",
" <th>NumberW2GIncluded</th>\n",
" <th>ObjectId</th>\n",
" <th>Occupancy</th>\n",
" <th>OccupancyGrp</th>\n",
" <th>OfficeExpenses</th>\n",
" <th>OfficeExpensesGrp</th>\n",
" <th>OfficerEntityWithBsnssRltnshp</th>\n",
" <th>OfficerMailingAddress</th>\n",
" <th>OfficerMailingAddressInd</th>\n",
" <th>OnBehalfOfIssuer</th>\n",
" <th>OnBehalfOfIssuerInd</th>\n",
" <th>OperateHospitalInd</th>\n",
" <th>Organization501c</th>\n",
" <th>Organization501c3</th>\n",
" <th>Organization501c3Ind</th>\n",
" <th>OrganizationFollowsSFAS117Ind</th>\n",
" <th>OrganizationName</th>\n",
" <th>OthNotesLoansReceivableNetGrp</th>\n",
" <th>OtherAssetsTotal</th>\n",
" <th>OtherAssetsTotalGrp</th>\n",
" <th>OtherChangesInNetAssetsAmt</th>\n",
" <th>OtherEmployeeBenefits</th>\n",
" <th>OtherEmployeeBenefitsGrp</th>\n",
" <th>OtherExpensePriorYear</th>\n",
" <th>OtherExpenses</th>\n",
" <th>OtherExpensesCurrentYear</th>\n",
" <th>OtherExpensesGrp</th>\n",
" <th>OtherLiabilities</th>\n",
" <th>OtherLiabilitiesGrp</th>\n",
" <th>OtherNotesLoansReceivableNet</th>\n",
" <th>OtherRevenueCurrentYear</th>\n",
" <th>OtherRevenueMisc</th>\n",
" <th>OtherRevenueMiscGrp</th>\n",
" <th>OtherRevenuePriorYear</th>\n",
" <th>OtherRevenueTotalAmt</th>\n",
" <th>OtherSalariesAndWages</th>\n",
" <th>OtherSalariesAndWagesGrp</th>\n",
" <th>OtherWebsite</th>\n",
" <th>OtherWebsiteInd</th>\n",
" <th>OwnWebsite</th>\n",
" <th>OwnWebsiteInd</th>\n",
" <th>PYBenefitsPaidToMembersAmt</th>\n",
" <th>PYContributionsGrantsAmt</th>\n",
" <th>PYExcessBenefitTransInd</th>\n",
" <th>PYGrantsAndSimilarPaidAmt</th>\n",
" <th>PYInvestmentIncomeAmt</th>\n",
" <th>PYOtherExpensesAmt</th>\n",
" <th>PYOtherRevenueAmt</th>\n",
" <th>PYProgramServiceRevenueAmt</th>\n",
" <th>PYRevenuesLessExpensesAmt</th>\n",
" <th>PYSalariesCompEmpBnftPaidAmt</th>\n",
" <th>PYTotalExpensesAmt</th>\n",
" <th>PYTotalProfFndrsngExpnsAmt</th>\n",
" <th>PYTotalRevenueAmt</th>\n",
" <th>PartialLiquidation</th>\n",
" <th>PartialLiquidationInd</th>\n",
" <th>PayPremiumsPrsnlBnftCntrctInd</th>\n",
" <th>PaymentsToAffiliates</th>\n",
" <th>PaymentsToAffiliatesGrp</th>\n",
" <th>PayrollTaxes</th>\n",
" <th>PayrollTaxesGrp</th>\n",
" <th>PensionPlanContributions</th>\n",
" <th>PensionPlanContributionsGrp</th>\n",
" <th>PermanentlyRestrictedNetAssets</th>\n",
" <th>PermanentlyRstrNetAssetsGrp</th>\n",
" <th>PledgesAndGrantsReceivable</th>\n",
" <th>PledgesAndGrantsReceivableGrp</th>\n",
" <th>PoliticalActivities</th>\n",
" <th>PoliticalCampaignActyInd</th>\n",
" <th>PremiumsPaid</th>\n",
" <th>PrepaidExpensesDeferredCharges</th>\n",
" <th>PrepaidExpensesDefrdChargesGrp</th>\n",
" <th>PrincipalOfficerNm</th>\n",
" <th>PriorExcessBenefitTransaction</th>\n",
" <th>PriorPeriodAdjustmentsAmt</th>\n",
" <th>ProfessionalFundraising</th>\n",
" <th>ProfessionalFundraisingInd</th>\n",
" <th>ProgSrvcAccomActy2Grp</th>\n",
" <th>ProgSrvcAccomActy3Grp</th>\n",
" <th>ProgSrvcAccomActyOtherGrp</th>\n",
" <th>ProgramServiceRevenue</th>\n",
" <th>ProgramServiceRevenueCY</th>\n",
" <th>ProgramServiceRevenueGrp</th>\n",
" <th>ProgramServiceRevenuePriorYear</th>\n",
" <th>ProhibitedTaxShelterTrans</th>\n",
" <th>ProhibitedTaxShelterTransInd</th>\n",
" <th>PymtTravelEntrtnmntPubOfclGrp</th>\n",
" <th>QuidProQuoContriDisclInd</th>\n",
" <th>QuidProQuoContributions</th>\n",
" <th>QuidProQuoContributionsInd</th>\n",
" <th>QuidProQuoDisclosure</th>\n",
" <th>RcvFndsToPayPrsnlBnftCntrctInd</th>\n",
" <th>RcvblFromDisqualifiedPrsnGrp</th>\n",
" <th>ReceivablesFromDisqualPersons</th>\n",
" <th>ReceivablesFromOfficersEtc</th>\n",
" <th>ReceivablesFromOfficersEtcGrp</th>\n",
" <th>ReconcilationOtherChanges</th>\n",
" <th>ReconcilationRevenueExpenses</th>\n",
" <th>ReconcilationRevenueExpnssAmt</th>\n",
" <th>ReconciliationUnrealizedInvest</th>\n",
" <th>RegularMonitoringEnforcement</th>\n",
" <th>RegularMonitoringEnfrcInd</th>\n",
" <th>RelatedEntity</th>\n",
" <th>RelatedEntityInd</th>\n",
" <th>RelatedOrgControlledEntity</th>\n",
" <th>RelatedOrganizationCtrlEntInd</th>\n",
" <th>RentalIncomeOrLoss</th>\n",
" <th>RentalIncomeOrLossGrp</th>\n",
" <th>ReportFin48Footnote</th>\n",
" <th>ReportInvestOthSecurities</th>\n",
" <th>ReportInvestmentsOtherSecInd</th>\n",
" <th>ReportLandBldgEquip</th>\n",
" <th>ReportLandBuildingEquipmentInd</th>\n",
" <th>ReportOtherAssets</th>\n",
" <th>ReportOtherAssetsInd</th>\n",
" <th>ReportOtherLiabilities</th>\n",
" <th>ReportOtherLiabilitiesInd</th>\n",
" <th>ReportProgRelInvest</th>\n",
" <th>ReportProgramRelatedInvstInd</th>\n",
" <th>ReturnHeader</th>\n",
" <th>Revenue</th>\n",
" <th>RevenueAmt</th>\n",
" <th>RevenuesLessExpensesCY</th>\n",
" <th>RevenuesLessExpensesPriorYear</th>\n",
" <th>Royalties</th>\n",
" <th>RoyaltiesGrp</th>\n",
" <th>RoyaltiesRevenue</th>\n",
" <th>RoyaltiesRevenueGrp</th>\n",
" <th>SalariesEtcCurrentYear</th>\n",
" <th>SalariesEtcPriorYear</th>\n",
" <th>SavingsAndTempCashInvestments</th>\n",
" <th>SavingsAndTempCashInvstGrp</th>\n",
" <th>ScheduleBRequired</th>\n",
" <th>ScheduleBRequiredInd</th>\n",
" <th>ScheduleJRequired</th>\n",
" <th>ScheduleJRequiredInd</th>\n",
" <th>ScheduleORequired</th>\n",
" <th>ScheduleORequiredInd</th>\n",
" <th>School</th>\n",
" <th>SchoolOperatingInd</th>\n",
" <th>SignificantChange</th>\n",
" <th>SignificantChangeInd</th>\n",
" <th>SignificantNewProgramServices</th>\n",
" <th>SignificantNewProgramSrvcInd</th>\n",
" <th>StateLegalDomicile</th>\n",
" <th>StatesWhereCopyOfReturnIsFiled</th>\n",
" <th>StatesWhereCopyOfReturnIsFldCd</th>\n",
" <th>SubjectToProxyTax</th>\n",
" <th>SubjectToProxyTaxInd</th>\n",
" <th>SubmittedOn</th>\n",
" <th>TaxExemptBondLiabilities</th>\n",
" <th>TaxExemptBondLiabilitiesGrp</th>\n",
" <th>TaxExemptBonds</th>\n",
" <th>TaxExemptBondsInd</th>\n",
" <th>TaxPeriod</th>\n",
" <th>TaxableDistributions</th>\n",
" <th>TaxablePartyNotification</th>\n",
" <th>TaxablePartyNotificationInd</th>\n",
" <th>TempOrPermanentEndowmentsInd</th>\n",
" <th>TemporarilyRestrictedNetAssets</th>\n",
" <th>TemporarilyRstrNetAssetsGrp</th>\n",
" <th>TermOrPermanentEndowments</th>\n",
" <th>TerminateOperationsInd</th>\n",
" <th>Terminated</th>\n",
" <th>TheBooksAreInCareOf</th>\n",
" <th>TotLiabNetAssetsFundBalanceGrp</th>\n",
" <th>TotReportableCompRltdOrgAmt</th>\n",
" <th>TotalAssets</th>\n",
" <th>TotalAssetsBOY</th>\n",
" <th>TotalAssetsBOYAmt</th>\n",
" <th>TotalAssetsEOY</th>\n",
" <th>TotalAssetsEOYAmt</th>\n",
" <th>TotalAssetsGrp</th>\n",
" <th>TotalCompGT150K</th>\n",
" <th>TotalCompGreaterThan150KInd</th>\n",
" <th>TotalContributions</th>\n",
" <th>TotalContributionsAmt</th>\n",
" <th>TotalEmployeeCnt</th>\n",
" <th>TotalExpensesCurrentYear</th>\n",
" <th>TotalExpensesPriorYear</th>\n",
" <th>TotalFunctionalExpenses</th>\n",
" <th>TotalFunctionalExpensesGrp</th>\n",
" <th>TotalFundrsngExpCurrentYear</th>\n",
" <th>TotalGrossUBI</th>\n",
" <th>TotalGrossUBIAmt</th>\n",
" <th>TotalLiabNetAssetsFundBalances</th>\n",
" <th>TotalLiabilities</th>\n",
" <th>TotalLiabilitiesBOY</th>\n",
" <th>TotalLiabilitiesBOYAmt</th>\n",
" <th>TotalLiabilitiesEOY</th>\n",
" <th>TotalLiabilitiesEOYAmt</th>\n",
" <th>TotalLiabilitiesGrp</th>\n",
" <th>TotalNbrEmployees</th>\n",
" <th>TotalNbrVolunteers</th>\n",
" <th>TotalNetAssetsFundBalanceGrp</th>\n",
" <th>TotalNetAssetsFundBalances</th>\n",
" <th>TotalOfOtherProgramServiceExp</th>\n",
" <th>TotalOfOtherProgramServiceGrnt</th>\n",
" <th>TotalOfOtherProgramServiceRev</th>\n",
" <th>TotalOthProgramServiceRevGrp</th>\n",
" <th>TotalOthProgramServiceRevenue</th>\n",
" <th>TotalOtherCompensation</th>\n",
" <th>TotalOtherCompensationAmt</th>\n",
" <th>TotalOtherProgSrvcExpenseAmt</th>\n",
" <th>TotalOtherProgSrvcGrantAmt</th>\n",
" <th>TotalOtherProgSrvcRevenueAmt</th>\n",
" <th>TotalOtherRevenue</th>\n",
" <th>TotalProfFundrsngExpCY</th>\n",
" <th>TotalProfFundrsngExpPriorYear</th>\n",
" <th>TotalProgramServiceExpense</th>\n",
" <th>TotalProgramServiceExpensesAmt</th>\n",
" <th>TotalProgramServiceRevenue</th>\n",
" <th>TotalProgramServiceRevenueAmt</th>\n",
" <th>TotalReportableCompFrmRltdOrgs</th>\n",
" <th>TotalReportableCompFromOrg</th>\n",
" <th>TotalReportableCompFromOrgAmt</th>\n",
" <th>TotalRevenue</th>\n",
" <th>TotalRevenueCurrentYear</th>\n",
" <th>TotalRevenueGrp</th>\n",
" <th>TotalRevenuePriorYear</th>\n",
" <th>TotalVolunteersCnt</th>\n",
" <th>TransactionRelatedEntity</th>\n",
" <th>TransfersToExemptNonChrtblOrg</th>\n",
" <th>Travel</th>\n",
" <th>TravelEntrtnmntPublicOfficials</th>\n",
" <th>TravelGrp</th>\n",
" <th>TrnsfrExmptNonChrtblRltdOrgInd</th>\n",
" <th>TypeOfOrganizationCorpInd</th>\n",
" <th>TypeOfOrganizationCorporation</th>\n",
" <th>URL</th>\n",
" <th>USAddress</th>\n",
" <th>UnrelatedBusIncmOverLimitInd</th>\n",
" <th>UnrelatedBusinessIncome</th>\n",
" <th>UnrestrictedNetAssets</th>\n",
" <th>UnrestrictedNetAssetsGrp</th>\n",
" <th>UnsecuredNotesLoansPayable</th>\n",
" <th>UnsecuredNotesLoansPayableGrp</th>\n",
" <th>UponRequest</th>\n",
" <th>UponRequestInd</th>\n",
" <th>VotingMembersGoverningBodyCnt</th>\n",
" <th>VotingMembersIndependentCnt</th>\n",
" <th>WebSite</th>\n",
" <th>WebsiteAddressTxt</th>\n",
" <th>WhistleblowerPolicy</th>\n",
" <th>WhistleblowerPolicyInd</th>\n",
" <th>YearFormation</th>\n",
" <th>_id</th>\n",
" <th>FYE</th>\n",
" <th>audit_committee</th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>fsfa_0</th>\n",
" <th>fsfa_Fundraising</th>\n",
" <th>fsfa_ManagementAndGeneral</th>\n",
" <th>fsfa_ManagementAndGeneralAmt</th>\n",
" <th>fsfa_ProgramServices</th>\n",
" <th>fsfa_ProgramServicesAmt</th>\n",
" <th>fsfa_Total</th>\n",
" <th>fsfa_TotalAmt</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>12</td>\n",
" <td>NaN</td>\n",
" <td>RetDoc1038000001</td>\n",
" <td>RetDoc1044400001</td>\n",
" <td>2016v3.0</td>\n",
" <td>http://www.irs.gov/efile</td>\n",
" <td>http://www.w3.org/2001/XMLSchema-instance</td>\n",
" <td>http://www.irs.gov/efile</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>{u'BOYAmt': u'608210', u'EOYAmt': u'612870'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'97288', u'EOYAmt': u'154729'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>TO DEVELOP SOLUTIONS TO COMPLEX HUMAN &amp; ENVIRONMENTAL HEALTH PROBLEMS THROUGH RESEARCH &amp; EDUCATION.</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'21250', u'ProgramServicesAmt': u'3757', u'ManagementAndGeneralAmt': u'17493'}</td>\n",
" <td>NaN</td>\n",
" <td>1355629</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'389401', u'ProgramServicesAmt': u'306348', u'ManagementAndGeneralAmt': u'50796', u'FundraisingAmt': u'32257'}</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'PhoneNum': u'2072889880', u'PersonNm': u'CLAUDINE D LURVEY', u'USAddress': {u'CityNm': u'SALISBURY COVE', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'04672', u'AddressLine1Txt': u'OLD BAR HARBOR ROAD'}}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042800001'}</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042800001'}</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042800001'}</td>\n",
" <td>0</td>\n",
" <td>11105361</td>\n",
" <td>310727</td>\n",
" <td>12499</td>\n",
" <td>6984547</td>\n",
" <td>0</td>\n",
" <td>1651517</td>\n",
" <td>1027859</td>\n",
" <td>4446244</td>\n",
" <td>11741518</td>\n",
" <td>366005</td>\n",
" <td>0</td>\n",
" <td>12769377</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'2591885', u'EOYAmt': u'2198078'}</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>4</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>{u'TotalAmt': u'267714', u'ProgramServicesAmt': u'256470', u'ManagementAndGeneralAmt': u'5622', u'FundraisingAmt': u'5622'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'80692', u'ProgramServicesAmt': u'52261', u'ManagementAndGeneralAmt': u'24636', u'FundraisingAmt': u'3795'}</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>[{u'ContractorName': {u'BusinessName': {u'BusinessNameLine1Txt': u'EL SHEA INC'}}, u'CompensationAmt': u'957155', u'ContractorAddress': {u'USAddress': {u'CityNm': u'ELLSWORTH', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'04605', u'AddressLine1Txt'...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>93493318082337</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'883923', u'EOYAmt': u'1039674'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'1192633', u'ProgramServicesAmt': u'954106', u'ManagementAndGeneralAmt': u'178895', u'FundraisingAmt': u'59632'}</td>\n",
" <td>THE MDI BIOLOGICAL LABORATORY IS A VIBRANT BIOMEDICAL RESEARCH INSTITUTION FOCUSED ON THE DEVELOPMENT OF THERAPIES TO REGENERATE TISSUES AND ORGANS LOST TO DISEASE OR INJURY AND TO EXTENDING HEALTHY LIFESPAN. MDI BIOLOGICAL LABORATORY SCIENTISTS ...</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1039100001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>010202467</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>82</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>9267582</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'SeparateBasisFinclStmtInd': u'X'}</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'116314', u'ManagementAndGeneralAmt': u'116314'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'1270', u'ManagementAndGeneralAmt': u'1270'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'179642', u'ProgramServicesAmt': u'61189', u'ManagementAndGeneralAmt': u'97527', u'FundraisingAmt': u'20926'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'2705', u'ManagementAndGeneralAmt': u'2705'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040500001'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[{u'PersonNm': u'DR EDWARD J BENZ', u'IndividualTrusteeOrDirectorInd': u'X', u'AverageHoursPerWeekRt': u'2.00', u'ReportableCompFromRltdOrgAmt': u'0', u'OtherCompensationAmt': u'0', u'TitleTxt': u'VICE CHAIRMAN OF THE BOARD', u'OfficerInd': u'X',...</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>990</td>\n",
" <td>1898</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'OtherAmt': u'-119368'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24</td>\n",
" <td>NaN</td>\n",
" <td>9747587</td>\n",
" <td>310727</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'294987', u'ProgramServicesAmt': u'294987'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'15740', u'ProgramServicesAmt': u'15740'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1041900001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1041900001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'OtherAmt': u'35500'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>12924245</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>161</td>\n",
" <td>0</td>\n",
" <td>{u'PublicSupportTotal170Amt': u'47410364', u'PublicOrganization170Ind': u'X', u'SubstantialContributorsTotAmt': u'1786320', u'TotalCalendarYear170Grp': {u'CurrentTaxYearMinus4YearsAmt': u'9906204', u'TotalAmt': u'49196684', u'CurrentTaxYearMinus3...</td>\n",
" <td>{u'ContributorInformationGrp': {u'ContributorBusinessName': {u'BusinessNameLine1': u'RESTRICTED'}, u'ContributorUSAddress': {u'City': u'RESTRICTED', u'State': u'RESTRICTED', u'AddressLine2': u'RESTRICTED', u'ZIPCode': u'RESTRICTED', u'AddressLine...</td>\n",
" <td>{u'DirectContactLegislatorsInd': u'0', u'PublicationsOrBroadcastInd': u'0', u'VolunteersInd': u'0', u'NotDescribedSection501c3Ind': u'0', u'MailingsMembersInd': u'0', u'TotalLobbyingExpendituresAmt': u'1270', u'MediaAdvertisementsInd': u'0', u'Ra...</td>\n",
" <td>{u'PrmnntEndowmentBalanceEOYPct': u'0.35000', u'TotalRevEtcAuditedFinclStmtAmt': u'13273967', u'OtherRevenuesNotIncludedAmt': u'-119869', u'CYEndwmtFundGrp': {u'OtherExpendituresAmt': u'819732', u'InvestmentEarningsOrLossesAmt': u'505175', u'Begi...</td>\n",
" <td>{u'TotalSpentAmt': u'25070', u'ContinutationTotalEmployeeCnt': u'0', u'ForeignPartnershipInd': u'0', u'ContinuationSpentAmt': u'0', u'TotalEmployeeCnt': u'0', u'SubtotalEmployeesCnt': u'0', u'ContinutationTotalOfficeCnt': u'0', u'ForeignCorpOwner...</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalOtherOrgCnt': u'1', u'GrantRecordsMaintainedInd': u'1', u'GrantsOtherAsstToIndivInUSGrp': [{u'CashGrantAmt': u'157509', u'GrantTypeTxt': u'STUDENT FELLOWSHIPS (STIPEND, HOUSING, MEALS, TRAVEL)', u'RecipientCnt': u'24'}, {u'CashGrantAmt': ...</td>\n",
" <td>{u'BoardOrCommitteeApprovalInd': u'X', u'SupplementalNonqualRtrPlanInd': u'0', u'RltdOrgOfficerTrstKeyEmplGrp': {u'DeferredCompRltdOrgsAmt': u'0', u'PersonNm': u'DR KEVIN STRANGE', u'CompensationBasedOnRltdOrgsAmt': u'0', u'BonusFilingOrganizatio...</td>\n",
" <td>{u'TaxExemptBondsArbitrageGrp': {u'VariableRateIssueInd': u'1', u'ExceptionToRebateInd': u'0', u'BondReferenceCd': u'A', u'WrittenProcToMonitorReqsInd': u'0', u'GrossProceedsInvestedInd': u'0', u'Form8038TFiledInd': u'0', u'HedgeIdentifiedInBksAn...</td>\n",
" <td>{u'BusTrInvolveInterestedPrsnGrp': {u'TransactionAmt': u'89348', u'RelationshipDescriptionTxt': u'WIFE OF PRESIDENT OF ORGANIZATION', u'NameOfInterested': {u'PersonNm': u'REBECCA MORRISON'}, u'TransactionDesc': u'EMPLOYEE OF ORGANIZATION - PAID S...</td>\n",
" <td>{u'ReviewProcessUnusualNCGiftsInd': u'0', u'OtherNonCashContriTableGrp': {u'NonCashCheckboxInd': u'X', u'Desc': u'INKIND', u'ContributionCnt': u'1', u'MethodOfDeterminingRevenuesTxt': u'FAIR MARKET VALUE', u'NoncashContributionsRptF990Amt': u'800...</td>\n",
" <td>{u'SupplementalInformationDetail': [{u'FormAndLineReferenceDesc': u'FORM 990, PART VI, SECTION A, LINE 6', u'ExplanationTxt': u'THERE ARE 229 MEMBERS OF THE CORPORATION,165 ARE FULL MEMBERS WITH VOTING POWER.'}, {u'FormAndLineReferenceDesc': u'FO...</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>23</td>\n",
" <td>5</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'225032', u'ProgramServicesAmt': u'134327', u'ManagementAndGeneralAmt': u'89086', u'FundraisingAmt': u'1619'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'108559', u'ProgramServicesAmt': u'14903', u'ManagementAndGeneralAmt': u'93367', u'FundraisingAmt': u'289'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'134850', u'EOYAmt': u'0'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'245155', u'ProgramServicesAmt': u'183866', u'ManagementAndGeneralAmt': u'61289'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumnAmt': u'131867', u'ExclusionAmt': u'131867'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'4764803', u'EOYAmt': u'5248614'}</td>\n",
" <td>NaN</td>\n",
" <td>13563731</td>\n",
" <td>{u'BOYAmt': u'13102988', u'EOYAmt': u'14409381'}</td>\n",
" <td>27973112</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>2018-01-18T21:29:43</td>\n",
" <td>ME</td>\n",
" <td>NaN</td>\n",
" <td>{u'OtherAmt': u'154868'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1039700001'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>2145</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>THE MDI BIOLOGICAL LABORATORY IS A RAPIDLY GROWING, INDEPENDENT NON-PROFIT BIOMEDICAL RESEARCH INSTITUTION WHOSE MISSION IS TO IMPROVE HUMAN HEALTH AND WELL-BEING THROUGH RESEARCH, EDUCATION,AND THE DEVELOPMENT OF VENTURES THAT TRANSFORM DISCOVER...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040500001'}</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040500001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23789445</td>\n",
" <td>NaN</td>\n",
" <td>25391681</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumnAmt': u'-119368', u'ExclusionAmt': u'-119368'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>387426</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>53408</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>201733189349308233</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'446738', u'ProgramServicesAmt': u'363511', u'ManagementAndGeneralAmt': u'83227'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'130565', u'ProgramServicesAmt': u'18118', u'ManagementAndGeneralAmt': u'89576', u'FundraisingAmt': u'22871'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>X</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>-1</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'353124', u'ProgramServicesAmt': u'265314', u'ManagementAndGeneralAmt': u'79536', u'FundraisingAmt': u'8274'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[{u'TotalAmt': u'2532940', u'ProgramServicesAmt': u'2532940', u'Desc': u'SUBAWARDS'}, {u'TotalAmt': u'638439', u'ProgramServicesAmt': u'625151', u'ManagementAndGeneralAmt': u'11154', u'FundraisingAmt': u'2134', u'Desc': u'SUPPLIES'}, {u'TotalAmt'...</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'4425998', u'EOYAmt': u'4181078'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'3362018', u'ProgramServicesAmt': u'2282415', u'ManagementAndGeneralAmt': u'927025', u'FundraisingAmt': u'152578'}</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>0</td>\n",
" <td>10132125</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'}</td>\n",
" <td>0</td>\n",
" <td>564945</td>\n",
" <td>7454539</td>\n",
" <td>0</td>\n",
" <td>1725888</td>\n",
" <td>559472</td>\n",
" <td>4408947</td>\n",
" <td>11863486</td>\n",
" <td>0</td>\n",
" <td>12422958</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'295748', u'ProgramServicesAmt': u'207097', u'ManagementAndGeneralAmt': u'75624', u'FundraisingAmt': u'13027'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'167640', u'ProgramServicesAmt': u'111231', u'ManagementAndGeneralAmt': u'49277', u'FundraisingAmt': u'7132'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'4435347', u'EOYAmt': u'4448382'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'1944094', u'EOYAmt': u'2053078'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1039700001'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'20950', u'EOYAmt': u'5070'}</td>\n",
" <td>KEVIN STRANGE PHD</td>\n",
" <td>NaN</td>\n",
" <td>186952</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>[{u'TotalRevenueColumnAmt': u'435407', u'BusinessCd': u'611710', u'RelatedOrExemptFuncIncomeAmt': u'435407', u'Desc': u'CONFER &amp; COURSE FEES'}, {u'TotalRevenueColumnAmt': u'429819', u'BusinessCd': u'541700', u'RelatedOrExemptFuncIncomeAmt': u'429...</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1027859</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>{u'Filer': {u'BusinessNameControlTxt': u'MOUN', u'EIN': u'010202467', u'PhoneNum': u'2072883605', u'USAddress': {u'CityNm': u'SALISBURY COVE', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'046720035', u'AddressLine1Txt': u'PO BOX 35'}, u'BusinessNam...</td>\n",
" <td>NaN</td>\n",
" <td>1651517</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'7050718', u'EOYAmt': u'7156353'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1234500001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042400001'}</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>ME</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'0', u'@referenceDocumentId': u'RetDoc1039700001'}</td>\n",
" <td>2018-01-05</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1053100001'}</td>\n",
" <td>201612</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>{u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'}</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'8149926', u'EOYAmt': u'8065808'}</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'29707576', u'EOYAmt': u'31225303'}</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>29707576</td>\n",
" <td>NaN</td>\n",
" <td>31225303</td>\n",
" <td>{u'BOYAmt': u'29707576', u'EOYAmt': u'31225303'}</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>11105361</td>\n",
" <td>82</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'11741518', u'ProgramServicesAmt': u'9267582', u'ManagementAndGeneralAmt': u'2107931', u'FundraisingAmt': u'366005'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>5918131</td>\n",
" <td>NaN</td>\n",
" <td>5833622</td>\n",
" <td>{u'BOYAmt': u'5918131', u'EOYAmt': u'5833622'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'23789445', u'EOYAmt': u'25391681'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalRevenueColumnAmt': u'46317', u'RelatedOrExemptFuncIncomeAmt': u'46317'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>72494</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>9267582</td>\n",
" <td>NaN</td>\n",
" <td>1651517</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>662994</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'UnrelatedBusinessRevenueAmt': u'0', u'TotalRevenueColumnAmt': u'12769377', u'ExclusionAmt': u'12499', u'RelatedOrExemptFuncIncomeAmt': u'1651517'}</td>\n",
" <td>NaN</td>\n",
" <td>103</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'TotalAmt': u'143674', u'ProgramServicesAmt': u'125462', u'ManagementAndGeneralAmt': u'10598', u'FundraisingAmt': u'7614'}</td>\n",
" <td>0</td>\n",
" <td>X</td>\n",
" <td>NaN</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201733189349308233_public.xml</td>\n",
" <td>{u'CityNm': u'SALISBURY COVE', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'046720035', u'AddressLine1Txt': u'PO BOX 35'}</td>\n",
" <td>0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>{u'BOYAmt': u'11204172', u'EOYAmt': u'12877491'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>X</td>\n",
" <td>24</td>\n",
" <td>23</td>\n",
" <td>NaN</td>\n",
" <td>HTTPS://MDIBL.ORG</td>\n",
" <td>NaN</td>\n",
" <td>1</td>\n",
" <td>NaN</td>\n",
" <td>5adf755035fd3fd83d06d454</td>\n",
" <td>FY2016</td>\n",
" <td>1</td>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" @documentCnt @documentCount @documentId @referenceDocumentId \\\n",
"29 12 NaN RetDoc1038000001 RetDoc1044400001 \n",
"\n",
" @returnVersion @xmlns \\\n",
"29 2016v3.0 http://www.irs.gov/efile \n",
"\n",
" @xmlns:xsi @xsi:schemaLocation \\\n",
"29 http://www.w3.org/2001/XMLSchema-instance http://www.irs.gov/efile \n",
"\n",
" AccountantCompileOrReview AccountantCompileOrReviewBasis \\\n",
"29 NaN NaN \n",
"\n",
" AccountantCompileOrReviewInd AccountsPayableAccrExpnssGrp \\\n",
"29 0 {u'BOYAmt': u'608210', u'EOYAmt': u'612870'} \n",
"\n",
" AccountsPayableAccruedExpenses AccountsReceivable \\\n",
"29 NaN NaN \n",
"\n",
" AccountsReceivableGrp \\\n",
"29 {u'BOYAmt': u'97288', u'EOYAmt': u'154729'} \n",
"\n",
" ActivitiesConductedPartnership ActivitiesConductedPrtshpInd Activity2 \\\n",
"29 NaN 0 NaN \n",
"\n",
" Activity3 ActivityCode \\\n",
"29 NaN NaN \n",
"\n",
" ActivityOrMissionDesc \\\n",
"29 TO DEVELOP SOLUTIONS TO COMPLEX HUMAN & ENVIRONMENTAL HEALTH PROBLEMS THROUGH RESEARCH & EDUCATION. \n",
"\n",
" ActivityOrMissionDescription ActivityOther AddressPrincipalOfficerUS \\\n",
"29 NaN NaN NaN \n",
"\n",
" Advertising \\\n",
"29 NaN \n",
"\n",
" AdvertisingGrp \\\n",
"29 {u'TotalAmt': u'21250', u'ProgramServicesAmt': u'3757', u'ManagementAndGeneralAmt': u'17493'} \n",
"\n",
" AllOtherContributions AllOtherContributionsAmt AllOtherExpenses \\\n",
"29 NaN 1355629 NaN \n",
"\n",
" AllOtherExpensesGrp \\\n",
"29 {u'TotalAmt': u'389401', u'ProgramServicesAmt': u'306348', u'ManagementAndGeneralAmt': u'50796', u'FundraisingAmt': u'32257'} \n",
"\n",
" AnnualDisclosureCoveredPersons AnnualDisclosureCoveredPrsnInd \\\n",
"29 NaN 1 \n",
"\n",
" AuditCommittee AuditCommitteeInd BackupWthldComplianceInd \\\n",
"29 NaN 1 1 \n",
"\n",
" BalanceSheetAmountsReported BenefitsPaidToMembersCY \\\n",
"29 NaN NaN \n",
"\n",
" BenefitsPaidToMembersPriorYear BenefitsToMembers BenefitsToMembersGrp \\\n",
"29 NaN NaN NaN \n",
"\n",
" BooksInCareOfDetail \\\n",
"29 {u'PhoneNum': u'2072889880', u'PersonNm': u'CLAUDINE D LURVEY', u'USAddress': {u'CityNm': u'SALISBURY COVE', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'04672', u'AddressLine1Txt': u'OLD BAR HARBOR ROAD'}} \n",
"\n",
" BsnssRltnshpThruFamilyMember BsnssRltnshpWithOrganization \\\n",
"29 NaN NaN \n",
"\n",
" BusinessRlnWithFamMemInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042800001'} \n",
"\n",
" BusinessRlnWithOfficerEntInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042800001'} \n",
"\n",
" BusinessRlnWithOrgMemInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042800001'} \n",
"\n",
" CYBenefitsPaidToMembersAmt CYContributionsGrantsAmt \\\n",
"29 0 11105361 \n",
"\n",
" CYGrantsAndSimilarPaidAmt CYInvestmentIncomeAmt CYOtherExpensesAmt \\\n",
"29 310727 12499 6984547 \n",
"\n",
" CYOtherRevenueAmt CYProgramServiceRevenueAmt CYRevenuesLessExpensesAmt \\\n",
"29 0 1651517 1027859 \n",
"\n",
" CYSalariesCompEmpBnftPaidAmt CYTotalExpensesAmt \\\n",
"29 4446244 11741518 \n",
"\n",
" CYTotalFundraisingExpenseAmt CYTotalProfFndrsngExpnsAmt CYTotalRevenueAmt \\\n",
"29 366005 0 12769377 \n",
"\n",
" CashNonInterestBearing CashNonInterestBearingGrp \\\n",
"29 NaN {u'BOYAmt': u'2591885', u'EOYAmt': u'2198078'} \n",
"\n",
" ChangeToOrgDocumentsInd ChangesToOrganizingDocs \\\n",
"29 0 NaN \n",
"\n",
" CntrbtnsRprtdFundraisingEvents CntrctRcvdGreaterThan100KCnt \\\n",
"29 NaN 4 \n",
"\n",
" CollectionsOfArt \\\n",
"29 NaN \n",
"\n",
" CollectionsOfArtInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" CompCurrentOfcrDirectorsGrp \\\n",
"29 {u'TotalAmt': u'267714', u'ProgramServicesAmt': u'256470', u'ManagementAndGeneralAmt': u'5622', u'FundraisingAmt': u'5622'} \n",
"\n",
" CompCurrentOfficersDirectors CompDisqualPersons CompDisqualPersonsGrp \\\n",
"29 NaN NaN NaN \n",
"\n",
" CompensationFromOtherSources CompensationFromOtherSrcsInd \\\n",
"29 NaN 0 \n",
"\n",
" CompensationProcessCEO CompensationProcessCEOInd CompensationProcessOther \\\n",
"29 NaN 1 NaN \n",
"\n",
" CompensationProcessOtherInd ComplianceWithBackupWitholding \\\n",
"29 1 NaN \n",
"\n",
" ConferencesMeetings \\\n",
"29 NaN \n",
"\n",
" ConferencesMeetingsGrp \\\n",
"29 {u'TotalAmt': u'80692', u'ProgramServicesAmt': u'52261', u'ManagementAndGeneralAmt': u'24636', u'FundraisingAmt': u'3795'} \n",
"\n",
" ConflictOfInterestPolicy ConflictOfInterestPolicyInd ConservationEasements \\\n",
"29 NaN 1 NaN \n",
"\n",
" ConservationEasementsInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ConsolidatedAuditFinancialStmt \\\n",
"29 NaN \n",
"\n",
" ConsolidatedAuditFinclStmtInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ContractorCompensation \\\n",
"29 NaN \n",
"\n",
" ContractorCompensationGrp \\\n",
"29 [{u'ContractorName': {u'BusinessName': {u'BusinessNameLine1Txt': u'EL SHEA INC'}}, u'CompensationAmt': u'957155', u'ContractorAddress': {u'USAddress': {u'CityNm': u'ELLSWORTH', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'04605', u'AddressLine1Txt'... \n",
"\n",
" ContriRptFundraisingEventAmt ContributionsGrantsCurrentYear \\\n",
"29 NaN NaN \n",
"\n",
" ContributionsGrantsPriorYear CostOfGoodsSold CostOfGoodsSoldAmt \\\n",
"29 NaN NaN NaN \n",
"\n",
" CreditCounseling \\\n",
"29 NaN \n",
"\n",
" CreditCounselingInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" DLN DecisionsSubjectToApprovaInd DecisionsSubjectToApproval \\\n",
"29 93493318082337 0 NaN \n",
"\n",
" DeductibleArtContributionInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'} \n",
"\n",
" DeductibleContributionsOfArt \\\n",
"29 NaN \n",
"\n",
" DeductibleNonCashContriInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042900001'} \n",
"\n",
" DeductibleNonCashContributions DeferredRevenue \\\n",
"29 NaN NaN \n",
"\n",
" DeferredRevenueGrp \\\n",
"29 {u'BOYAmt': u'883923', u'EOYAmt': u'1039674'} \n",
"\n",
" DelegationOfManagementDuties DelegationOfMgmtDutiesInd \\\n",
"29 NaN 0 \n",
"\n",
" DepreciationDepletion \\\n",
"29 NaN \n",
"\n",
" DepreciationDepletionGrp \\\n",
"29 {u'TotalAmt': u'1192633', u'ProgramServicesAmt': u'954106', u'ManagementAndGeneralAmt': u'178895', u'FundraisingAmt': u'59632'} \n",
"\n",
" Desc \\\n",
"29 THE MDI BIOLOGICAL LABORATORY IS A VIBRANT BIOMEDICAL RESEARCH INSTITUTION FOCUSED ON THE DEVELOPMENT OF THERAPIES TO REGENERATE TISSUES AND ORGANS LOST TO DISEASE OR INJURY AND TO EXTENDING HEALTHY LIFESPAN. MDI BIOLOGICAL LABORATORY SCIENTISTS ... \n",
"\n",
" DescribedIn501C3 \\\n",
"29 NaN \n",
"\n",
" DescribedInSection501c3Ind \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1039100001'} \n",
"\n",
" Description DisregardedEntity DisregardedEntityInd DistributionToDonor \\\n",
"29 NaN NaN 0 NaN \n",
"\n",
" DocumentRetentionPolicy DocumentRetentionPolicyInd \\\n",
"29 NaN 1 \n",
"\n",
" DonatedServicesAndUseFcltsAmt \\\n",
"29 NaN \n",
"\n",
" DonorAdvisedFundInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" DonorAdvisedFunds EIN ElectionOfBoardMembers \\\n",
"29 NaN 010202467 NaN \n",
"\n",
" ElectionOfBoardMembersInd EmployeeCnt EmploymentTaxReturnsFiled \\\n",
"29 1 82 NaN \n",
"\n",
" EmploymentTaxReturnsFiledInd \\\n",
"29 1 \n",
"\n",
" EngagedInExcessBenefitTransInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'} \n",
"\n",
" EscrowAccount EscrowAccountInd EscrowAccountLiability \\\n",
"29 NaN 0 NaN \n",
"\n",
" EscrowAccountLiabilityGrp ExcessBenefitTransaction Expense ExpenseAmt \\\n",
"29 NaN NaN NaN 9267582 \n",
"\n",
" FSAudited FSAuditedBasis FSAuditedBasisGrp \\\n",
"29 NaN NaN {u'SeparateBasisFinclStmtInd': u'X'} \n",
"\n",
" FSAuditedInd FamilyOrBusinessRelationship FamilyOrBusinessRlnInd \\\n",
"29 1 NaN 0 \n",
"\n",
" FederalGrantAuditPerformed FederalGrantAuditPerformedInd \\\n",
"29 NaN 1 \n",
"\n",
" FederalGrantAuditRequired FederalGrantAuditRequiredInd FederatedCampaigns \\\n",
"29 NaN 1 NaN \n",
"\n",
" FederatedCampaignsAmt FeesForServicesAccounting \\\n",
"29 NaN NaN \n",
"\n",
" FeesForServicesAccountingGrp \\\n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} \n",
"\n",
" FeesForServicesInvstMgmntFees FeesForServicesLegal \\\n",
"29 NaN NaN \n",
"\n",
" FeesForServicesLegalGrp \\\n",
"29 {u'TotalAmt': u'116314', u'ManagementAndGeneralAmt': u'116314'} \n",
"\n",
" FeesForServicesLobbying \\\n",
"29 NaN \n",
"\n",
" FeesForServicesLobbyingGrp \\\n",
"29 {u'TotalAmt': u'1270', u'ManagementAndGeneralAmt': u'1270'} \n",
"\n",
" FeesForServicesManagement FeesForServicesManagementGrp \\\n",
"29 NaN NaN \n",
"\n",
" FeesForServicesOther \\\n",
"29 NaN \n",
"\n",
" FeesForServicesOtherGrp \\\n",
"29 {u'TotalAmt': u'179642', u'ProgramServicesAmt': u'61189', u'ManagementAndGeneralAmt': u'97527', u'FundraisingAmt': u'20926'} \n",
"\n",
" FeesForServicesProfFundraising \\\n",
"29 NaN \n",
"\n",
" FeesForSrvcInvstMgmntFeesGrp \\\n",
"29 {u'TotalAmt': u'2705', u'ManagementAndGeneralAmt': u'2705'} \n",
"\n",
" FinancialStatementConsolidated FinancialStatementSeparate FollowSFAS117 \\\n",
"29 NaN NaN NaN \n",
"\n",
" ForeignActivities \\\n",
"29 NaN \n",
"\n",
" ForeignActivitiesInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040500001'} \n",
"\n",
" ForeignFinancialAccount ForeignFinancialAccountInd ForeignGrants \\\n",
"29 NaN 0 NaN \n",
"\n",
" ForeignGrantsGrp ForeignOffice ForeignOfficeInd Form8282PropertyDisposedOf \\\n",
"29 NaN NaN 0 NaN \n",
"\n",
" Form8282PropertyDisposedOfInd Form990-TFiled Form990PartVIISectionA \\\n",
"29 0 NaN NaN \n",
"\n",
" Form990PartVIISectionAGrp \\\n",
"29 [{u'PersonNm': u'DR EDWARD J BENZ', u'IndividualTrusteeOrDirectorInd': u'X', u'AverageHoursPerWeekRt': u'2.00', u'ReportableCompFromRltdOrgAmt': u'0', u'OtherCompensationAmt': u'0', u'TitleTxt': u'VICE CHAIRMAN OF THE BOARD', u'OfficerInd': u'X',... \n",
"\n",
" Form990ProvidedToGoverningBody Form990ProvidedToGvrnBodyInd \\\n",
"29 NaN 1 \n",
"\n",
" Form990TFiledInd FormType FormationYr FormerOfcrEmployeesListedInd \\\n",
"29 NaN 990 1898 0 \n",
"\n",
" FormersListed FundraisingActivities FundraisingActivitiesInd \\\n",
"29 NaN NaN 0 \n",
"\n",
" FundraisingAmt FundraisingDirectExpenses FundraisingDirectExpensesAmt \\\n",
"29 NaN NaN NaN \n",
"\n",
" FundraisingEvents FundraisingGrossIncomeAmt FundsToPayPremiums GainOrLoss \\\n",
"29 NaN NaN NaN NaN \n",
"\n",
" GainOrLossGrp Gaming GamingActivitiesInd \\\n",
"29 {u'OtherAmt': u'-119368'} NaN 0 \n",
"\n",
" GamingDirectExpensesAmt GamingGrossIncomeAmt GoverningBodyVotingMembersCnt \\\n",
"29 NaN NaN 24 \n",
"\n",
" GovernmentGrants GovernmentGrantsAmt GrantAmt GrantToRelatedPerson \\\n",
"29 NaN 9747587 310727 NaN \n",
"\n",
" GrantToRelatedPersonInd Grants \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'} NaN \n",
"\n",
" GrantsAndSimilarAmntsCY GrantsAndSimilarAmntsPriorYear GrantsPayable \\\n",
"29 NaN NaN NaN \n",
"\n",
" GrantsPayableGrp GrantsToDomesticIndividuals \\\n",
"29 NaN NaN \n",
"\n",
" GrantsToDomesticIndividualsGrp \\\n",
"29 {u'TotalAmt': u'294987', u'ProgramServicesAmt': u'294987'} \n",
"\n",
" GrantsToDomesticOrgs \\\n",
"29 NaN \n",
"\n",
" GrantsToDomesticOrgsGrp \\\n",
"29 {u'TotalAmt': u'15740', u'ProgramServicesAmt': u'15740'} \n",
"\n",
" GrantsToIndividuals \\\n",
"29 NaN \n",
"\n",
" GrantsToIndividualsInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1041900001'} \n",
"\n",
" GrantsToOrganizations \\\n",
"29 NaN \n",
"\n",
" GrantsToOrganizationsInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1041900001'} \n",
"\n",
" GrossAmountSalesAssets GrossAmountSalesAssetsGrp \\\n",
"29 NaN {u'OtherAmt': u'35500'} \n",
"\n",
" GrossIncomeFundraisingEvents GrossIncomeGaming GrossReceipts \\\n",
"29 NaN NaN NaN \n",
"\n",
" GrossReceiptsAmt GrossRents GrossRentsGrp GrossSalesOfInventory \\\n",
"29 12924245 NaN NaN NaN \n",
"\n",
" GrossSalesOfInventoryAmt GroupReturnForAffiliates \\\n",
"29 NaN NaN \n",
"\n",
" GroupReturnForAffiliatesInd Hospital IRPDocumentCnt IRPDocumentW2GCnt \\\n",
"29 0 NaN 161 0 \n",
"\n",
" IRS990ScheduleA \\\n",
"29 {u'PublicSupportTotal170Amt': u'47410364', u'PublicOrganization170Ind': u'X', u'SubstantialContributorsTotAmt': u'1786320', u'TotalCalendarYear170Grp': {u'CurrentTaxYearMinus4YearsAmt': u'9906204', u'TotalAmt': u'49196684', u'CurrentTaxYearMinus3... \n",
"\n",
" IRS990ScheduleB \\\n",
"29 {u'ContributorInformationGrp': {u'ContributorBusinessName': {u'BusinessNameLine1': u'RESTRICTED'}, u'ContributorUSAddress': {u'City': u'RESTRICTED', u'State': u'RESTRICTED', u'AddressLine2': u'RESTRICTED', u'ZIPCode': u'RESTRICTED', u'AddressLine... \n",
"\n",
" IRS990ScheduleC \\\n",
"29 {u'DirectContactLegislatorsInd': u'0', u'PublicationsOrBroadcastInd': u'0', u'VolunteersInd': u'0', u'NotDescribedSection501c3Ind': u'0', u'MailingsMembersInd': u'0', u'TotalLobbyingExpendituresAmt': u'1270', u'MediaAdvertisementsInd': u'0', u'Ra... \n",
"\n",
" IRS990ScheduleD \\\n",
"29 {u'PrmnntEndowmentBalanceEOYPct': u'0.35000', u'TotalRevEtcAuditedFinclStmtAmt': u'13273967', u'OtherRevenuesNotIncludedAmt': u'-119869', u'CYEndwmtFundGrp': {u'OtherExpendituresAmt': u'819732', u'InvestmentEarningsOrLossesAmt': u'505175', u'Begi... \n",
"\n",
" IRS990ScheduleF \\\n",
"29 {u'TotalSpentAmt': u'25070', u'ContinutationTotalEmployeeCnt': u'0', u'ForeignPartnershipInd': u'0', u'ContinuationSpentAmt': u'0', u'TotalEmployeeCnt': u'0', u'SubtotalEmployeesCnt': u'0', u'ContinutationTotalOfficeCnt': u'0', u'ForeignCorpOwner... \n",
"\n",
" IRS990ScheduleG \\\n",
"29 NaN \n",
"\n",
" IRS990ScheduleI \\\n",
"29 {u'TotalOtherOrgCnt': u'1', u'GrantRecordsMaintainedInd': u'1', u'GrantsOtherAsstToIndivInUSGrp': [{u'CashGrantAmt': u'157509', u'GrantTypeTxt': u'STUDENT FELLOWSHIPS (STIPEND, HOUSING, MEALS, TRAVEL)', u'RecipientCnt': u'24'}, {u'CashGrantAmt': ... \n",
"\n",
" IRS990ScheduleJ \\\n",
"29 {u'BoardOrCommitteeApprovalInd': u'X', u'SupplementalNonqualRtrPlanInd': u'0', u'RltdOrgOfficerTrstKeyEmplGrp': {u'DeferredCompRltdOrgsAmt': u'0', u'PersonNm': u'DR KEVIN STRANGE', u'CompensationBasedOnRltdOrgsAmt': u'0', u'BonusFilingOrganizatio... \n",
"\n",
" IRS990ScheduleK \\\n",
"29 {u'TaxExemptBondsArbitrageGrp': {u'VariableRateIssueInd': u'1', u'ExceptionToRebateInd': u'0', u'BondReferenceCd': u'A', u'WrittenProcToMonitorReqsInd': u'0', u'GrossProceedsInvestedInd': u'0', u'Form8038TFiledInd': u'0', u'HedgeIdentifiedInBksAn... \n",
"\n",
" IRS990ScheduleL \\\n",
"29 {u'BusTrInvolveInterestedPrsnGrp': {u'TransactionAmt': u'89348', u'RelationshipDescriptionTxt': u'WIFE OF PRESIDENT OF ORGANIZATION', u'NameOfInterested': {u'PersonNm': u'REBECCA MORRISON'}, u'TransactionDesc': u'EMPLOYEE OF ORGANIZATION - PAID S... \n",
"\n",
" IRS990ScheduleM \\\n",
"29 {u'ReviewProcessUnusualNCGiftsInd': u'0', u'OtherNonCashContriTableGrp': {u'NonCashCheckboxInd': u'X', u'Desc': u'INKIND', u'ContributionCnt': u'1', u'MethodOfDeterminingRevenuesTxt': u'FAIR MARKET VALUE', u'NoncashContributionsRptF990Amt': u'800... \n",
"\n",
" IRS990ScheduleO \\\n",
"29 {u'SupplementalInformationDetail': [{u'FormAndLineReferenceDesc': u'FORM 990, PART VI, SECTION A, LINE 6', u'ExplanationTxt': u'THERE ARE 229 MEMBERS OF THE CORPORATION,165 ARE FULL MEMBERS WITH VOTING POWER.'}, {u'FormAndLineReferenceDesc': u'FO... \n",
"\n",
" IRS990ScheduleR \\\n",
"29 NaN \n",
"\n",
" IncludeFIN48FootnoteInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" IncmFromInvestBondProceedsGrp IncomeFromInvestBondProceeds \\\n",
"29 NaN NaN \n",
"\n",
" IndependentAuditFinancialStmt \\\n",
"29 NaN \n",
"\n",
" IndependentAuditFinclStmtInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" IndependentVotingMemberCnt IndivRcvdGreaterThan100KCnt \\\n",
"29 23 5 \n",
"\n",
" IndoorTanningServices IndoorTanningServicesInd InfoInScheduleOPartIII \\\n",
"29 NaN 0 NaN \n",
"\n",
" InfoInScheduleOPartIIIInd InfoInScheduleOPartIX InfoInScheduleOPartIXInd \\\n",
"29 X NaN NaN \n",
"\n",
" InfoInScheduleOPartVI InfoInScheduleOPartVIIInd InfoInScheduleOPartVIInd \\\n",
"29 NaN NaN X \n",
"\n",
" InfoInScheduleOPartX InfoInScheduleOPartXI InfoInScheduleOPartXII \\\n",
"29 NaN NaN NaN \n",
"\n",
" InfoInScheduleOPartXIIInd InfoInScheduleOPartXIInd InformationTechnology \\\n",
"29 NaN X NaN \n",
"\n",
" InformationTechnologyGrp \\\n",
"29 {u'TotalAmt': u'225032', u'ProgramServicesAmt': u'134327', u'ManagementAndGeneralAmt': u'89086', u'FundraisingAmt': u'1619'} \n",
"\n",
" Insurance \\\n",
"29 NaN \n",
"\n",
" InsuranceGrp \\\n",
"29 {u'TotalAmt': u'108559', u'ProgramServicesAmt': u'14903', u'ManagementAndGeneralAmt': u'93367', u'FundraisingAmt': u'289'} \n",
"\n",
" IntangibleAssets IntangibleAssetsGrp Interest \\\n",
"29 NaN {u'BOYAmt': u'134850', u'EOYAmt': u'0'} NaN \n",
"\n",
" InterestGrp \\\n",
"29 {u'TotalAmt': u'245155', u'ProgramServicesAmt': u'183866', u'ManagementAndGeneralAmt': u'61289'} \n",
"\n",
" InventoriesForSaleOrUse InventoriesForSaleOrUseGrp InvestTaxExemptBonds \\\n",
"29 NaN NaN NaN \n",
"\n",
" InvestTaxExemptBondsInd InvestmentInJointVenture \\\n",
"29 0 NaN \n",
"\n",
" InvestmentInJointVentureInd InvestmentIncome InvestmentIncomeCurrentYear \\\n",
"29 0 NaN NaN \n",
"\n",
" InvestmentIncomeGrp \\\n",
"29 {u'TotalRevenueColumnAmt': u'131867', u'ExclusionAmt': u'131867'} \n",
"\n",
" InvestmentIncomePriorYear InvestmentsOtherSecurities \\\n",
"29 NaN NaN \n",
"\n",
" InvestmentsOtherSecuritiesGrp InvestmentsProgramRelated \\\n",
"29 NaN NaN \n",
"\n",
" InvestmentsProgramRelatedGrp \\\n",
"29 NaN \n",
"\n",
" InvestmentsPubTradedSecGrp \\\n",
"29 {u'BOYAmt': u'4764803', u'EOYAmt': u'5248614'} \n",
"\n",
" InvestmentsPubTradedSecurities LandBldgEquipAccumDeprecAmt \\\n",
"29 NaN 13563731 \n",
"\n",
" LandBldgEquipBasisNetGrp \\\n",
"29 {u'BOYAmt': u'13102988', u'EOYAmt': u'14409381'} \n",
"\n",
" LandBldgEquipCostOrOtherBssAmt LandBldgEquipmentAccumDeprec \\\n",
"29 27973112 NaN \n",
"\n",
" LandBuildingsEquipmentBasis LandBuildingsEquipmentBasisNet \\\n",
"29 NaN NaN \n",
"\n",
" LastUpdated LegalDomicileStateCd LessCostOthBasisSalesExpenses \\\n",
"29 2018-01-18T21:29:43 ME NaN \n",
"\n",
" LessCostOthBasisSalesExpnssGrp LessRentalExpenses LessRentalExpensesGrp \\\n",
"29 {u'OtherAmt': u'154868'} NaN NaN \n",
"\n",
" LoanOutstandingInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'} \n",
"\n",
" LoanToOfficerOrDQP LoansFromOfficersDirectors \\\n",
"29 NaN NaN \n",
"\n",
" LoansFromOfficersDirectorsGrp LobbyingActivities \\\n",
"29 NaN NaN \n",
"\n",
" LobbyingActivitiesInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1039700001'} \n",
"\n",
" LocalChapters LocalChaptersInd MaterialDiversionOrMisuse \\\n",
"29 NaN 0 NaN \n",
"\n",
" MaterialDiversionOrMisuseInd MembersOrStockholders \\\n",
"29 0 NaN \n",
"\n",
" MembersOrStockholdersInd MembershipDues MembershipDuesAmt \\\n",
"29 1 NaN 2145 \n",
"\n",
" MethodOfAccountingAccrual MethodOfAccountingAccrualInd MinutesOfCommittees \\\n",
"29 NaN X NaN \n",
"\n",
" MinutesOfCommitteesInd MinutesOfGoverningBody MinutesOfGoverningBodyInd \\\n",
"29 1 NaN 1 \n",
"\n",
" MissionDesc \\\n",
"29 THE MDI BIOLOGICAL LABORATORY IS A RAPIDLY GROWING, INDEPENDENT NON-PROFIT BIOMEDICAL RESEARCH INSTITUTION WHOSE MISSION IS TO IMPROVE HUMAN HEALTH AND WELL-BEING THROUGH RESEARCH, EDUCATION,AND THE DEVELOPMENT OF VENTURES THAT TRANSFORM DISCOVER... \n",
"\n",
" MissionDescription MoreThan5000KToIndividuals \\\n",
"29 NaN NaN \n",
"\n",
" MoreThan5000KToIndividualsInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040500001'} \n",
"\n",
" MoreThan5000KToOrgInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040500001'} \n",
"\n",
" MoreThan5000KToOrganizations MortNotesPyblSecuredInvestProp \\\n",
"29 NaN NaN \n",
"\n",
" MortgNotesPyblScrdInvstPropGrp NameOfForeignCountry \\\n",
"29 NaN NaN \n",
"\n",
" NameOfPrincipalOfficerPerson NbrIndependentVotingMembers \\\n",
"29 NaN NaN \n",
"\n",
" NbrVotingGoverningBodyMembers NbrVotingMembersGoverningBody \\\n",
"29 NaN NaN \n",
"\n",
" NetAssetsOrFundBalancesBOY NetAssetsOrFundBalancesBOYAmt \\\n",
"29 NaN 23789445 \n",
"\n",
" NetAssetsOrFundBalancesEOY NetAssetsOrFundBalancesEOYAmt \\\n",
"29 NaN 25391681 \n",
"\n",
" NetGainOrLossInvestments \\\n",
"29 NaN \n",
"\n",
" NetGainOrLossInvestmentsGrp \\\n",
"29 {u'TotalRevenueColumnAmt': u'-119368', u'ExclusionAmt': u'-119368'} \n",
"\n",
" NetIncmFromFundraisingEvtGrp NetIncomeFromFundraisingEvents \\\n",
"29 NaN NaN \n",
"\n",
" NetIncomeFromGaming NetIncomeFromGamingGrp NetIncomeOrLoss \\\n",
"29 NaN NaN NaN \n",
"\n",
" NetIncomeOrLossGrp NetRentalIncomeOrLoss NetRentalIncomeOrLossGrp \\\n",
"29 NaN NaN NaN \n",
"\n",
" NetUnrelatedBusTxblIncmAmt NetUnrelatedBusinessTxblIncome \\\n",
"29 0 NaN \n",
"\n",
" NetUnrlzdGainsLossesInvstAmt NoListedPersonsCompensatedInd \\\n",
"29 387426 X \n",
"\n",
" NonDeductibleContributions NoncashContributions NoncashContributionsAmt \\\n",
"29 NaN NaN 53408 \n",
"\n",
" NondeductibleContributionsInd NumberFormsTransmittedWith1096 \\\n",
"29 0 NaN \n",
"\n",
" NumberIndependentVotingMembers NumberIndividualsGT100K \\\n",
"29 NaN NaN \n",
"\n",
" NumberOfContractorsGT100K NumberOfEmployees NumberW2GIncluded \\\n",
"29 NaN NaN NaN \n",
"\n",
" ObjectId Occupancy \\\n",
"29 201733189349308233 NaN \n",
"\n",
" OccupancyGrp \\\n",
"29 {u'TotalAmt': u'446738', u'ProgramServicesAmt': u'363511', u'ManagementAndGeneralAmt': u'83227'} \n",
"\n",
" OfficeExpenses \\\n",
"29 NaN \n",
"\n",
" OfficeExpensesGrp \\\n",
"29 {u'TotalAmt': u'130565', u'ProgramServicesAmt': u'18118', u'ManagementAndGeneralAmt': u'89576', u'FundraisingAmt': u'22871'} \n",
"\n",
" OfficerEntityWithBsnssRltnshp OfficerMailingAddress \\\n",
"29 NaN NaN \n",
"\n",
" OfficerMailingAddressInd OnBehalfOfIssuer OnBehalfOfIssuerInd \\\n",
"29 0 NaN 0 \n",
"\n",
" OperateHospitalInd Organization501c Organization501c3 Organization501c3Ind \\\n",
"29 0 NaN NaN X \n",
"\n",
" OrganizationFollowsSFAS117Ind OrganizationName \\\n",
"29 X MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"\n",
" OthNotesLoansReceivableNetGrp OtherAssetsTotal OtherAssetsTotalGrp \\\n",
"29 NaN NaN NaN \n",
"\n",
" OtherChangesInNetAssetsAmt OtherEmployeeBenefits \\\n",
"29 -1 NaN \n",
"\n",
" OtherEmployeeBenefitsGrp \\\n",
"29 {u'TotalAmt': u'353124', u'ProgramServicesAmt': u'265314', u'ManagementAndGeneralAmt': u'79536', u'FundraisingAmt': u'8274'} \n",
"\n",
" OtherExpensePriorYear OtherExpenses OtherExpensesCurrentYear \\\n",
"29 NaN NaN NaN \n",
"\n",
" OtherExpensesGrp \\\n",
"29 [{u'TotalAmt': u'2532940', u'ProgramServicesAmt': u'2532940', u'Desc': u'SUBAWARDS'}, {u'TotalAmt': u'638439', u'ProgramServicesAmt': u'625151', u'ManagementAndGeneralAmt': u'11154', u'FundraisingAmt': u'2134', u'Desc': u'SUPPLIES'}, {u'TotalAmt'... \n",
"\n",
" OtherLiabilities OtherLiabilitiesGrp \\\n",
"29 NaN {u'BOYAmt': u'4425998', u'EOYAmt': u'4181078'} \n",
"\n",
" OtherNotesLoansReceivableNet OtherRevenueCurrentYear OtherRevenueMisc \\\n",
"29 NaN NaN NaN \n",
"\n",
" OtherRevenueMiscGrp OtherRevenuePriorYear OtherRevenueTotalAmt \\\n",
"29 NaN NaN NaN \n",
"\n",
" OtherSalariesAndWages \\\n",
"29 NaN \n",
"\n",
" OtherSalariesAndWagesGrp \\\n",
"29 {u'TotalAmt': u'3362018', u'ProgramServicesAmt': u'2282415', u'ManagementAndGeneralAmt': u'927025', u'FundraisingAmt': u'152578'} \n",
"\n",
" OtherWebsite OtherWebsiteInd OwnWebsite OwnWebsiteInd \\\n",
"29 NaN X NaN X \n",
"\n",
" PYBenefitsPaidToMembersAmt PYContributionsGrantsAmt \\\n",
"29 0 10132125 \n",
"\n",
" PYExcessBenefitTransInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1042800001'} \n",
"\n",
" PYGrantsAndSimilarPaidAmt PYInvestmentIncomeAmt PYOtherExpensesAmt \\\n",
"29 0 564945 7454539 \n",
"\n",
" PYOtherRevenueAmt PYProgramServiceRevenueAmt PYRevenuesLessExpensesAmt \\\n",
"29 0 1725888 559472 \n",
"\n",
" PYSalariesCompEmpBnftPaidAmt PYTotalExpensesAmt PYTotalProfFndrsngExpnsAmt \\\n",
"29 4408947 11863486 0 \n",
"\n",
" PYTotalRevenueAmt PartialLiquidation PartialLiquidationInd \\\n",
"29 12422958 NaN 0 \n",
"\n",
" PayPremiumsPrsnlBnftCntrctInd PaymentsToAffiliates PaymentsToAffiliatesGrp \\\n",
"29 NaN NaN NaN \n",
"\n",
" PayrollTaxes \\\n",
"29 NaN \n",
"\n",
" PayrollTaxesGrp \\\n",
"29 {u'TotalAmt': u'295748', u'ProgramServicesAmt': u'207097', u'ManagementAndGeneralAmt': u'75624', u'FundraisingAmt': u'13027'} \n",
"\n",
" PensionPlanContributions \\\n",
"29 NaN \n",
"\n",
" PensionPlanContributionsGrp \\\n",
"29 {u'TotalAmt': u'167640', u'ProgramServicesAmt': u'111231', u'ManagementAndGeneralAmt': u'49277', u'FundraisingAmt': u'7132'} \n",
"\n",
" PermanentlyRestrictedNetAssets \\\n",
"29 NaN \n",
"\n",
" PermanentlyRstrNetAssetsGrp PledgesAndGrantsReceivable \\\n",
"29 {u'BOYAmt': u'4435347', u'EOYAmt': u'4448382'} NaN \n",
"\n",
" PledgesAndGrantsReceivableGrp PoliticalActivities \\\n",
"29 {u'BOYAmt': u'1944094', u'EOYAmt': u'2053078'} NaN \n",
"\n",
" PoliticalCampaignActyInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1039700001'} \n",
"\n",
" PremiumsPaid PrepaidExpensesDeferredCharges \\\n",
"29 NaN NaN \n",
"\n",
" PrepaidExpensesDefrdChargesGrp PrincipalOfficerNm \\\n",
"29 {u'BOYAmt': u'20950', u'EOYAmt': u'5070'} KEVIN STRANGE PHD \n",
"\n",
" PriorExcessBenefitTransaction PriorPeriodAdjustmentsAmt \\\n",
"29 NaN 186952 \n",
"\n",
" ProfessionalFundraising ProfessionalFundraisingInd ProgSrvcAccomActy2Grp \\\n",
"29 NaN 0 NaN \n",
"\n",
" ProgSrvcAccomActy3Grp ProgSrvcAccomActyOtherGrp ProgramServiceRevenue \\\n",
"29 NaN NaN NaN \n",
"\n",
" ProgramServiceRevenueCY \\\n",
"29 NaN \n",
"\n",
" ProgramServiceRevenueGrp \\\n",
"29 [{u'TotalRevenueColumnAmt': u'435407', u'BusinessCd': u'611710', u'RelatedOrExemptFuncIncomeAmt': u'435407', u'Desc': u'CONFER & COURSE FEES'}, {u'TotalRevenueColumnAmt': u'429819', u'BusinessCd': u'541700', u'RelatedOrExemptFuncIncomeAmt': u'429... \n",
"\n",
" ProgramServiceRevenuePriorYear ProhibitedTaxShelterTrans \\\n",
"29 NaN NaN \n",
"\n",
" ProhibitedTaxShelterTransInd PymtTravelEntrtnmntPubOfclGrp \\\n",
"29 0 NaN \n",
"\n",
" QuidProQuoContriDisclInd QuidProQuoContributions \\\n",
"29 NaN NaN \n",
"\n",
" QuidProQuoContributionsInd QuidProQuoDisclosure \\\n",
"29 0 NaN \n",
"\n",
" RcvFndsToPayPrsnlBnftCntrctInd RcvblFromDisqualifiedPrsnGrp \\\n",
"29 NaN NaN \n",
"\n",
" ReceivablesFromDisqualPersons ReceivablesFromOfficersEtc \\\n",
"29 NaN NaN \n",
"\n",
" ReceivablesFromOfficersEtcGrp ReconcilationOtherChanges \\\n",
"29 NaN NaN \n",
"\n",
" ReconcilationRevenueExpenses ReconcilationRevenueExpnssAmt \\\n",
"29 NaN 1027859 \n",
"\n",
" ReconciliationUnrealizedInvest RegularMonitoringEnforcement \\\n",
"29 NaN NaN \n",
"\n",
" RegularMonitoringEnfrcInd RelatedEntity RelatedEntityInd \\\n",
"29 1 NaN 0 \n",
"\n",
" RelatedOrgControlledEntity RelatedOrganizationCtrlEntInd \\\n",
"29 NaN 0 \n",
"\n",
" RentalIncomeOrLoss RentalIncomeOrLossGrp ReportFin48Footnote \\\n",
"29 NaN NaN NaN \n",
"\n",
" ReportInvestOthSecurities \\\n",
"29 NaN \n",
"\n",
" ReportInvestmentsOtherSecInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ReportLandBldgEquip \\\n",
"29 NaN \n",
"\n",
" ReportLandBuildingEquipmentInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ReportOtherAssets \\\n",
"29 NaN \n",
"\n",
" ReportOtherAssetsInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ReportOtherLiabilities \\\n",
"29 NaN \n",
"\n",
" ReportOtherLiabilitiesInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ReportProgRelInvest \\\n",
"29 NaN \n",
"\n",
" ReportProgramRelatedInvstInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" ReturnHeader \\\n",
"29 {u'Filer': {u'BusinessNameControlTxt': u'MOUN', u'EIN': u'010202467', u'PhoneNum': u'2072883605', u'USAddress': {u'CityNm': u'SALISBURY COVE', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'046720035', u'AddressLine1Txt': u'PO BOX 35'}, u'BusinessNam... \n",
"\n",
" Revenue RevenueAmt RevenuesLessExpensesCY RevenuesLessExpensesPriorYear \\\n",
"29 NaN 1651517 NaN NaN \n",
"\n",
" Royalties RoyaltiesGrp RoyaltiesRevenue RoyaltiesRevenueGrp \\\n",
"29 NaN NaN NaN NaN \n",
"\n",
" SalariesEtcCurrentYear SalariesEtcPriorYear SavingsAndTempCashInvestments \\\n",
"29 NaN NaN NaN \n",
"\n",
" SavingsAndTempCashInvstGrp ScheduleBRequired \\\n",
"29 {u'BOYAmt': u'7050718', u'EOYAmt': u'7156353'} NaN \n",
"\n",
" ScheduleBRequiredInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1234500001'} \n",
"\n",
" ScheduleJRequired \\\n",
"29 NaN \n",
"\n",
" ScheduleJRequiredInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1042400001'} \n",
"\n",
" ScheduleORequired ScheduleORequiredInd School SchoolOperatingInd \\\n",
"29 NaN 1 NaN 0 \n",
"\n",
" SignificantChange SignificantChangeInd SignificantNewProgramServices \\\n",
"29 NaN 0 NaN \n",
"\n",
" SignificantNewProgramSrvcInd StateLegalDomicile \\\n",
"29 0 NaN \n",
"\n",
" StatesWhereCopyOfReturnIsFiled StatesWhereCopyOfReturnIsFldCd \\\n",
"29 NaN ME \n",
"\n",
" SubjectToProxyTax \\\n",
"29 NaN \n",
"\n",
" SubjectToProxyTaxInd \\\n",
"29 {u'#text': u'0', u'@referenceDocumentId': u'RetDoc1039700001'} \n",
"\n",
" SubmittedOn TaxExemptBondLiabilities TaxExemptBondLiabilitiesGrp \\\n",
"29 2018-01-05 NaN NaN \n",
"\n",
" TaxExemptBonds \\\n",
"29 NaN \n",
"\n",
" TaxExemptBondsInd TaxPeriod \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1053100001'} 201612 \n",
"\n",
" TaxableDistributions TaxablePartyNotification TaxablePartyNotificationInd \\\n",
"29 NaN NaN 0 \n",
"\n",
" TempOrPermanentEndowmentsInd \\\n",
"29 {u'#text': u'1', u'@referenceDocumentId': u'RetDoc1040000001'} \n",
"\n",
" TemporarilyRestrictedNetAssets \\\n",
"29 NaN \n",
"\n",
" TemporarilyRstrNetAssetsGrp TermOrPermanentEndowments \\\n",
"29 {u'BOYAmt': u'8149926', u'EOYAmt': u'8065808'} NaN \n",
"\n",
" TerminateOperationsInd Terminated TheBooksAreInCareOf \\\n",
"29 0 NaN NaN \n",
"\n",
" TotLiabNetAssetsFundBalanceGrp \\\n",
"29 {u'BOYAmt': u'29707576', u'EOYAmt': u'31225303'} \n",
"\n",
" TotReportableCompRltdOrgAmt TotalAssets TotalAssetsBOY TotalAssetsBOYAmt \\\n",
"29 0 NaN NaN 29707576 \n",
"\n",
" TotalAssetsEOY TotalAssetsEOYAmt \\\n",
"29 NaN 31225303 \n",
"\n",
" TotalAssetsGrp TotalCompGT150K \\\n",
"29 {u'BOYAmt': u'29707576', u'EOYAmt': u'31225303'} NaN \n",
"\n",
" TotalCompGreaterThan150KInd TotalContributions TotalContributionsAmt \\\n",
"29 1 NaN 11105361 \n",
"\n",
" TotalEmployeeCnt TotalExpensesCurrentYear TotalExpensesPriorYear \\\n",
"29 82 NaN NaN \n",
"\n",
" TotalFunctionalExpenses \\\n",
"29 NaN \n",
"\n",
" TotalFunctionalExpensesGrp \\\n",
"29 {u'TotalAmt': u'11741518', u'ProgramServicesAmt': u'9267582', u'ManagementAndGeneralAmt': u'2107931', u'FundraisingAmt': u'366005'} \n",
"\n",
" TotalFundrsngExpCurrentYear TotalGrossUBI TotalGrossUBIAmt \\\n",
"29 NaN NaN 0 \n",
"\n",
" TotalLiabNetAssetsFundBalances TotalLiabilities TotalLiabilitiesBOY \\\n",
"29 NaN NaN NaN \n",
"\n",
" TotalLiabilitiesBOYAmt TotalLiabilitiesEOY TotalLiabilitiesEOYAmt \\\n",
"29 5918131 NaN 5833622 \n",
"\n",
" TotalLiabilitiesGrp TotalNbrEmployees \\\n",
"29 {u'BOYAmt': u'5918131', u'EOYAmt': u'5833622'} NaN \n",
"\n",
" TotalNbrVolunteers TotalNetAssetsFundBalanceGrp \\\n",
"29 NaN {u'BOYAmt': u'23789445', u'EOYAmt': u'25391681'} \n",
"\n",
" TotalNetAssetsFundBalances TotalOfOtherProgramServiceExp \\\n",
"29 NaN NaN \n",
"\n",
" TotalOfOtherProgramServiceGrnt TotalOfOtherProgramServiceRev \\\n",
"29 NaN NaN \n",
"\n",
" TotalOthProgramServiceRevGrp \\\n",
"29 {u'TotalRevenueColumnAmt': u'46317', u'RelatedOrExemptFuncIncomeAmt': u'46317'} \n",
"\n",
" TotalOthProgramServiceRevenue TotalOtherCompensation \\\n",
"29 NaN NaN \n",
"\n",
" TotalOtherCompensationAmt TotalOtherProgSrvcExpenseAmt \\\n",
"29 72494 NaN \n",
"\n",
" TotalOtherProgSrvcGrantAmt TotalOtherProgSrvcRevenueAmt TotalOtherRevenue \\\n",
"29 NaN NaN NaN \n",
"\n",
" TotalProfFundrsngExpCY TotalProfFundrsngExpPriorYear \\\n",
"29 NaN NaN \n",
"\n",
" TotalProgramServiceExpense TotalProgramServiceExpensesAmt \\\n",
"29 NaN 9267582 \n",
"\n",
" TotalProgramServiceRevenue TotalProgramServiceRevenueAmt \\\n",
"29 NaN 1651517 \n",
"\n",
" TotalReportableCompFrmRltdOrgs TotalReportableCompFromOrg \\\n",
"29 NaN NaN \n",
"\n",
" TotalReportableCompFromOrgAmt TotalRevenue TotalRevenueCurrentYear \\\n",
"29 662994 NaN NaN \n",
"\n",
" TotalRevenueGrp \\\n",
"29 {u'UnrelatedBusinessRevenueAmt': u'0', u'TotalRevenueColumnAmt': u'12769377', u'ExclusionAmt': u'12499', u'RelatedOrExemptFuncIncomeAmt': u'1651517'} \n",
"\n",
" TotalRevenuePriorYear TotalVolunteersCnt TransactionRelatedEntity \\\n",
"29 NaN 103 NaN \n",
"\n",
" TransfersToExemptNonChrtblOrg Travel TravelEntrtnmntPublicOfficials \\\n",
"29 NaN NaN NaN \n",
"\n",
" TravelGrp \\\n",
"29 {u'TotalAmt': u'143674', u'ProgramServicesAmt': u'125462', u'ManagementAndGeneralAmt': u'10598', u'FundraisingAmt': u'7614'} \n",
"\n",
" TrnsfrExmptNonChrtblRltdOrgInd TypeOfOrganizationCorpInd \\\n",
"29 0 X \n",
"\n",
" TypeOfOrganizationCorporation \\\n",
"29 NaN \n",
"\n",
" URL \\\n",
"29 https://s3.amazonaws.com/irs-form-990/201733189349308233_public.xml \n",
"\n",
" USAddress \\\n",
"29 {u'CityNm': u'SALISBURY COVE', u'StateAbbreviationCd': u'ME', u'ZIPCd': u'046720035', u'AddressLine1Txt': u'PO BOX 35'} \n",
"\n",
" UnrelatedBusIncmOverLimitInd UnrelatedBusinessIncome UnrestrictedNetAssets \\\n",
"29 0 NaN NaN \n",
"\n",
" UnrestrictedNetAssetsGrp \\\n",
"29 {u'BOYAmt': u'11204172', u'EOYAmt': u'12877491'} \n",
"\n",
" UnsecuredNotesLoansPayable UnsecuredNotesLoansPayableGrp UponRequest \\\n",
"29 NaN NaN NaN \n",
"\n",
" UponRequestInd VotingMembersGoverningBodyCnt VotingMembersIndependentCnt \\\n",
"29 X 24 23 \n",
"\n",
" WebSite WebsiteAddressTxt WhistleblowerPolicy WhistleblowerPolicyInd \\\n",
"29 NaN HTTPS://MDIBL.ORG NaN 1 \n",
"\n",
" YearFormation _id FYE audit_committee \\\n",
"29 NaN 5adf755035fd3fd83d06d454 FY2016 1 \n",
"\n",
" fees_for_services_accounting fsfa_0 \\\n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} NaN \n",
"\n",
" fsfa_Fundraising fsfa_ManagementAndGeneral fsfa_ManagementAndGeneralAmt \\\n",
"29 NaN NaN 24600 \n",
"\n",
" fsfa_ProgramServices fsfa_ProgramServicesAmt fsfa_Total fsfa_TotalAmt \n",
"29 NaN NaN NaN 24600 "
]
},
"execution_count": 519,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.concat([df, df['fees_for_services_accounting'].apply(pd.Series).add_prefix('fsfa_')], axis=1)\n",
"df[:1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>To see the 8 new columns you'll have to scroll *way* over to the right. If we want to see only the new columns, we can generate a list of the new columns and set up a view of only those columns."
]
},
{
"cell_type": "code",
"execution_count": 522,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>fsfa_0</th>\n",
" <th>fsfa_Fundraising</th>\n",
" <th>fsfa_ManagementAndGeneral</th>\n",
" <th>fsfa_ManagementAndGeneralAmt</th>\n",
" <th>fsfa_ProgramServices</th>\n",
" <th>fsfa_ProgramServicesAmt</th>\n",
" <th>fsfa_Total</th>\n",
" <th>fsfa_TotalAmt</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>{u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>{u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23835</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23835</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>{u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23700</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23700</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>{u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>21725</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>21725</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>{u'ManagementAndGeneral': u'20875', u'Total': u'20875'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>20875</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>20875</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>{u'ManagementAndGeneral': u'19650', u'Total': u'19650'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>19650</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>19650</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>{u'ManagementAndGeneral': u'18400', u'Total': u'18400'}</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>18400</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>18400</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" fees_for_services_accounting fsfa_0 \\\n",
"29 {u'TotalAmt': u'24600', u'ManagementAndGeneralAmt': u'24600'} NaN \n",
"27 {u'TotalAmt': u'23835', u'ManagementAndGeneralAmt': u'23835'} NaN \n",
"21 {u'TotalAmt': u'23700', u'ManagementAndGeneralAmt': u'23700'} NaN \n",
"15 {u'TotalAmt': u'21725', u'ManagementAndGeneralAmt': u'21725'} NaN \n",
"8 {u'ManagementAndGeneral': u'20875', u'Total': u'20875'} NaN \n",
"7 {u'ManagementAndGeneral': u'19650', u'Total': u'19650'} NaN \n",
"3 {u'ManagementAndGeneral': u'18400', u'Total': u'18400'} NaN \n",
"31 NaN NaN \n",
"26 NaN NaN \n",
"22 NaN NaN \n",
"\n",
" fsfa_Fundraising fsfa_ManagementAndGeneral fsfa_ManagementAndGeneralAmt \\\n",
"29 NaN NaN 24600 \n",
"27 NaN NaN 23835 \n",
"21 NaN NaN 23700 \n",
"15 NaN NaN 21725 \n",
"8 NaN 20875 NaN \n",
"7 NaN 19650 NaN \n",
"3 NaN 18400 NaN \n",
"31 NaN NaN NaN \n",
"26 NaN NaN NaN \n",
"22 NaN NaN NaN \n",
"\n",
" fsfa_ProgramServices fsfa_ProgramServicesAmt fsfa_Total fsfa_TotalAmt \n",
"29 NaN NaN NaN 24600 \n",
"27 NaN NaN NaN 23835 \n",
"21 NaN NaN NaN 23700 \n",
"15 NaN NaN NaN 21725 \n",
"8 NaN NaN 20875 NaN \n",
"7 NaN NaN 19650 NaN \n",
"3 NaN NaN 18400 NaN \n",
"31 NaN NaN NaN NaN \n",
"26 NaN NaN NaN NaN \n",
"22 NaN NaN NaN NaN "
]
},
"execution_count": 522,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"new_cols = df['fees_for_services_accounting'].apply(pd.Series).add_prefix('fsfa_').columns.tolist()\n",
"df[['fees_for_services_accounting'] + new_cols][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>At this point we would just have to combine the *fsfa_Total* and *fsfa_TotalAmt* columns and we'd be set. "
]
},
{
"cell_type": "code",
"execution_count": 529,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>fsfa_0</th>\n",
" <th>fsfa_Fundraising</th>\n",
" <th>fsfa_ManagementAndGeneral</th>\n",
" <th>fsfa_ManagementAndGeneralAmt</th>\n",
" <th>fsfa_ProgramServices</th>\n",
" <th>fsfa_ProgramServicesAmt</th>\n",
" <th>fsfa_Total</th>\n",
" <th>fsfa_TotalAmt</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>24600.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>23835.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23835</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23835</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>23700.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23700</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>23700</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>21725.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>21725</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>21725</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>20875.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>20875</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>20875</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>19650.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>19650</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>19650</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>18400.0</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>18400</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>18400</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" fees_for_services_accounting fsfa_0 fsfa_Fundraising \\\n",
"29 24600.0 NaN NaN \n",
"27 23835.0 NaN NaN \n",
"21 23700.0 NaN NaN \n",
"15 21725.0 NaN NaN \n",
"8 20875.0 NaN NaN \n",
"7 19650.0 NaN NaN \n",
"3 18400.0 NaN NaN \n",
"31 NaN NaN NaN \n",
"26 NaN NaN NaN \n",
"22 NaN NaN NaN \n",
"\n",
" fsfa_ManagementAndGeneral fsfa_ManagementAndGeneralAmt \\\n",
"29 NaN 24600 \n",
"27 NaN 23835 \n",
"21 NaN 23700 \n",
"15 NaN 21725 \n",
"8 20875 NaN \n",
"7 19650 NaN \n",
"3 18400 NaN \n",
"31 NaN NaN \n",
"26 NaN NaN \n",
"22 NaN NaN \n",
"\n",
" fsfa_ProgramServices fsfa_ProgramServicesAmt fsfa_Total fsfa_TotalAmt \n",
"29 NaN NaN NaN 24600 \n",
"27 NaN NaN NaN 23835 \n",
"21 NaN NaN NaN 23700 \n",
"15 NaN NaN NaN 21725 \n",
"8 NaN NaN 20875 NaN \n",
"7 NaN NaN 19650 NaN \n",
"3 NaN NaN 18400 NaN \n",
"31 NaN NaN NaN NaN \n",
"26 NaN NaN NaN NaN \n",
"22 NaN NaN NaN NaN "
]
},
"execution_count": 529,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"combine_dict(df, 'fees_for_services_accounting', 'fsfa_Total', 'fsfa_TotalAmt')\n",
"df['fees_for_services_accounting'] = df['fees_for_services_accounting'].astype('float')\n",
"df[['fees_for_services_accounting'] + new_cols][:10]"
]
},
{
"cell_type": "code",
"execution_count": 532,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>OrganizationName</th>\n",
" <th>FYE</th>\n",
" <th>fees_for_services_accounting</th>\n",
" <th>fsfa_Total</th>\n",
" <th>fsfa_TotalAmt</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>FY2016</td>\n",
" <td>24600.0</td>\n",
" <td>NaN</td>\n",
" <td>24600</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>FY2015</td>\n",
" <td>23835.0</td>\n",
" <td>NaN</td>\n",
" <td>23835</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>FY2014</td>\n",
" <td>23700.0</td>\n",
" <td>NaN</td>\n",
" <td>23700</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>FY2013</td>\n",
" <td>21725.0</td>\n",
" <td>NaN</td>\n",
" <td>21725</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>FY2012</td>\n",
" <td>20875.0</td>\n",
" <td>20875</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>FY2011</td>\n",
" <td>19650.0</td>\n",
" <td>19650</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>FY2010</td>\n",
" <td>18400.0</td>\n",
" <td>18400</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>FY2017</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>FY2016</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>FY2015</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" OrganizationName FYE \\\n",
"29 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY FY2016 \n",
"27 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY FY2015 \n",
"21 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY FY2014 \n",
"15 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY FY2013 \n",
"8 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY FY2012 \n",
"7 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY FY2011 \n",
"3 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY FY2010 \n",
"31 UNITED WAY OF EASTERN MAINE FY2017 \n",
"26 UNITED WAY OF EASTERN MAINE FY2016 \n",
"22 UNITED WAY OF EASTERN MAINE FY2015 \n",
"\n",
" fees_for_services_accounting fsfa_Total fsfa_TotalAmt \n",
"29 24600.0 NaN 24600 \n",
"27 23835.0 NaN 23835 \n",
"21 23700.0 NaN 23700 \n",
"15 21725.0 NaN 21725 \n",
"8 20875.0 20875 NaN \n",
"7 19650.0 19650 NaN \n",
"3 18400.0 18400 NaN \n",
"31 NaN NaN NaN \n",
"26 NaN NaN NaN \n",
"22 NaN NaN NaN "
]
},
"execution_count": 532,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df[['OrganizationName', 'FYE', 'fees_for_services_accounting', 'fsfa_Total', 'fsfa_TotalAmt']][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Generate the descriptive statistics for our combined variable."
]
},
{
"cell_type": "code",
"execution_count": 537,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"count 25.000000\n",
"mean 99924.520000\n",
"std 140836.720325\n",
"min 12750.000000\n",
"25% 15750.000000\n",
"50% 21000.000000\n",
"75% 183156.000000\n",
"max 436577.000000\n",
"Name: fees_for_services_accounting, dtype: float64"
]
},
"execution_count": 537,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df['fees_for_services_accounting'].describe().T"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>Run some additional summary stats."
]
},
{
"cell_type": "code",
"execution_count": 538,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# of rows where fees less than 0: 0 \n",
"\n",
"# of rows where fees equal zero: 0 \n",
"\n",
"# of rows where fees greater than zero: 25 \n",
"\n",
"# of rows missing values: 8 \n",
"\n"
]
}
],
"source": [
"print '# of rows where fees less than 0:', len(df[df['fees_for_services_accounting']<0]), '\\n'\n",
"print '# of rows where fees equal zero:', len(df[df['fees_for_services_accounting']==0]), '\\n'\n",
"print '# of rows where fees greater than zero:', len(df[df['fees_for_services_accounting']>0]), '\\n'\n",
"print '# of rows missing values:', len(df[df['fees_for_services_accounting'].isnull()]), '\\n'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Save and Output the Data\n",
"I like to save the data in PANDAS' native format &mdash; known as 'pickling'."
]
},
{
"cell_type": "code",
"execution_count": 553,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"df.to_pickle('990 e-file data for 33 filings.pkl')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>We can then save it in CSV format for importing into *R* or *SAS* or *Stata*, etc."
]
},
{
"cell_type": "code",
"execution_count": 554,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"df.to_csv('990 e-file data for 33 filings.csv', index=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br>We can also export directly into *Stata*, but outputting will break if we attempt to do all the columns (the dictionary columns are especially troublesome). The following code block shows how you can select a subset of columns and export those to a *Stata* version of the dataframe."
]
},
{
"cell_type": "code",
"execution_count": 555,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>FYE</th>\n",
" <th>EIN</th>\n",
" <th>OrganizationName</th>\n",
" <th>URL</th>\n",
" <th>SubmittedOn</th>\n",
" <th>TaxPeriod</th>\n",
" <th>FormType</th>\n",
" <th>audit_committee</th>\n",
" <th>fees_for_services_accounting</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>29</th>\n",
" <td>FY2016</td>\n",
" <td>10202467.0</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201733189349308233_public.xml</td>\n",
" <td>2018-01-05</td>\n",
" <td>201612.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>24600.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>27</th>\n",
" <td>FY2015</td>\n",
" <td>10202467.0</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201643209349300339_public.xml</td>\n",
" <td>2017-04-11</td>\n",
" <td>201512.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>23835.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>21</th>\n",
" <td>FY2014</td>\n",
" <td>10202467.0</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201533209349308003_public.xml</td>\n",
" <td>2016-02-16</td>\n",
" <td>201412.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>23700.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>FY2013</td>\n",
" <td>10202467.0</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201443169349303124_public.xml</td>\n",
" <td>2014-12-10</td>\n",
" <td>201312.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>21725.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>FY2012</td>\n",
" <td>10202467.0</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201303199349304005_public.xml</td>\n",
" <td>2013-12-31</td>\n",
" <td>201212.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>20875.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>FY2011</td>\n",
" <td>10202467.0</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201213199349302441_public.xml</td>\n",
" <td>2012-12-21</td>\n",
" <td>201112.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>19650.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>FY2010</td>\n",
" <td>10202467.0</td>\n",
" <td>MOUNT DESERT ISLAND BIOLOGICAL LABORATORY</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201123189349301402_public.xml</td>\n",
" <td>2011-12-12</td>\n",
" <td>201012.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>18400.0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>31</th>\n",
" <td>FY2017</td>\n",
" <td>10211478.0</td>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201800389349301200_public.xml</td>\n",
" <td>2018-03-02</td>\n",
" <td>201706.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>26</th>\n",
" <td>FY2016</td>\n",
" <td>10211478.0</td>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201613339349300131_public.xml</td>\n",
" <td>2017-04-17</td>\n",
" <td>201606.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>FY2015</td>\n",
" <td>10211478.0</td>\n",
" <td>UNITED WAY OF EASTERN MAINE</td>\n",
" <td>https://s3.amazonaws.com/irs-form-990/201600329349301255_public.xml</td>\n",
" <td>2016-03-29</td>\n",
" <td>201506.0</td>\n",
" <td>990.0</td>\n",
" <td>1.0</td>\n",
" <td>NaN</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" FYE EIN OrganizationName \\\n",
"29 FY2016 10202467.0 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"27 FY2015 10202467.0 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"21 FY2014 10202467.0 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"15 FY2013 10202467.0 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"8 FY2012 10202467.0 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"7 FY2011 10202467.0 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"3 FY2010 10202467.0 MOUNT DESERT ISLAND BIOLOGICAL LABORATORY \n",
"31 FY2017 10211478.0 UNITED WAY OF EASTERN MAINE \n",
"26 FY2016 10211478.0 UNITED WAY OF EASTERN MAINE \n",
"22 FY2015 10211478.0 UNITED WAY OF EASTERN MAINE \n",
"\n",
" URL \\\n",
"29 https://s3.amazonaws.com/irs-form-990/201733189349308233_public.xml \n",
"27 https://s3.amazonaws.com/irs-form-990/201643209349300339_public.xml \n",
"21 https://s3.amazonaws.com/irs-form-990/201533209349308003_public.xml \n",
"15 https://s3.amazonaws.com/irs-form-990/201443169349303124_public.xml \n",
"8 https://s3.amazonaws.com/irs-form-990/201303199349304005_public.xml \n",
"7 https://s3.amazonaws.com/irs-form-990/201213199349302441_public.xml \n",
"3 https://s3.amazonaws.com/irs-form-990/201123189349301402_public.xml \n",
"31 https://s3.amazonaws.com/irs-form-990/201800389349301200_public.xml \n",
"26 https://s3.amazonaws.com/irs-form-990/201613339349300131_public.xml \n",
"22 https://s3.amazonaws.com/irs-form-990/201600329349301255_public.xml \n",
"\n",
" SubmittedOn TaxPeriod FormType audit_committee \\\n",
"29 2018-01-05 201612.0 990.0 1.0 \n",
"27 2017-04-11 201512.0 990.0 1.0 \n",
"21 2016-02-16 201412.0 990.0 1.0 \n",
"15 2014-12-10 201312.0 990.0 1.0 \n",
"8 2013-12-31 201212.0 990.0 1.0 \n",
"7 2012-12-21 201112.0 990.0 1.0 \n",
"3 2011-12-12 201012.0 990.0 1.0 \n",
"31 2018-03-02 201706.0 990.0 1.0 \n",
"26 2017-04-17 201606.0 990.0 1.0 \n",
"22 2016-03-29 201506.0 990.0 1.0 \n",
"\n",
" fees_for_services_accounting \n",
"29 24600.0 \n",
"27 23835.0 \n",
"21 23700.0 \n",
"15 21725.0 \n",
"8 20875.0 \n",
"7 19650.0 \n",
"3 18400.0 \n",
"31 NaN \n",
"26 NaN \n",
"22 NaN "
]
},
"execution_count": 555,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cols = ['FYE', 'EIN', 'OrganizationName', 'URL', 'SubmittedOn', 'TaxPeriod', 'FormType', 'audit_committee',\n",
" 'fees_for_services_accounting']\n",
"df[cols].to_stata('XY_test.dta')\n",
"df[cols][:10]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<br><br>We have gone through a lot in this tutorial: 1) importing downloaded 990 data into PANDAS, 2) loading the codebook, 3) using the codebook to identify relevant variables, 4) wrangling the data , and 5) export the transformed data in *Stata* and CSV formats. As you have seen, the e-file data need *a lot* of wrangling to get in a useable format. I am hoping that this tutorial has provided a few good tools for you in your research efforts. If you have found this helpful, please share it with others. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python [default]",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.14"
},
"toc": {
"colors": {
"hover_highlight": "#DAA520",
"navigate_num": "#000000",
"navigate_text": "#333333",
"running_highlight": "#FF0000",
"selected_highlight": "#FFD700",
"sidebar_border": "#EEEEEE",
"wrapper_background": "#FFFFFF"
},
"moveMenuLeft": true,
"nav_menu": {
"height": "243px",
"width": "252px"
},
"navigate_menu": true,
"number_sections": true,
"sideBar": true,
"threshold": 4,
"toc_cell": false,
"toc_section_display": "block",
"toc_window_display": false,
"widenNotebook": false
}
},
"nbformat": 4,
"nbformat_minor": 1
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment