Skip to content

Instantly share code, notes, and snippets.

@lefnire
Created June 11, 2018 01:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save lefnire/e3affb0222a7bc6c5af775bac6208d38 to your computer and use it in GitHub Desktop.
Save lefnire/e3affb0222a7bc6c5af775bac6208d38 to your computer and use it in GitHub Desktop.
df = pd.read_csv(
data_path('Model_Update_Data_April_2018.txt'),
sep='\t',
true_values=['Y', 'True', 'TRUE'],
false_values=['N', 'False', 'FALSE'],
na_values=['?', 'U', 'Unknown']
)
na_cols = df.columns[df.isna().any()]
print(na_cols)
"""
policy_inception_date
policy_expiration_date
policy_longevity_days
days_until_policy_expiration
bi_limits
fk_bus_class_c
yrs_prior_carrier_n
prior_ins_ind
down_payment_amount
down_payment_percentages
auto_pay_i
bodyshopselected_any_ind
airbags_depolyed_any_ind
vehtowedind_any_ind
vehicle_inoperable_any_ind
salvaged_veh_any_ind
max_annualmileage
max_annualmileagerangenumber
min_annualmileage
min_annualmileagerangenumber
max_dailymiles
max_odometer
max_odometer_range_n
min_odometer
min_odometer_range_n
max_symbol_code
leased_veh_any_ind
min_ceh_purchased_date
max_veh_purchased_date
verified_mileage_discount_c_any
inspection_ratings_aggregate
number_of_claims_since_inception
avg_fault_percentage
percentage_no_fault
total_number_of_prior_claims
total_number_of_previous_policies
sum_accpts
sum_cvnpts
sum_pts
youthful_contact_ind
driver_gender_blender
min_driver_age
max_driver_age
income_level_max
total_annual_premium
multipolicy_type
membership_type_decoded
named_insured_age_range
named_insured_marital_status
total_loss_avoidance_amount
"""
num_cols = df.select_dtypes(include=[np.number]).columns
intersect = set(na_cols) & set(num_cols)
print(intersect)
"""
max_dailymiles
total_loss_avoidance_amount
min_driver_age
max_annualmileagerangenumber
avg_fault_percentage
min_annualmileagerangenumber
total_number_of_prior_claims
down_payment_percentages
policy_longevity_days
sum_accpts
youthful_contact_ind
driver_gender_blender
percentage_no_fault
days_until_policy_expiration
min_odometer_range_n
min_annualmileage
number_of_claims_since_inception
total_number_of_previous_policies
max_odometer_range_n
max_driver_age
down_payment_amount
max_odometer
min_odometer
sum_cvnpts
total_annual_premium
yrs_prior_carrier_n
sum_pts
max_annualmileage
"""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment