There are many ways to work with GIS data. I've used QGIS (or cartodb and mapbox online) when I need to explore shapefiles. When building a map or application built on a map, I like to use command line tools. The examples below use the GDAL command line tools as a lot of GIS libraries use the GDAL libraries under the hood. If you're somewhat familiar with SQL, the tools can be a quick way to explore the contents of shapefiles in a scriptable way.
For the examples below I used shapefiles that I downloaded directly from the US census: North Carolina Census Tracts
This shapefile contains polygon regions for all census tracts in NC. For the purposes of clustering we may only want the centroid of each tract. To create a CSV of all tracts in the shapefile, only a couple commands are required: