While Raspbian is my go-to distro for Raspberry Pi, the image includes a huge number of packages that I don't use.
So, to slim-down my Raspbian image, I run the following:
apt-get purge --auto-remove ^xserver ^x11 ^libx11 libxau6 libxdmcp6 ^libxcb penguinspuzzle hicolor-icon-theme menu-xdg ^epiphany ^gstreamer ^gvfs xdg-utils ^zenity ^gnome ^gconf ^libgnome ^gsettings desktop-file-utils wolfram-engine
apt-get autoremove --purge
This will remove the X11 stuff, GNOME, Wolfram, and a handful of other things. The number of packages installed dropped from ~900 to less than ~400. My installation goes from ~2.5gig down to 1.2gig.