WIkimedia Hackathon - Bits Pilani Hyderabad Campus

00 - Intro to Jupyter.ipynb

01 - Python Tutorial.ipynb

02 - Intro to pywikibot.ipynb

03 - Wikidata.ipynb

04 - SparQL as a Wikidata pywikibot generator.ipynb

05 - Getting census data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Census data in India"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The census data in India is available at <http://www.censusindia.gov.in/>, the official website for the India census."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Get census data\n",
    "\n",
    "Here are the steps to get some data for a speicifc district:\n",
    " - Open <http://www.censusindia.gov.in/>.\n",
    " - On this website go to **Census Digital Library** on the left sidebar.\n",
    " - In the DIgital Library, click on the **Table** bar on the top bar.\n",
    " - Choose the year of the census you want the data from in the dropdown. On choosing the year, the page refreshes to give more types of data available.\n",
    " - Search for **Primary Census Abstract** which is the datasource we want, as it has the census information in a concise manner.\n",
    " - Under the Primary Census Abstract option, choose **Primary Census Abstract Total** as it gives the data for demographies.\n",
    " - Another dropdown is now seen for the State. Choose the state for which you need the data.\n",
    "  - If \"India\" is chosen, it gives the data for India and all the states.\n",
    "  - If a state is chosen, it gives the data for the state and all districts in the state.\n",
    "   - Once a state is chosen, in the next page you will be able to filter by distict too. In the district level sheets, you can find population data for Taluk and village also.\n",
    " - In the data downloaded, you can find multiple columns which give different data. TOT_P is the one which has \"total number of people\" in that area. THe meaning of the other columns can be found at "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Read data into python\n",
    "\n",
    "Once the data has been downloaded, save it as a CSV file and load the data into a python list using:\n",
    "\n",
    " - The **`csv`** module \n",
    " - Using **`numpy`** into a numpy array\n",
    " - Using **`pandas`** into a Data Frame\n",
    " - Or simply read the file as a text file and read it line by line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Finding the WIkidata page for that area\n",
    "\n",
    "Now we need to match each row in our data to a Wikidata page which needs the population data.\n",
    "\n",
    "To ensure that the page is correct, some possible options are:\n",
    " - Find the Name of the area and match it to the english label.\n",
    " - Ensure that the [instance of](https://www.wikidata.org/wiki/Property:P31) is on the the follwing: [Town](https://www.wikidata.org/wiki/Q3957) (for Taluk or Town), [district of india](<https://www.wikidata.org/wiki/Q1149652>), or [state of india](https://www.wikidata.org/wiki/Q13390680) and so on.\n",
    " - Check that the state is mentioned somwehere in the data.\n",
    " - Check that the [country](https://www.wikidata.org/wiki/Property:P17) is set to [India](https://www.wikidata.org/wiki/Q668).\n",
    " - Check the English labels and aliases, and also other language labels if needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Adding the data to Wikidata\n",
    "\n",
    "First, read the ItemPage and check that the property already exists.\n",
    " - If it does exist, and the value is different add it with a different \"Source\" value.\n",
    " - It if does exist, and the value is same, add a new source to the same (property, value) pair.\n",
    " - If it does not exist, create a new (property, value) pair."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}