Working with Google Custom Search Engine using Python

gcse.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Google Custom Search API\n",
    "\n",
    "https://developers.google.com/custom-search/\n",
    "\n",
    "https://developers.google.com/custom-search/v1/overview\n",
    "\n",
    "![](https://cdn-images-1.medium.com/max/1600/1*GljTlf_8C-3eZwwncRmlEw.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Create a project on Google Developers Console\n",
    "\n",
    "https://console.developers.google.com\n",
    "\n",
    "\n",
    "## 2. Enable Custom Search API and generate API key\n",
    "\n",
    "\n",
    "## 3. Create a search engine\n",
    "\n",
    "https://cse.google.com/cse/all\n",
    "\n",
    "\n",
    "## 4. Installation\n",
    "\n",
    "```\n",
    "pip install google-api-python-client\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "api_key = \"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from apiclient.discovery import build"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "resource = build(\"customsearch\", 'v1', developerKey=api_key).cse()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = resource.list(q='python', cx='009557628044748784875:5lejfe73wrw').execute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'kind': 'customsearch#result',\n",
       " 'title': 'Internal working of Python - GeeksforGeeks',\n",
       " 'htmlTitle': 'Internal working of <b>Python</b> - GeeksforGeeks',\n",
       " 'link': 'https://cdncontribute.geeksforgeeks.org/wp-content/uploads/python_working.png',\n",
       " 'displayLink': 'www.geeksforgeeks.org',\n",
       " 'snippet': 'Internal working of Python - GeeksforGeeks',\n",
       " 'htmlSnippet': 'Internal working of <b>Python</b> - GeeksforGeeks',\n",
       " 'mime': 'image/png',\n",
       " 'image': {'contextLink': 'https://www.geeksforgeeks.org/internal-working-of-python/',\n",
       "  'height': 371,\n",
       "  'width': 846,\n",
       "  'byteSize': 12937,\n",
       "  'thumbnailLink': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTMR1g0292SKbY-iXUA6ijVFrKgp4yvSiBkOH3j8B5faYCBAjdIkCOL3RVk',\n",
       "  'thumbnailHeight': 64,\n",
       "  'thumbnailWidth': 145}}"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result['items'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(result['items'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python Programming Language - GeeksforGeeks https://www.geeksforgeeks.org/python-programming-language/\n",
      "Python Language Introduction - GeeksforGeeks https://www.geeksforgeeks.org/python-language-introduction/\n",
      "Taking multiple inputs from user in Python - GeeksforGeeks https://www.geeksforgeeks.org/taking-multiple-inputs-from-user-in-python/\n",
      "Taking input in Python - GeeksforGeeks https://www.geeksforgeeks.org/taking-input-in-python/\n",
      "Python program to add two numbers - GeeksforGeeks https://www.geeksforgeeks.org/python-program-to-add-two-numbers/\n",
      "Statement, Indentation and Comment in Python - GeeksforGeeks https://www.geeksforgeeks.org/statement-indentation-and-comment-in-python/\n",
      "Python Programming Examples - GeeksforGeeks https://www.geeksforgeeks.org/python-programming-examples/\n",
      "Polymorphism in Python - GeeksforGeeks https://www.geeksforgeeks.org/polymorphism-in-python/\n",
      "Namespaces and Scope in Python - GeeksforGeeks https://www.geeksforgeeks.org/namespaces-and-scope-in-python/\n",
      "Python 3 basics - GeeksforGeeks https://www.geeksforgeeks.org/python-3-basics/\n"
     ]
    }
   ],
   "source": [
    "for item in result['items']:\n",
    "    print(item['title'], item['link'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Image search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = resource.list(q='python', cx='009557628044748784875:5lejfe73wrw', searchType='image').execute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Suresh Kumar - Quora https://qph.fs.quoracdn.net/main-raw-350993202-zvusdjmacabgcydlxszxcwhhtjrarrdk.jpeg\n",
      "What does the Python logo stand for? - Quora https://qph.fs.quoracdn.net/main-qimg-28cadbd02699c25a88e5c78d73c7babc\n",
      "What is Python primarily used for? - Quora https://qph.fs.quoracdn.net/main-qimg-ff06c339780f23112c1cbbbb284158ff\n",
      "Python Django | Google authentication and Fetching mails from ... https://cdncontribute.geeksforgeeks.org/wp-content/uploads/django-google.png\n",
      "Implementation of Dynamic Array in Python - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/daynamic-array.png\n",
      "Can we use the Python programming language on Arduino? - Quora https://qph.fs.quoracdn.net/main-qimg-3cf229eedc092549277e8859aad2fca5\n",
      "Any & All in Python - GeeksforGeeks https://www.geeksforgeeks.org/wp-content/uploads/All-any-in-python.png\n",
      "Short Circuiting Techniques in Python - GeeksforGeeks http://cdncontribute.geeksforgeeks.org/wp-content/uploads/bool.png\n",
      "Where is Python mostly used? - Quora https://qph.fs.quoracdn.net/main-qimg-0613adfdf47a4691e947539c31be0247\n",
      "What are the pros and cons of Python? - Quora https://qph.fs.quoracdn.net/main-qimg-0b31556e89087f379dac3d482675987a\n"
     ]
    }
   ],
   "source": [
    "for item in result['items']:\n",
    "    print(item['title'], item['link'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pagination"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "images = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(1, 60, 10):\n",
    "    result = resource.list(q='python', cx='009557628044748784875:5lejfe73wrw',\n",
    "                      searchType='image', start=i).execute()\n",
    "    images += result['items']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "60"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Suresh Kumar - Quora https://qph.fs.quoracdn.net/main-raw-350993202-zvusdjmacabgcydlxszxcwhhtjrarrdk.jpeg\n",
      "What does the Python logo stand for? - Quora https://qph.fs.quoracdn.net/main-qimg-28cadbd02699c25a88e5c78d73c7babc\n",
      "What is Python primarily used for? - Quora https://qph.fs.quoracdn.net/main-qimg-ff06c339780f23112c1cbbbb284158ff\n",
      "Python Django | Google authentication and Fetching mails from ... https://cdncontribute.geeksforgeeks.org/wp-content/uploads/django-google.png\n",
      "Implementation of Dynamic Array in Python - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/daynamic-array.png\n",
      "Can we use the Python programming language on Arduino? - Quora https://qph.fs.quoracdn.net/main-qimg-3cf229eedc092549277e8859aad2fca5\n",
      "Any & All in Python - GeeksforGeeks https://www.geeksforgeeks.org/wp-content/uploads/All-any-in-python.png\n",
      "Short Circuiting Techniques in Python - GeeksforGeeks http://cdncontribute.geeksforgeeks.org/wp-content/uploads/bool.png\n",
      "Where is Python mostly used? - Quora https://qph.fs.quoracdn.net/main-qimg-0613adfdf47a4691e947539c31be0247\n",
      "What are the pros and cons of Python? - Quora https://qph.fs.quoracdn.net/main-qimg-0b31556e89087f379dac3d482675987a\n",
      "Can I make an Android app with Python? - Quora https://qph.fs.quoracdn.net/main-qimg-522be9c947f2ddec8713e8211a85e3a6\n",
      "Python 3 basics - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/Python_console.png\n",
      "How to learn Python for scripting - Quora https://qph.fs.quoracdn.net/main-qimg-0044b853c6516c030c17a17f4ee85153\n",
      "What are the advantages of using Python over other languages like ... https://qph.fs.quoracdn.net/main-qimg-7e9eb6a83fb8110e14b9cc06cd651f5b\n",
      "What is the best book to learn Python for data science? - Quora https://qph.fs.quoracdn.net/main-qimg-ccc07bd0e779be4c3fc9a3226f8f8409\n",
      "Create and Access a Python Package - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/Python-Packages.jpg\n",
      "Can you suggest me some good Python training institutes Bengaluru ... https://qph.fs.quoracdn.net/main-qimg-0821b15bd96a296f6dd1ee62341bdf53\n",
      "How good is it to learn Python with Django? - Quora https://qph.fs.quoracdn.net/main-qimg-7b0167d398e33e6075f975bf11d80e14\n",
      "Is it possible to learn Java AND Python in two months by myself ... https://qph.fs.quoracdn.net/main-qimg-e849c739b4c7a3ff50a892c7ab3da974\n",
      "Python | Print list after removing element at given index ... https://cdncontribute.geeksforgeeks.org/wp-content/uploads/WhatsApp-Image-2017-12-24-at-12.09.55-AM.jpeg\n",
      "Python | Find missing and additional values in two lists ... https://cdncontribute.geeksforgeeks.org/wp-content/uploads/WhatsApp-Image-2018-01-03-at-12.46.34-PM.jpeg\n",
      "Logging in Python - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/Python-log-levels.png\n",
      "Multithreading in Python | Set 1 - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/multithreading-python-11.png\n",
      "Why should I learn Python over other programming languages? - Quora https://qph.fs.quoracdn.net/main-qimg-05c5032f8e16855f0915e1b4621f0dbc\n",
      "Are pythons poisonous? - Quora https://qph.fs.quoracdn.net/main-qimg-1abbd0df1bce918455f5c83007a0ea81\n",
      "Python Desktop News Notifier in 20 lines - GeeksforGeeks http://cdncontribute.geeksforgeeks.org/wp-content/uploads/Screenshot-from-2017-04-24-10_00_06.png\n",
      "Are pythons poisonous? - Quora https://qph.fs.quoracdn.net/main-qimg-ae429dc47e62c045bf799a267ad7d594-c\n",
      "Simple Chat Room using Python - GeeksforGeeks http://cdncontribute.geeksforgeeks.org/wp-content/uploads/Screenshot-from-2017-04-18-18-31-03.png\n",
      "Taking multiple inputs from user in Python - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/Capture8-1.png\n",
      "Implementing Web Scraping in Python with BeautifulSoup - GeeksforGeeks https://indianpythonista.files.wordpress.com/2016/11/11.png\n",
      "What can Python do? - Quora https://qph.fs.quoracdn.net/main-qimg-48370fbb8d060920a2c3e20edfd37385\n",
      "What are the 10 best features of Python? - Quora https://qph.fs.quoracdn.net/main-qimg-8b1812786f86b6028e6e139dfeda4e7d\n",
      "Python | Pandas Series - GeeksforGeeks https://media.geeksforgeeks.org/wp-content/uploads/dataSER-1.png\n",
      "What are the pros and cons of learning Python as a first ... https://qph.fs.quoracdn.net/main-qimg-6972f0e767994131bd6b363a7730f2b8\n",
      "Twitter Sentiment Analysis using Python - GeeksforGeeks https://www.geeksforgeeks.org/wp-content/uploads/Twitter-Sentiment-Analysis-using-Python.jpg\n",
      "Coroutine in Python - GeeksforGeeks http://cdncontribute.geeksforgeeks.org/wp-content/uploads/subroutine.png\n",
      "Union() function in Python - GeeksforGeeks https://www.geeksforgeeks.org/wp-content/uploads/Union-in-python.jpg\n",
      "Keywords in Python | Set 1 - GeeksforGeeks https://media.geeksforgeeks.org/wp-content/uploads/and1.png\n",
      "What is the scope of Python in India? - Quora https://qph.fs.quoracdn.net/main-qimg-0ec877591c43dcc862ecd91d9ffefcd8\n",
      "Python | Iterate over multiple lists simultaneously - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/WhatsApp-Image-2017-12-30-at-2.28.37-AM-1.jpeg\n",
      "Python | Pandas Dataframe/Series.head() method - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/nba-1-1.png\n",
      "For building a website, which programming language is easier to ... https://qph.fs.quoracdn.net/main-qimg-b18fd8d57290e40a045f114f231c7fcc\n",
      "Namespaces and Scope in Python - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/types_namespace-1.png\n",
      "What are the best free Python IDEs on PC for a beginner? - Quora https://qph.fs.quoracdn.net/main-qimg-a216add584bb8638388db9a9006b9fd4\n",
      "Python | Check if all the values in a list that are greater than a ... https://cdncontribute.geeksforgeeks.org/wp-content/uploads/WhatsApp-Image-2017-12-23-at-6.50.28-PM-232x300.jpeg\n",
      "Intersection() function Python - GeeksforGeeks https://www.geeksforgeeks.org/wp-content/uploads/intersection-in-python.jpg\n",
      "Junk File Organizer in Python - GeeksforGeeks http://cdncontribute.geeksforgeeks.org/wp-content/uploads/1112.png\n",
      "os.walk() in Python - GeeksforGeeks https://contribute.geeksforgeeks.org/wp-content/uploads/osWalk-1.jpg\n",
      "Python | Django Admin Interface - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/Screenshot-534-1.png\n",
      "Multi-Messenger : A python project, messaging via Terminal ... https://www.geeksforgeeks.org/wp-content/uploads/multi_messenger_python1.png\n",
      "Internal working of Python - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/python_working.png\n",
      "Python 3 basics - GeeksforGeeks http://cdncontribute.geeksforgeeks.org/wp-content/uploads/Screenshot-from-2017-07-20-18-47-54.png\n",
      "Python | Numbers in a list within a given range - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/WhatsApp-Image-2017-12-23-at-11.01.00-PM-300x200.jpeg\n",
      "Multithreading in Python | Set 1 - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/multithreading-python-21.png\n",
      "What qualifications are required to learn Python? - Quora https://qph.fs.quoracdn.net/main-qimg-c46d66564ba2dfcf09095a8130092803\n",
      "What is negative index in Python? - Quora https://qph.fs.quoracdn.net/main-qimg-a380b1bc159589df5e0b9842e5b56b6d\n",
      "Python range() function - GeeksforGeeks https://cdncontribute.geeksforgeeks.org/wp-content/uploads/PythonRange6.png\n",
      "Are pythons poisonous? - Quora https://qph.fs.quoracdn.net/main-qimg-7531b169489c7d442b5a6cc0c3aa68e6\n",
      "What are the features of Python? - Quora https://qph.fs.quoracdn.net/main-qimg-ed18fef07c0cd9ebebb74daf147a6275\n",
      "Python Slicing | Reverse an array in groups of given size ... https://cdncontribute.geeksforgeeks.org/wp-content/uploads/geeks-4.png\n"
     ]
    }
   ],
   "source": [
    "for item in images:\n",
    "    print(item['title'], item['link'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}