Google Indexing API V3 Working example with Python 3

GoogleIndexing.py

from oauth2client.service_account import ServiceAccountCredentials
import httplib2
import json

import pandas as pd

# https://developers.google.com/search/apis/indexing-api/v3/prereqs#header_2
JSON_KEY_FILE = "json_key_file_downloaded_after_creating_your_google_service_account_see_above_details_on_how_to_do.json"
SCOPES = ["https://www.googleapis.com/auth/indexing"]

credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_KEY_FILE, scopes=SCOPES)
http = credentials.authorize(httplib2.Http())

def indexURL(urls, http):
    # print(type(url)); print("URL: {}".format(url));return;

    ENDPOINT = "https://indexing.googleapis.com/v3/urlNotifications:publish"
    
    for u in urls:
        # print("U: {} type: {}".format(u, type(u)))
    
        content = {}
        content['url'] = u.strip()
        content['type'] = "URL_UPDATED"
        json_ctn = json.dumps(content)    
        # print(json_ctn);return
    
        response, content = http.request(ENDPOINT, method="POST", body=json_ctn)

        result = json.loads(content.decode())

        # For debug purpose only
        if("error" in result):
            print("Error({} - {}): {}".format(result["error"]["code"], result["error"]["status"], result["error"]["message"]))
        else:
            print("urlNotificationMetadata.url: {}".format(result["urlNotificationMetadata"]["url"]))
            print("urlNotificationMetadata.latestUpdate.url: {}".format(result["urlNotificationMetadata"]["latestUpdate"]["url"]))
            print("urlNotificationMetadata.latestUpdate.type: {}".format(result["urlNotificationMetadata"]["latestUpdate"]["type"]))
            print("urlNotificationMetadata.latestUpdate.notifyTime: {}".format(result["urlNotificationMetadata"]["latestUpdate"]["notifyTime"]))

"""
data.csv has 2 columns: URL and date.
I just need the URL column.
"""
csv = pd.read_csv("my_data.csv")
csv[["URL"]].apply(lambda x: indexURL(x, http))

• Where do I put my URLs in the code?
  You put your URL in a csv file.

• Can I run this code in Google Colab?
  I did not test it but yes, I think you can.

Remember to read the comment in my code to get a better understanding of it.

If you are think about copy-paste it into your Colab and run it, it would not work.

Do you have any youtube video tutorial for the whole process? Or any blog post tutorial?

Thank you very much.

Hello, Many Thanks for contribute.
After running this for a URL how would you check if it got indexed ?
I know of only two ways, site: function and google search. Someone did mentioned it in thread above and you mentioned
"It seems that Google has disabled "index site" function late last year. At present you cannot index it in Google Search Console (GSC)"
Can you please explain this statement ?

@modestt

1. Yes, one way is to use site: you mentioned. And another way is actually using GSC to see if your page get indexed.
2. It is exactly what I mean: Some history: Several years ago, Google released a function that you can submit your URL for them to index directly from their search result page, and later on, probably due to abusive use of it, this function is disabled and you can only index your site in GSC.

However, about a year (or more?) ago, you can't even index your site in GSC, they remove this function in your GSC panel. Which means even if you are verified as the owner of the website, you cannot submit a URL to Google.

So up to this writing, you can only create and submit a sitemap.xml in GSC to let Google know that you have a list of page in your website. Then Google will work their magic to index your site.

There is no progress, no report, no timeline, all up to Google to see if they want to index your site or not.

Hope it helps.

tested and worked, thanks

I can't do this for my website time to move

Thank you @alucard001 it seems to work well.

I do have a question, at the end I receive as part of the output these lines:
URL None
dtype: object

Is that ok?

Appreciate your help

FREE IPTV LINKS

DAILY M3U PLAYLISTS

WORLD TV CHANNELS

Thank you so much!

More of my websites:

http://happinesspig.com, http://premiumpig.com, http://3apig.com, http://wow.esdlife.com

How to determine the status of indexing is completed or not ?

Hi Dear,
I did all the steps you mentioned in your youtube video. [(https://www.youtube.com/watch?v=Wwo8YbUHlj8)]
But Still, It's not working for me. I'm Getting "Error(403 - PERMISSION_DENIED): Permission denied. Failed to verify the URL ownership"

Thanks. :-)

@ZAKhan123 Please make sure that you own those URLs.
By "own" I mean you can verify your URL ownership in Google Search Console.

If you did not "own" your URL, you cannot index it.

Hope it helps.

I have submitted lots of URLs for our website https://allpng.in/

But after checking on google search console, I got the message 'URL is not on Google'
Like this

Hi Amit,
I followed all the steps, but I found this error and even tried hard to resolve it myself. i am using Python 3.11.1 version.
Traceback (most recent call last):
File "D:\Tutorials\indexing.py", line 11, in
credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_KEY_FILE, scopes=SCOPES)
FileNotFoundError: [Errno 2] No such file or directory: 'apidetails'

any help will be highly appreciated.
Thank you

Do you know how to improve the optimization of google mobile friendly api?

…

On Mon, Dec 12, 2022 at 4:19 AM Nabeel Ahmed Khan ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Hi Amit, I followed all the steps, but I found this error and even tried hard to resolve it myself. i am using Python 3.11.1 version. Traceback (most recent call last): File "D:\Tutorials\indexing.py", line 11, in credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_KEY_FILE, scopes=SCOPES) FileNotFoundError: [Errno 2] No such file or directory: 'apidetails' any help will be highly appreciated. Thank you — Reply to this email directly, view it on GitHub <https://gist.github.com/fedce21b80a3de7aa3c07299838e1462#gistcomment-4399113> or unsubscribe <https://github.com/notifications/unsubscribe-auth/A27TIY5UZU2GA7GG4DYPBDDWMYZLHBFKMF2HI4TJMJ2XIZLTSKBKK5TBNR2WLJDHNFZXJJDOMFWWLK3UNBZGKYLEL52HS4DFQKSXMYLMOVS2I5DSOVS2I3TBNVS3W5DIOJSWCZC7OBQXE5DJMNUXAYLOORPWCY3UNF3GS5DZVRZXKYTKMVRXIX3UPFYGLK2HNFZXIQ3PNVWWK3TUUZ2G64DJMNZZDAVEOR4XAZNEM5UXG5FFOZQWY5LFVA4TMNBRGI3DSNFHORZGSZ3HMVZKMY3SMVQXIZI> . You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub> .

-- Florussel Gallibot | Project Development Officer II *-- HRMDD --* a: DSWD CARAGA REGION | Rpalma Street | 8600 e: ***@***.*** & *jrgallibot@ ***@***.***>gmail.com <http://gmail.com>* w: www.caraga-portal.dswd.gov.ph <http://www.caraga.dswd.gov.ph/> / www.caraga-oas.dswd.gov.ph <http://www.caraga.dswd.gov.ph/> p: +639124811686 & +639276756872 ----------------------------------------------------------------------------------------------- [image: https://osm.dswd.gov.ph/] <https://osm.dswd.gov.ph/>

Hello thankyou for this code. It worked well for 200 urls but showed the above error for the rest. I had 500 urls in the the csv. Can you please help me with this

@confusedkuala The error message already told you: there is a quota for indexing URL. So please do the indexing next day.

Hi Amit, I followed all the steps, but I found this error and even tried hard to resolve it myself. i am using Python 3.11.1 version. Traceback (most recent call last): File "D:\Tutorials\indexing.py", line 11, in credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_KEY_FILE, scopes=SCOPES) FileNotFoundError: [Errno 2] No such file or directory: 'apidetails'

any help will be highly appreciated. Thank you

It said it cannot find your JSON_KEY_FILE, please make sure you are using the correct path.

I have submitted lots of URLs for our website https://allpng.in/

But after checking on google search console, I got the message 'URL is not on Google' Like this

Have you checked that URL today? When you are using this tool, it is not IMMEDIATELY get your URL index right away. It is up to Google to decide when to add your URL to their index.

So you can think of this tool as a "Reminder" to notify Google to add your URL to their index.

@confusedkuala The error message already told you: there is a quota for indexing URL. So please do the indexing next day.

then how do i submit 1000 urls in one query. Help me with this

@confusedkuala The error message already told you: there is a quota for indexing URL. So please do the indexing next day.

then how do i submit 1000 urls in one query. Help me with this

You don't. You split it into 5 days, or you can check if there are ways to increase quota through your Google cloud console.

@confusedkuala The error message already told you: there is a quota for indexing URL. So please do the indexing next day.

then how do i submit 1000 urls in one query. Help me with this

You don't. You split it into 5 days, or you can check if there are ways to increase quota through your Google cloud console.

okay thankyou so much

I am getting this error quite a few times. I checked the url also everything seems right.

can you please help me with this error

Ola ajuda aqui o meu código deu esse erro

Read Comics Online

@alucard001
Please HELP! Why it is not working, wht is thz issue?

@weirdbozer found a latest version of this https://pypi.org/project/gsc-bulk-indexer/ you can try it out.

I want to remove 404 urls with the help of this code,
plz help how can i do that.

On which line number will the code be edited?