Skip to content

Instantly share code, notes, and snippets.

@wahidsadik
Created May 7, 2019 18:43
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save wahidsadik/147d0390bd4953f2259ce774e26ee9e1 to your computer and use it in GitHub Desktop.
Save wahidsadik/147d0390bd4953f2259ce774e26ee9e1 to your computer and use it in GitHub Desktop.

Motivation

As part of my study of Analytics, I realize that SEO and SEM are relevant topics. This blog captures some of the learning.

Notes

What I learned from here:

  • Organic Traffic: When search engines route traffic to your website without you spending any money.
  • Paid Traffic: When search engines route traffic to your website when you spend some money to do so.
  • Search Engine Optimization/SEO: Describes a set of practices to apply on your website for organic traffic.
  • Search Engine Marketing/SEM or Pay Per Click/PPC: Describes a set of practices to apply on your website for paid traffic.
  • Search Engine Result List/SERL: Self explanatory.
  • /robot.txt file: Search engine bots look for this file on every website. This file holds instruction on which pages to list with the search engine.

Robot Exclusion Protocol / REP

Discussed here at length.

REP defines a set of rules for your website to help the crawlers to figure which pages to show/not show.

  • Crawler Directive: Through /robot.txt (most common) and also /sitemap.xml, you can define which paths to include/exclude and which user agents (read bots) to allow/disallow, etc.
    • Absence of these files implies implicit approval.
    • Read more about robot.txt here, here
    • Read more about sitemap.xml here.
  • REL tags: You can use certain HTML attribute in tags to tell crawlers to not include a page. As mentioned here, just add rel='nofollow' on anchors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment