Skip to content

Instantly share code, notes, and snippets.

View Subrahmanyajoshi's full-sized avatar

Subrahmanya Joshi Subrahmanyajoshi

View GitHub Profile
@Subrahmanyajoshi
Subrahmanyajoshi / yt_comments_producer.ipynb
Created March 11, 2023 15:00
Notebook to scrape YouTube comments related to a particular topic and send it to a Pub/Sub topic
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Subrahmanyajoshi
Subrahmanyajoshi / pipeline.py
Created March 11, 2023 12:52
Hate Speech Detection pipeline
from argparse import Namespace
from datetime import datetime
import apache_beam as beam
from apache_beam import io
from google.cloud import bigquery
from pipeline.components import PipelineComponents, ResultsFilter
@Subrahmanyajoshi
Subrahmanyajoshi / components.py
Last active March 11, 2023 12:52
Components of dataflow pipeline for hate speech detection in youtube comments
import json
import re
from typing import Dict
import apache_beam as beam
import numpy as np
from apache_beam import pvalue
from apache_beam.ml.gcp import naturallanguageml as nlp
from google.cloud import language
@Subrahmanyajoshi
Subrahmanyajoshi / nlp_text_classification.ipynb
Created February 12, 2022 06:44
Classify text into different known categories using Google Cloud Natural Language API
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Subrahmanyajoshi
Subrahmanyajoshi / nlp_syntax_analysis.ipynb
Last active February 13, 2022 06:15
Syntax analysis using Google Cloud Natural Language API
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Subrahmanyajoshi
Subrahmanyajoshi / nlp_sentiment_analysis.ipynb
Created February 12, 2022 06:27
Sentiment Analysis using Google Cloud Natural Language API
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Subrahmanyajoshi
Subrahmanyajoshi / nlp_entity_analysis.ipynb
Created February 12, 2022 06:16
Entity analysis using Google Cloud Natural Language API
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Subrahmanyajoshi
Subrahmanyajoshi / translation_api_lang_detection.ipynb
Created February 12, 2022 05:43
Detect language of a text using Google Cloud Translation API
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Subrahmanyajoshi
Subrahmanyajoshi / translation_api.ipynb
Created February 12, 2022 05:38
Translate text into hundreds of languages using Google Cloud Translation API
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Subrahmanyajoshi
Subrahmanyajoshi / video_intel_speech_transcription.ipynb
Created February 12, 2022 04:52
Transcribe speech in videos using Google Cloud Video Intelligence API
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.