This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
swagger: '2.0' | |
info: | |
title: Read News Articles | |
description: This API returns news articles related to an entity | |
version: 1.0.0 | |
host: [GENERATED SERVICE HOSTNAME] | |
schemes: | |
- https | |
paths: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
from datetime import datetime, timezone | |
import time | |
import hashlib | |
import json | |
import pandas as pd | |
from flask import escape | |
from flask import jsonify |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
swagger: '2.0' | |
info: | |
title: RSS Search Process | |
description: This API triggers an RSS search process | |
version: 1.0.0 | |
host: [GENERATED SERVICE HOSTNAME] | |
schemes: | |
- https | |
paths: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
class gcp_util: | |
def get_projectid(self,projectid=None): | |
if projectid is None: | |
env_var=['GCP_PROJECT','GOOGLE_CLOUD_PROJECT'] | |
for v in env_var: | |
try: | |
my_projectid=os.environ.get(v) | |
print('OK') |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Public packages | |
from datetime import datetime, timezone | |
import pandas as pd | |
from flask import escape | |
from flask import jsonify | |
#Private dependency package | |
from mylib.gcp_cloud_util.gcp_cloud_util import gcp_pubsub |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from datetime import datetime, timezone | |
import pandas as pd | |
from pandas.io.json import json_normalize | |
import requests | |
import feedparser | |
from bs4 import BeautifulSoup | |
from google.cloud import storage | |
from google.cloud import pubsub_v1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
from bs4 import BeautifulSoup | |
import pandas as pd | |
#Test url | |
url = 'https://www.theguardian.com/technology/2019/aug/28/apple-ends-contracts-hundreds-workers-hired-to-listen-siri' | |
#Request the article url to get the web page content. | |
article = requests.get(url) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import feedparser | |
from pandas.io.json import json_normalize | |
import pandas as pd | |
import requests | |
rss_url='https://news.google.com/rss/search?q=apple' | |
#Read feed xml data | |
news_feed = feedparser.parse(rss_url) | |
#Flatten data |