This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from selenium import webdriver | |
from selenium.webdriver.chrome.options import Options | |
import pandas as pd | |
import smtplib | |
from time import sleep |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def send_email(subject, msg): | |
try: | |
mail_from = 'your email as string' | |
password = 'Your password as string' | |
mail_to = 'The email address that will receive the email as string' | |
server = smtplib.SMTP('smtp.gmail.com:587') | |
server.ehlo() | |
server.starttls() | |
server.login(mail_from, password) | |
message = f'Subject: {subject}\n\n{msg}' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
while True: | |
my_url = 'https://community.dataquest.io/c/qa/44' | |
option = Options() | |
option.headless = True | |
driver = webdriver.Chrome(options=option) | |
driver.get(my_url) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tables = pd.read_html(driver.page_source) | |
table = tables[0] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
table = table.iloc[1:10, [0, 2, 5]] | |
table['time'] = table['Activity'].str[-1] | |
table['Activity'] = table['Activity'].str[:-1].astype(int) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
new_topics = table[(table['time'] == 'm') & (table['Activity'] <= 10) & (table['Replies'] == 0)] | |
new_topics = new_topics.reset_index(drop=True) | |
num_new = new_topics.shape[0] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
if num_new > 0: | |
subject = f'{num_new} new topics!' | |
msg = f'New topics: \n {new_topics}\n\n {my_url}' | |
send_email(subject, msg) | |
else: | |
print('No new topics found.') | |
sleep(600) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Topic Replies Activity time | |
1 Star Wars Survey Project - Converting Yes/No r... 0 1 m | |
2 Distance above and below the mean of a distrib... 3 21 m | |
3 Help : Reading from dictionary Non-DQ Courses... 10 1 h | |
4 Categorical and Quantitative data Non-DQ Courses 4 2 h | |
5 Problem to finish the exercise and mission 392... 2 2 h | |
6 Cannot connect with DB from Jupyter Notebook ... 8 5 h | |
7 Intro courses not working (possible bug) DQ C... 1 7 h | |
8 How does one solve “invalid literal for int() ... 1 8 h | |
9 Problem with saving takeaways DQ Courses dat... 3 8 h |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Topic | Unnamed: 1 | Replies | Likes | Views | Activity | |
---|---|---|---|---|---|---|
About the Q&A category Q&A Post technical questions and get solutions - we want to help you make progress towards your goals as fast as possible. Remember “there are no stupid questions”. It has 2 rooms for you to participate in: Course related … read more | 0 | 2.0 | 132 | 9d | ||
CIA FACTDATA db and null values Q&A 257 | 0 | 2 | 8m | |||
Guided Project 1: Deleting Wrong Data DQ Courses | 4 | 2.0 | 7 | 1h | ||
Technical issues with answer submission DQ Courses help dataquest-platform 243-6 243 | 4 | 1.0 | 21 | 1h | ||
Guided Project: Profitable App exercise - dictionary to keep only max reviews Q&A help | 5 | 8 | 2h | |||
R Data Cleaning Error DQ Courses 323 323-7 | 0 | 4 | 2h | |||
Exploring Ebay Cars Sales Data DQ Courses 294 294-4 | 18 | 7.0 | 524 | 2h | ||
Bitwise & and Logical and DQ Courses 370 370-6 | 1 | 4 | 2h | |||
Pandas: how to get column statistics without outliers? DQ Courses 294 294-5 | 0 | 7 | 3h |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Topic | Replies | Activity | time | |
---|---|---|---|---|
CIA FACTDATA db and null values Q&A 257 | 0 | 8 | m | |
Guided Project 1: Deleting Wrong Data DQ Courses | 4 | 1 | h | |
Technical issues with answer submission DQ Courses help dataquest-platform 243-6 243 | 4 | 1 | h | |
Guided Project: Profitable App exercise - dictionary to keep only max reviews Q&A help | 5 | 2 | h | |
R Data Cleaning Error DQ Courses 323 323-7 | 0 | 2 | h | |
Exploring Ebay Cars Sales Data DQ Courses 294 294-4 | 18 | 2 | h | |
Bitwise & and Logical and DQ Courses 370 370-6 | 1 | 2 | h | |
Pandas: how to get column statistics without outliers? DQ Courses 294 294-5 | 0 | 3 | h | |
Profitable App Profiles - header row DQ Courses 350 350-5 | 3 | 3 | h |
OlderNewer