Skip to content

Instantly share code, notes, and snippets.

@ipeluffo
Last active July 1, 2019 23:01
Show Gist options
  • Save ipeluffo/8966c6e5aaac289e90f8267fba069982 to your computer and use it in GitHub Desktop.
Save ipeluffo/8966c6e5aaac289e90f8267fba069982 to your computer and use it in GitHub Desktop.
Tweets hashtags real-time processing with Faust - Command
@app.command(option("--track", type=str, help="List of words in CSV format to filter stream of tweets", required=True))
async def hashtags_events_generator(self, track: str):
"""
Generate Kafka events for each hashtag found in tweets matching with the track used to filter the stream.
"""
logger.info(f"Searching tweets with hashtags: {track}")
client = _get_twitter_client()
req = client.stream.statuses.filter.post(track=track)
async with req as stream:
async for tweet in stream:
tweet_hashtags = _get_hashtags_from_tweet(tweet=tweet)
await _produce_events_for_hashtags(hashtags=tweet_hashtags)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment