Skip to content

Instantly share code, notes, and snippets.

@vibhcool

vibhcool/gist:64b9334fc4aa64dcfd86c49e6aec9cad

Last active August 28, 2017 17:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vibhcool/64b9334fc4aa64dcfd86c49e6aec9cad to your computer and use it in GitHub Desktop.
Save vibhcool/64b9334fc4aa64dcfd86c49e6aec9cad to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.md

GSOC-2017 Work Product Submission, FOSSASIA

Student: Vibhor Verma

Github Handle: @vibhcool

Project: Add more harvester to loklak server

Organisation: FOSSASIA

This summer, I contributed to FOSSASIA organization with my peer-developers in Loklak projects like Loklak Server and Loklak Scraper JS. I also engaged in Loklak Search, Loklak Wok and Loklak Apps projects in review process and discussions.

I contributed primarily in Loklak Server to set up multiscraper system for the web-scrapers. This task included:

  • Add BaseScraper as the abstract class for the web-scrapers.
  • Add Post object as generic message holder for all results scraped by web scrapers
  • Create iterator object based on existing iterator Timeline
  • Configure SearchServlet to set up multiscraper system.
  • Index the results scraped by webscrapers and fetch them from cache. The related PR to this change is currently in review process.

Other tasks in Loklak Server included:

  • To add filter for TwitterScraper
  • Kubernetes deployment of Loklak Server with Nginx Ingress Controller and Kube-Lego for SSL encryption.
  • Refactoring of existing code to improve code quality.
  • Set up Testing framework for Loklak Server.

Project Link

Project Repo

My Contributions

Loklak Server

Loklak Scraper JS

Additional Links

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment