I am Satyam Kumar and I worked on the project Improve Python Scrapers with Kiwix. My project was aimed at improving the current status of the python based scrapers and adding Zim Farm support for them. The key goals of the project were -
- Adding support for S3 based optimization cache in scrapers
- Adding support for new and improved scrapers in the Zim Farm
- Using python-libzim instead of zimwriterfs in the scrapers to write ZIM files