Skip to content

Instantly share code, notes, and snippets.

View rupeshtiwari's full-sized avatar
🎯
Focusing

Rupesh Tiwari rupeshtiwari

🎯
Focusing
View GitHub Profile
@rupeshtiwari
rupeshtiwari / apple script learning.md
Last active May 13, 2024 21:15
apple script learning

Hellow World App

tell application "Finder"
	display dialog "Hello World"
end tell
@rupeshtiwari
rupeshtiwari / apple script to move the presentation on top left fhd.md
Last active May 13, 2024 22:22
apple script to move the presentation on top left with full hd 1920x1080, resize, fhd,

Resize keynote to FHD (1920x1080)

tell application "Keynote"
	activate
	try
		set theDoc to front document
	on error
		set theDoc to make new document
	end try
@rupeshtiwari
rupeshtiwari / Mathura and Vrindavan.md
Last active May 7, 2024 19:56
Mathura and Vrindavan

Here's a streamlined guide with highlighted key points for each major attraction in Mathura, focusing on the essential aspects and must-do activities, particularly from a Hindu cultural perspective:

1. Shri Krishna Janmabhoomi Temple

  • Essentials: Birthplace of Lord Krishna, spiritual ceremonies.
  • Must-Do: Participate in the evening aarti; it's a profound devotional experience.

2. Dwarkadhish Temple

  • Essentials: Rich architecture, vibrant festivities.
  • Must-Do: Attend the festival celebrations during Janmashtami for a lively and colorful experience.
@rupeshtiwari
rupeshtiwari / seafile-with-segate-drive.md
Last active April 29, 2024 00:08
Personal cloud on MacOs, local cloud,

Certainly! Below is a README file you can use to guide yourself or others on setting up a Seafile Docker instance with data storage on an external drive (in this case, a Seagate external drive, which we'll refer to as ExtDrive). This document will include all necessary steps to ensure that Seafile uses this external drive for storing files.


Setting Up Seafile on Docker with External Drive Storage

This README provides detailed instructions on how to set up Seafile using Docker, specifically configured to store data on an external Seagate drive named ExtDrive. Follow these steps to ensure your data is stored securely and externally from your main system.

Prerequisites

@rupeshtiwari
rupeshtiwari / Data Analytics Late Arrival Handling.md
Last active April 25, 2024 14:21
Data Analytics Late Arrival Handling

Techniques to handle late arrival

Watermarks and Allowed Lateness are both vital techniques in managing late data in stream processing systems, but they serve slightly different purposes and are often used in conjunction to maximize data integrity and processing efficiency. Here’s an in-depth look at when and why you might choose to use each technique, or both together, along with real-world industry examples.

Watermarks

Purpose: Watermarks are primarily used to handle out-of-order data. They provide a way to estimate the "completeness" of data up to a certain point in time, based on event timestamps.

When to Use: Use watermarks when:

  • You expect data to arrive out of order.
  • You need a mechanism to know when to close a window and process its data.
@rupeshtiwari
rupeshtiwari / 00_GCP Data Services.md
Last active April 25, 2024 14:57
GCP Data Services, GCP services, Google services, Google

Here's the table sorted chronologically based on the release date of each Google Cloud service:

Google Cloud Service Release Date Based on/Open-source Inspiration Open-source Start Date Notes
Google BigQuery 2010 Dremel (Internal Google Tech) N/A BigQuery is inspired by Dremel but is not directly based on open-source technology.
Google Cloud Dataflow 2014 Apache Beam 2016 (as Apache Beam) Initially developed by Google as Google Dataflow, then donated to the Apache Software Foundation as Apache Beam.
Google Cloud Composer 2018 Apache Airflow 2015 Developed by Airbnb and later open-sourced as Apache Airflow, which Google adopted for Cloud Composer.
Google Data Fusion 2019 CDAP (Cask Data Application Platform) 2011
@rupeshtiwari
rupeshtiwari / 01_usecase Scalable Triumph: Migrating FinTrust Bank's Data Infrastructure to the Cloud.md
Last active April 26, 2024 14:59
Use cases for Data Analytics Customer Story, GCP, AWS, customer story, use cases, real world, usecase

Data Warehouse Migration Story for FinTrust Bank

Framework Step Details
Situation
  1. FinTrust Bank, with an annual revenue of $12 billion, was facing a high-stakes challenge when its existing systems couldn't handle over 500 million transactions per day during a critical testing phase with a key e-commerce client.
  2. This client was projected to increase annual revenues by 15% ($1.8 billion).
  3. Key stakeholders involved were the client's CIO, CTO, and CSO, highlighting the strategic importance of the project.
Task The urgent task was to stabilize and scale the bank’s data processing capabilities to not only retain the e-commerce client but also to set a foundation for scalable, compliant growth suitable for high-volume transaction environments.
@rupeshtiwari
rupeshtiwari / 00_README.md
Last active April 23, 2024 20:40
Kubernetes from Basics to Guru
@rupeshtiwari
rupeshtiwari / 00_Data Architecture README.md
Last active April 25, 2024 17:17
All Apache Data Processing Frameworks and Tools