Skip to content

Instantly share code, notes, and snippets.

View jamiefrench's full-sized avatar

Jamie French jamiefrench

1 follower · 0 following
View GitHub Profile
@jamiefrench
jamiefrench / aws-glue-pycharm-mac.md
Last active September 12, 2024 16:18
PyCharm developing and testing AWS Glue job scripts locally on a MAC

Developing and testing AWS Glue job scripts locally on a MAC

Prerequisites

  • Docker installed
  • PyCharm Professional installed
  • PyCharm AWS Toolkit plugin installed
  • GIT installed
  • AWS Account with credentials file and profiles

PyCharm/Docker initial setup

@jamiefrench
jamiefrench / pyenv_virtualenv.md
Last active April 11, 2023 15:34
Python Environments With Pyenv And Virtualenv

brew upgrade

brew install pyenv

brew install pyenv-virtualenv

Add the following lines to ~/.zprofile and ~/.zshrc (or ~/.bash_profile and ~/.bashrc if you’re still using bash):

~/.zprofile

@jamiefrench
jamiefrench / install_pyspark_on_m1_mac.md
Created January 16, 2023 16:07 — forked from brianspiering/install_pyspark_on_m1_mac.md
Installation guide to pyspark on M1 Mac

Install Spark

Run all of these commands at the command line (not in a Jupyter Notebook). The command line will have more informative error messages and if we need complete additional steps, we'll get the messages.

Spark is a framework within the Scala programming language. Scala uses the JVM (Java Virtual Machine) so you'll need install Java.

If you use homebrew:




  


  





    


          
    

      

        
        

          

              @jamiefrench
          

          

            
                jamiefrench
                / athena-elb-partitioned-logs.txt
            
            

              Last active
              July 10, 2019 19:26
            

              
                AWS Athena Classic ELB - Partitioned by Year, Month, Day
              
          

        

      

        


      

    
    


        



  


  

        

          
          CREATE EXTERNAL TABLE IF NOT EXISTS {{DATABASE_NAME.TABLE_NAME}} (
        

        

          
           
        

        

          
           timestamp string,
        

        

          
           elb_name string,
        

        

          
           request_ip string,
        

        

          
           request_port int,
        

        

          
           backend_ip string,
        

        

          
           backend_port int,
        

        

          
           request_processing_time double,
        

        

          
           backend_processing_time double,
        

  





    


  





    


          
    

      

        
        

          

              @jamiefrench
          

          

            
                jamiefrench
                / keybase.md
            
            

              Created
              October 20, 2018 14:43
            

          

        

      

        


      

      

    
Keybase proof


I hereby claim:


    

  • I am jamiefrench on github.
    • 

  • I am jamiefrench (https://keybase.io/jamiefrench) on keybase.
    • 

  • I have a public key ASDxsFdDUKoMkUgQSrxZp1-5WUHgD8DeYFbH-B-ta9J5-wo
    • 



To claim this, I am signing this object: