Skip to content

Instantly share code, notes, and snippets.

View pierdom's full-sized avatar

Pierdomenico Fiadino pierdom

2 followers · 2 following
View GitHub Profile
@pierdom
pierdom / cdf_pdf_subplots.py
Last active March 22, 2022 12:10
[CDF and PDF side by side in matplotlib] A Cumulative Distribution Function (CDF) and a Power Distribution Function (PDF) side-by-side using matplotlib's subplot and seaborn's distplot. In the example below, the dataset is a Pandas's DataFrame. #python #matplotlib #visualization #statistics #datascience
# plot hourly action rate (HAR) distribution
import matplotlib as plt
import seaborn as sns
# settings
f, axes = plt.subplots(1, 2, figsize=(18,6), dpi=320)
axes[0].set_ylabel('fraction (PDF)')
axes[1].set_ylabel('fraction (CDF)')
# left plot (PDF) # REMEMBER TO CHANGE bins, xlim PROPERLY!!
@pierdom
pierdom / ssh_timeout.md
Last active September 7, 2017 07:58
[Three methods for avoiding timeouts during SSH sessions] #linux #macosx #sysadmin #networking

How to disable SSH timeout

Three methods for avoiding timeouts during SSH sessions.

Client-side methods

Use Client-side methods if you have no admin rights on the SSH servers or do not what to alter server's settings. This methods consists in instructing the client to send a keep alive every specified interval.

If you have admin rights on the client

@pierdom
pierdom / crontab.md
Last active September 7, 2017 07:57
[crontab simple how-to] How to add a cron task with crontab #linux #sysadmin

Editing cron jobs in unix:

crontab -e # edit crontab
crontab -l # list jobs

*    *    *  *    *        command to be executed
- - - - -
@pierdom
pierdom / ssh_server_as_proxy.md
Last active September 7, 2017 07:56
[Set SSH server as SOCKS proxy] #linux #macosx #sysadmin #networking

Set SSH server as SOCKS proxy

$ ssh -D 12345 username@remoteserver

Don't close the terminal and set localhost:12345 as SOCKS in the browser (or system wide).

@pierdom
pierdom / ssh_tunnel_background.md
Last active September 7, 2017 07:56
[SSH tunnels on background] #linux #macosx #sysadmin #networking

Background tunnel (it works with -D for creating a SOCKS proxy, as well)

ssh -f -N -L 2222:localhost:22 username@serveraddr

To avoid connection timeout, install autossh (which uses the very same syntax of the standard ssh). Then:




  


  




    


          
    

      

        
        

          

              @pierdom
          

          

            
                pierdom
                / until_command.md
            
            

              Last active
              September 7, 2017 07:56
            

              
                [until bash command] An example on how to use the until command on bash to check when connectivity is back #linux #macosx #syadmin
              
          

        

      

        

  

      

    
Example: ping google.com until it becomes reachable (for testing when internet connection is back), then notify with an audio message


until ping -W1 -c1 google.com; do sleep 5; done && say connected


It's also possible to use different notification means (e.g., notify-send on Ubuntu, growl on Mac, pushover, etc.).


Notes on ping:





  


  




    


          
    

      

        
        

          

              @pierdom
          

          

            
                pierdom
                / ssh_no_password.md
            
            

              Last active
              September 7, 2017 07:55
            

              
                [Set up public key authentication (no password) for SSH] #linux #mac #sysadmin #networking
              
          

        

      

        

  

      

    
On client, generate public and private keys pair:


localusr@client:~$ ssh-keygen -t rsa


Copy public key on the server (create the remote ~/.ssh directory if it does not exists):


localusr@client:~$ cat .ssh/id_rsa.pub | ssh remoteusr@server 'cat >> .ssh/authorized_keys'



  


  




    


          
    

      

        
        

          

              @pierdom
          

          

            
                pierdom
                / local_net_discovery.md
            
            

              Last active
              September 7, 2017 07:55
            

              
                [Discover IP and MAC addresses in local network] using standard ping or arp-scan #linux #macosx #networking #sysadmin
              
          

        

      

        

  

      

    
Note: I assume the addressing space of the local network is 192.168.1.1/255.255.255.0.


Look for IP addresses (no admin rights required)


for ip in $(seq 1 254); do ping -c 1 192.168.1.$ip>/dev/null; [ $? -eq 0 ] && echo "192.168.1.$ip UP" || : ; done


Look for MAC and IP addresses (with admin rights)



  


  




    


          
    

      

        
        

          

              @pierdom
          

          

            
                pierdom
                / zeppelin_and_matplotlib.py
            
            

              Last active
              September 7, 2017 07:53
            

              
                [Visualize matplotlib plots in Zeppelin notebook] #zeppelin #python #spark #bigdata #matplotlib #visualization
              
          

        

      

        

  

    
    


        



  


  

        

          
          def show(graph):
        

        

          
            img = StringIO.StringIO()
        

        

          
            graph.savefig(img, format='svg')
        

        

          
            img.seek(0)
        

        

          
            print "%html <div style='width:1000px'>" + img.buf + "</div>"
        

        

          
           
        

        

          
           
        

        

          
          show(plt)
        

  





    


  




    


          
    

      

        
        

          

              @pierdom
          

          

            
                pierdom
                / conditional_update_pandas.py
            
            

              Last active
              October 28, 2019 21:50
            

              
                [Conditionally update Pandas DataFrame column] It is equivalent to SQL: UPDATE table SET column_to_update = 'value' WHERE condition #python #pandas #datascience
              
          

        

      

        

  

    
    


        



  


  

        

          
          # set 'column_to_update' to 0 when 'condition_column' is 0
        

        

          
          df.loc[df['condition_column'] == 0, 'column_to_update'] = 0
        

        

          
          

        

        

          
          # or with a if-then-else scheme
        

        

          
          df["mycol"] = np.where((df.mycol2=="test"), 0, 1)
        

        

          
          #                           condition     then else