QPyWarrior sjoshiAI

## positional_encoding (1).ipynb

      
              1 file
            
          
              0 forks
            
          
                0 comments
              
            
              0 stars
            
          
                sjoshiAI
                / positional_encoding (1).ipynb
            
            
              Created
              February 10, 2026 20:08
            
              
                pe_sin_cos
              
          
      Loading

      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## layernorm.ipynb

      
              1 file
            
          
              0 forks
            
          
                0 comments
              
            
              0 stars
            
          
                sjoshiAI
                / layernorm.ipynb
            
            
              Created
              February 3, 2026 21:47
            
          
      Loading

      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## ib_2.ipynb

      
              1 file
            
          
              0 forks
            
          
                0 comments
              
            
              0 stars
            
          
                sjoshiAI
                / ib_2.ipynb
            
            
              Created
              March 26, 2022 10:43
            
          
      Loading

      Sorry, something went wrong. Reload?
      Sorry, we cannot display this file.
      Sorry, this file is invalid so it cannot be displayed.
      
          Viewer requires iframe.
      
    
## gist:d9b4d101a0b4877a790552bee622d878
Fair coin = 0.5 probability for head.
Single toss = 2 outcomes
Entropy = -0.5*log(0.5) - 0.5*log(0.5) = 1

N tosses = 2^N outcomes
Entropy = -2^N*(1/2^N)*log((1/2^N)) = N

Consider a biased coin
p(head) = 1, p(tails) = 0
Entropy = -1*log(1) - 0*log(0) = 0 (assuming log(0) is defined)
	Fair coin = 0.5 probability for head.
	Single toss = 2 outcomes
	Entropy = -0.5log(0.5) - 0.5log(0.5) = 1

	N tosses = 2^N outcomes
	Entropy = -2^N(1/2^N)log((1/2^N)) = N

	Consider a biased coin
	p(head) = 1, p(tails) = 0
	Entropy = -1log(1) - 0log(0) = 0 (assuming log(0) is defined)