This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I see where the confusion might have come from, especially with the introduction of a query value "2" without a clear explanation. Let's clarify the concept of a query in the context of the attention mechanism and adjust the example for better understanding. | |
In the context of attention mechanisms, a **query** (\(\mathbf{q}\)) represents the current item or piece of information the model is focusing on or trying to relate to other pieces of information. The **keys** (\(\mathbf{k}_i\)) represent aspects or features of other items in a dataset, and the **values** (\(\mathbf{v}_i\)) represent the content or information of those items. The goal of the attention mechanism is to determine how relevant each item (key-value pair) in the dataset is to the query and to produce a weighted combination of these items' values based on their relevance. | |
Let's correct and simplify the example without introducing arbitrary numbers for the query: | |
Imagine we have a dataset of items, each with a key (\(\mathbf{k}_i\)) and a val |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Environment - dynamics of the problem | |
Reward - specifies the goal | |
Agent - | |
* agent state | |
* policy | |
* value function estimate - optional | |
* model - optional | |
## 1. Inside the agent: state | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
t is single timestep | |
y hat of t = function of input vector x at time t | |
y hat is predicted | |
temporal dependence | |
recurence relationship | |
h is internal state of memeory passed timestep to timestep |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!DOCTYPE html> | |
<html> | |
<head> | |
<style> | |
.editor { font-family: 'Roboto Mono', monospace; font-size: 12px; outline: none; overflow-y: auto; padding-left: 48px; counter-reset: line; } | |
.editor div { display: block; position: relative; white-space: pre-wrap; } | |
.editor div::before { content: counter(line); counter-increment: line; position: absolute; right: calc(100% + 16px); opacity: 0.5; } | |
</style> | |
</head> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//from https://css-tricks.com/snippets/jquery/smooth-scrolling/ | |
// Scroll to specific values | |
// scrollTo is the same | |
window.scroll({ | |
top: 2500, | |
left: 0, | |
behavior: 'smooth' | |
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//----------------------------------*\ | |
// TRIGONOMETRY FUNCTIONS | |
//----------------------------------*/ | |
// # Trigonometry in CSS | |
// | |
// - Through Taylor/Maclaurin polynomial representation: http://people.math.sc.edu/girardi/m142/handouts/10sTaylorPolySeries.pdf | |
// - Useful if you don't want to use JS. | |
// - With CSS Variables. | |
// - `calc()` can't do power (x ^ y) so I used multiplication instead. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apiVersion: v1 | |
kind: Service | |
metadata: | |
name: hello-kubernetes-first | |
spec: | |
type: ClusterIP | |
ports: | |
- port: 80 | |
targetPort: 8080 | |
selector: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import {JWK, JWS} from 'node-jose'; | |
import {join} from 'path'; | |
import {writeFileSync, readFileSync} from 'fs'; | |
import fetch from 'node-fetch'; | |
/** | |
* @requires Models | |
*/ | |
import {IdTokenPayload, AccessTokenPayload} from '../models'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/** | |
* @namespace @monorepo/common | |
* @desc barrel file to export the common files | |
*/ | |
export * from './models'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"name": "@monorepo/nestjs", | |
"version": "1.0.0", | |
"private": true, | |
"dependencies": { | |
"@monorepo/common": "1.0.0" | |
} | |
} |
NewerOlder