Skip to content

Instantly share code, notes, and snippets.

@lozhnikov
lozhnikov / transformer_draft.cpp
Last active August 20, 2020 10:25
Transformer model draft
#include <mlpack/methods/ann/ffn.hpp>
#include <mlpack/methods/ann/layer/layer.hpp>
using namespace mlpack::ann;
/* This layer accepts the output of the positional encoding or the output
* of the previous encoder block as the input.
* The output is one key/value matrix.
*/
Sequential<>* CreateEncoderBlock() {
@lozhnikov
lozhnikov / speedup_test.cpp
Last active March 31, 2020 10:00
The test indicates that there's no sense in parallelizing simple loops over large datasets
#include <iostream>
#include <iomanip>
#include <random>
#include <chrono>
using namespace std;
using namespace chrono;
vector<pair<size_t, double>> TestParallel(const vector<size_t>& sizes) {
random_device rd;