Skip to content

Instantly share code, notes, and snippets.

@arrieta
Created March 4, 2018 00:17
Show Gist options
  • Save arrieta/d7fd2697f09202e29883ec091bcddc9f to your computer and use it in GitHub Desktop.
Save arrieta/d7fd2697f09202e29883ec091bcddc9f to your computer and use it in GitHub Desktop.
An approach to multithreaded logging to an existing buffer
#include <algorithm>
#include <atomic>
#include <functional>
#include <iostream>
#include <sstream>
#include <string>
#include <thread>
#include <vector>
#include <iterator>
#include <chrono>
struct Writer {
Writer(std::atomic<char*>* loc) : loc(loc) {}
void operator()() {
std::ostringstream os;
os << "[thread " << std::this_thread::get_id() << "] some message...\n";
const std::string s = os.str();
for (int k = 0; k < 20; ++k) {
auto beg = std::atomic_fetch_add(loc, s.size());
std::copy(s.begin(), s.end(), beg);
// I add the sleep because otherwise it is difficult for few threads with
// small messages to interleave. Of course, this is completely unnecessary
// in production, it is just to make the example clear.
std::this_thread::sleep_for(std::chrono::nanoseconds(1));
}
}
std::atomic<char*>* loc;
};
int main() {
// For production, a worker with two atomic pointers to the beginning and end
// of a memory map could work. The last pointer is fixed (in the worker), and
// the first pointer is used to claim a spot. When the pointers are equal,
// that means the memory map is full, and we start buffering or dumping
// messages. Another thread could swap the underlying memory map and
// atomically change the worker pointers again. The logic could me implemented
// with some sort of sentinel value (which could be nullptr, at the cost of
// checking every log write).
std::string log(10000, 'X'); // poor man's memory map
auto pos = std::atomic<char*>(log.data());
std::vector<std::thread> threads;
for (int k = 0; k < 12; ++k) {
threads.emplace_back(Writer(&pos)); // all receive the same atomic variable
}
for (auto&& thread : threads) {
thread.join();
}
std::cout << log << "\n";
}
@arrieta
Copy link
Author

arrieta commented Mar 4, 2018

Sample output:

[thread 0x700002a4e000] some message...
[thread 0x700002ad1000] some message...
[thread 0x700002b54000] some message...
[thread 0x700002d60000] some message...
[thread 0x700002bd7000] some message...
[thread 0x700002a4e000] some message...
[thread 0x700002cdd000] some message...
[thread 0x700002c5a000] some message...
[thread 0x7000029cb000] some message...
[thread 0x700002948000] some message...
[thread 0x700002b54000] some message...
[thread 0x700002de3000] some message...
[thread 0x700002ad1000] some message...
[thread 0x700002d60000] some message...
[thread 0x700002bd7000] some message...
[thread 0x700002a4e000] some message...
[thread 0x700002e66000] some message...
[thread 0x700002948000] some message...
[thread 0x700002cdd000] some message...
[thread 0x7000029cb000] some message...
[thread 0x700002c5a000] some message...
[thread 0x700002ee9000] some message...

[ many more messages ] 

[thread 0x700002d60000] some message...
[thread 0x700002bd7000] some message...
[thread 0x700002a4e000] some message...
[thread 0x700002de3000] some message...
[thread 0x700002e66000] some message...
[thread 0x700002c5a000] some message...
[thread 0x700002ad1000] some message...
[thread 0x700002948000] some message...
[thread 0x700002cdd000] some message...
[thread 0x7000029cb000] some message...
[thread 0x700002b54000] some message...
[thread 0x700002ee9000] some message...
[thread 0x700002bd7000] some message...
[thread 0x700002a4e000] some message...
[thread 0x700002de3000] some message...
[thread 0x700002c5a000] some message...
[thread 0x700002d60000] some message...
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment