Skip to content

Instantly share code, notes, and snippets.

View kofemann's full-sized avatar
Working on the next great thing....

Tiramisu Mokka kofemann

Working on the next great thing....
View GitHub Profile
@kofemann
kofemann / nfs_pdml.py
Last active July 13, 2023 12:34
Simple python script to parse NFS packes in pdml file generated by wireshark
#!/usr/bin/env python3
# Licensed under GNU GENERAL PUBLIC LICENSE Version 2
#
# parses files produced by : `tshark -r file.pcap -q -n -T pdml`
#
import xml.sax
import sys
DATA AND METADATA COHERENCE
Some modern cluster file systems provide perfect cache coherence among their clients. Perfect cache coherence among disparate NFS clients is expensive to achieve, especially on wide area networks. As such, NFS settles for weaker cache coherence that satisfies the requirements of most file sharing types.
Close-to-open cache consistency
Typically file sharing is completely sequential. First client A opens a file, writes something to it, then closes it. Then client B opens the same file, and reads the changes.
When an application opens a file stored on an NFS version 3 server, the NFS client checks that the file exists on the server and is permitted to the opener by sending a GETATTR or ACCESS request. The NFS client sends these requests regardless of the freshness of the file's cached attributes.
When the application closes the file, the NFS client writes back any pending changes to the file so that the next opener can view the changes. This al
@kofemann
kofemann / draft-mkrtchyan-token-based-storage-access.md
Last active July 8, 2023 21:55
Tocken based storage access problem statement

Token-based storage access problem statement

by Tigran Mkrtchyan, Deutsches Elektronen-Synchrotron DESY, June 2023.

Status of this Memo

By submitting this Internet-Draft, I certify that any applicable patent or other IPR claims of which I am aware have been disclosed, or will be disclosed, and any of which I become aware will be disclosed, in accordance with RFC 3668.

@kofemann
kofemann / CreateBenchmark.java
Last active August 29, 2022 12:54
Simple chimera file freate rate benchmark
package org.dcache.chimera.posix;
import static java.nio.charset.StandardCharsets.UTF_8;
import static java.util.UUID.randomUUID;
import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import java.io.FileReader;
import java.io.IOException;
@kofemann
kofemann / NettyGroupBenchmark.java
Last active October 30, 2022 05:21
Netty Event loop benchmark
package dev.kofemann.playground;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.epoll.EpollEventLoopGroup;

dCache admin interface with YubiKeys

(by assuming you already have a yubikey ... )

pre-requisites

  • OpenSSH version 8.2 or later on the client
  • dCache 7.2 or later

Generate a new key pair

@kofemann
kofemann / dcache-infra.sh
Last active May 10, 2021 09:58
simple layout file with minimal dcache-pNFS setup and zookeeper, kafka, postgres in containers
#!/bin/sh
if [ $# != 1 ]
then
echo "Usage `basename $0` <start|stop>"
exit 1
fi
POD_NAME=dcache-infrastructure
LOCAL_ADDRESS=`hostname -f`
@kofemann
kofemann / decode_nfs4j_fh.py
Created February 3, 2021 12:04
decode NFS file handle for nfs4j
#!/usr/bin/env python3
import sys
import struct
import binascii
fh = sys.argv[1].upper()
version = struct.unpack('b', bytes.fromhex(fh[0:2]) )[0]
magic = fh[2:8]
@kofemann
kofemann / 100-e7470.conf
Created September 2, 2020 09:15
systcl configuration for dell e7470
vm.laptop_mode=5
vm.dirty_writeback_centisecs = 15000
vm.swappiness=10
fs.inotify.max_user_watches = 524288
@kofemann
kofemann / vscode-init.py
Last active April 4, 2023 13:36
Make vscode ready for kernel development
#!/usr/bin/env python3
# This program runs "make --dry-run" then processes the output to create a visual studio code
# c_cpp_properties.json file
# Origin:
# https://iotexpert.com/2020/06/01/stupid-python-tricks-vscode-c_cpp_properties-json-for-linux-kernel-development/
# by: Alan Hawse
import json