tamuhey

How to calculate the alignment between BERT and spaCy tokens effectively and robustly

site: https://tamuhey.github.io/tokenizations/

Natural Language Processing (NLP) has made great progress in recent years because of neural networks, which allows us to solve various tasks with end-to-end architecture. However, many NLP systems still require language-specific pre- and post-processing, especially in tokenizations. In this article, I describe an algorithm that simplifies calculating correspondence between tokens (e.g. BERT vs. spaCy), one such process. And I introduce Python and Rust libraries that implement this algorithm. Here are the library and the demo site links:

• repo: https://github.com/tamuhey/tokenizations

piyueh

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# vim:fenc=utf-8
#
# Copyright © 2019 Pi-Yueh Chuang <pychuang@gwu.edu>
#
# Distributed under terms of the MIT license.

"""An example of using tfp.optimizer.lbfgs_minimize to optimize a TensorFlow model.

ZeccaLehn

jashkenas

var active = false;

function changeRefer(details) {
  if (!active) return;
  
  for (var i = 0; i < details.requestHeaders.length; ++i) {
    if (details.requestHeaders[i].name === 'Referer') {
      details.requestHeaders[i].value = 'http://www.google.com/';
      break;
    }

bcremer

systemd for Administrators

Chronological list of the "systemd for Administrators" series published on 0pointer.net/blog:

• Part I: Verifying Bootup
• Part II: Which Service Owns Which Processes?
• Part III: How Do I Convert A SysV Init Script Into A systemd Service File?
• Part IV: Killing Services
• Part V: The Three Levels of "Off"
• Part VI: Changing Roots