llllllllll

from itertools import chain, repeat
import sys
import types

# byte offsets for structures
if hasattr(sys, 'gettotalrefcount'):
    # under PyDEBUG builds
    _f_localsplus_offset = 392
    _ob_item_offset = 40
else:

llllllllll

#include <array>
#include <tuple>
#include <type_traits>
#include <utility>


namespace cdict {
template<char... cs>
using string = std::integer_sequence<char, cs...>;

llllllllll

#include <ctype.h>
#include <stdbool.h>
#include <stdio.h>
#include <string.h>

#include <Python.h>
#include <structmember.h>
#include <frameobject.h>

llllllllll

Comments on optimizations around string concatenation.

Note: The code links are to CPython 3.8.5, the most recent release when this was written.

I was recently asked about a performance optimization in CPython around using += and + for string objects. As some people may already know, if you use += or + a string, it can sometimes be just as fast as ''.join. The question was to explain when that optimization couldn't be performed.

We will be going through the following example scenarios:

llllllllll

Initial Observations

The provided benchmark script is using Python's timeit module for benchmarking. I noticed that I was getting very different results between runs even with the same compiler, so I first switched to pyperf to attempt to get more stable results.

I am not sure if there is a Python API for pyperf, so I started by writing a small bash wrapper for benchmark:

llllllllll

#include <cstdlib>
#include <compare>
#include <ranges>

namespace jj {
class iota : public std::ranges::view_base {
public:
    iota(long start, long end) : m_start(start), m_end(end) {}
    iota(long end) : iota(0, end) {}
    iota() : iota(0, 0) {}

llllllllll

from collections import UserString


class ResolvedDocString(UserString):
    __class__ = str


class DeferDocString(UserString):
    def __init__(self, f):
        self._f = f

llllllllll

class WithLazyDoc:                                                                   
    def __init__(self, get_doc, f): 
        self._get_doc = get_doc 
        self._f = f 
      
    @property 
    def __wrapped__(self): 
        return self._f 
 
    @property 

llllllllll

""" Example:

In [5]: class Class:
   ...:     def __init__(self, a, b):
   ...:         self.a = a
   ...:         self.b = b
   ...: 
   ...:     @self_scope
   ...:     def f(self):
   ...:         return a + b

llllllllll

In [1]: class Descr:
   ...:     def __init__(self):
   ...:         self.get_counter = 0
   ...: 
   ...:     def __get__(self, instance, owner):
   ...:         if instance is None:
   ...:             return self
   ...:         self.get_counter += 1
   ...:         return 1
   ...: