Skip to content

Instantly share code, notes, and snippets.

@antiduh
Last active April 16, 2019 03:11
Show Gist options
  • Save antiduh/426a4d22ab2449601842 to your computer and use it in GitHub Desktop.
Save antiduh/426a4d22ab2449601842 to your computer and use it in GitHub Desktop.
Concatenates a list of strings into a single string by performing exactly one allocation.
/// <summary>
/// Provides the ability to concatenate arrays of strings and arrays of character arrays into a
/// single string without performing intermediate copies of the strings/characters, nor the
/// arrays that contain them.
/// </summary>
/// <remarks>
/// Methods like `string.Concat( string[] )` make an intermediate copy of the array containing
/// the strings (but not the strings themselves). If you have a large array of strings, then
/// copying that array can represent a moderate amount of unnecessary work. For instance,
/// copying a 500k element array in a 64-bit process requires copying about 4 MB of memory.
///
/// Other methods like `string.Join()` may use StringBuilders and many intermediate copies of
/// the string/character data, causing more than a few full copies of all data, nevermind just
/// the arrays pointing to the data. If you had a 500k element array of strings, each 200
/// characters long, then that represents a total of 200 MB worth of unnecessary copies.
///
/// For the discussion that went into building this class, see the following StackOverflow post:
/// http://stackoverflow.com/questions/32217255/
/// </remarks>
public static unsafe class FastConcat
{
/// <summary>
/// Concatenates the list of strings into a single string, performing exactly one
/// allocation. This does not perform any intermediate copies of the character data nor the
/// arrays or lists containing the strings.
/// </summary>
/// <param name="list">A list of strings to concat.</param>
/// <returns></returns>
public static string Concat( IReadOnlyList<string> list )
{
string destinationString;
int destLengthChars = 0;
for( int i = 0; i < list.Count; i++ )
{
destLengthChars += list[i].Length;
}
destinationString = new string( '\0', destLengthChars );
unsafe
{
fixed( char* origDestPtr = destinationString )
{
char* destPtr = origDestPtr; // a pointer we can modify.
string source;
for( int i = 0; i < list.Count; i++ )
{
source = list[i];
fixed( char* sourcePtr = source )
{
// `Buffer.MemoryCopy()` was first introduced in .Net 4.6 There are a
// few techniques for implementing your own version of this method if
// you need support for .Net 4.5 or below,
// for instance: http://stackoverflow.com/questions/2658380/
Buffer.MemoryCopy(
sourcePtr,
destPtr,
long.MaxValue,
source.Length * sizeof( char )
);
}
destPtr += source.Length;
}
}
}
return destinationString;
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment