Skip to content

Instantly share code, notes, and snippets.

@robertmryan
Created June 12, 2022 21:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save robertmryan/1127027ca74414a792519eede0fab576 to your computer and use it in GitHub Desktop.
Save robertmryan/1127027ca74414a792519eede0fab576 to your computer and use it in GitHub Desktop.
func benchmark() {
let string = String(repeating: "This is it. ", count: 10000)
let i1 = 19000
let i2 = 19020
// time computation
func tc(computation: (Int, Int) -> Void) {
let startTime = DispatchTime.now()
for i in 0..<100 {
computation(i1 + i, i2 + i)
}
let endTime = DispatchTime.now()
let ns = (endTime.uptimeNanoseconds - startTime.uptimeNanoseconds)
print("Time: \(ns)")
}
tc { (s1, s2) in
let start = string.index(string.startIndex, offsetBy: s1)
let end = string.index(start, offsetBy: s2 - s1)
string[start..<end]
}
// Time: 23,571,282
tc { (s1, s2) in
(string as NSString).substring(with: NSRange(s1..<s2))
}
// Time: 646,777
tc { (s1, s2) in
let start = string.utf16.index(string.startIndex, offsetBy: s1)
let end = string.utf16.index(start, offsetBy: s2 - s1)
string[start..<end]
}
// Time: 31,606
}
@robertmryan
Copy link
Author

robertmryan commented Jun 12, 2022

That having been said, I would

  • use XCTest with its measure command, that repeats it ten times;
  • I would bump the number of iterations from 100 to 100_000 so that we're doing enough to make the results statistically significant;
  • I would add some test on the result just to make sure that the optimized build doesn't optimize the substring out of the code (because you weren't doing anything with the result)

E.g.

import XCTest

class StringTests: XCTestCase {

    let string = String(repeating: "This is it. ", count: 10_000)
    let i1 = 19_000
    let i2 = 19_020
    let iterations = 100_000

    // Not going to include this because it is so slow that repeating it a million times just takes too much time
    //
    // func testStringIndex() {
    //     measure {
    //         for _ in 0 ..< iterations {
    //             let start = string.index(string.startIndex, offsetBy: i1)
    //             let end = string.index(start, offsetBy: i2 - i1)
    //             let result = string[start..<end]
    //             XCTAssertEqual(result.count, i2-i1)
    //         }
    //     }
    // }

    func testNSStringIndex() {
        measure {
            for _ in 0 ..< iterations {
                let result = (string as NSString).substring(with: NSRange(i1..<i2))
                XCTAssertEqual(result.count, i2-i1)
            }
        }
    }

    func testStringUtf16Index() {
        measure {
            for _ in 0 ..< iterations {
                let start = string.utf16.index(string.startIndex, offsetBy: i1)
                let end = string.utf16.index(start, offsetBy: i2 - i1)
                let result = string[start..<end]
                XCTAssertEqual(result.count, i2-i1)
            }
        }
    }
}

That yields:

Test Suite 'All tests' started at 2022-06-12 14:54:29.109
Test Suite 'StringTestsTests.xctest' started at 2022-06-12 14:54:29.111
Test Suite 'StringTests' started at 2022-06-12 14:54:29.111
Test Case '-[StringTestsTests.StringTests testNSStringIndex]' started.
/.../StringTestsTests/StringTestsTests.swift:31: Test Case '-[StringTestsTests.StringTests testNSStringIndex]' measured [Time, seconds] average: 0.283, relative standard deviation: 7.002%, values: [0.302967, 0.326997, 0.290226, 0.274886, 0.267880, 0.263001, 0.259452, 0.269877, 0.293625, 0.280716], performanceMetricID:com.apple.XCTPerformanceMetric_WallClockTime, baselineName: "", baselineAverage: , polarity: prefers smaller, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.100, maxStandardDeviation: 0.100
Test Case '-[StringTestsTests.StringTests testNSStringIndex]' passed (3.184 seconds).
Test Case '-[StringTestsTests.StringTests testStringUtf16Index]' started.
/.../StringTestsTests/StringTestsTests.swift:40: Test Case '-[StringTestsTests.StringTests testStringUtf16Index]' measured [Time, seconds] average: 0.174, relative standard deviation: 4.619%, values: [0.182958, 0.186243, 0.173532, 0.178610, 0.164030, 0.182236, 0.173742, 0.172259, 0.164645, 0.162296], performanceMetricID:com.apple.XCTPerformanceMetric_WallClockTime, baselineName: "", baselineAverage: , polarity: prefers smaller, maxPercentRegression: 10.000%, maxPercentRelativeStandardDeviation: 10.000%, maxRegression: 0.100, maxStandardDeviation: 0.100
Test Case '-[StringTestsTests.StringTests testStringUtf16Index]' passed (1.994 seconds).
Test Suite 'StringTests' passed at 2022-06-12 14:54:34.290.
	 Executed 2 tests, with 0 failures (0 unexpected) in 5.178 (5.179) seconds
Test Suite 'StringTestsTests.xctest' passed at 2022-06-12 14:54:34.290.
	 Executed 2 tests, with 0 failures (0 unexpected) in 5.178 (5.180) seconds
Test Suite 'All tests' passed at 2022-06-12 14:54:34.291.
	 Executed 2 tests, with 0 failures (0 unexpected) in 5.178 (5.181) seconds

I.e.
Screen Shot 2022-06-12 at 2 56 26 PM

Now that result is less compelling (rather than an order of magnitude faster, it is only 37% faster), but that is going to be skewed because we’re benchmarking not only the substring logic, but also the retrieval of the count and performance of the comparison (i.e. the substring logic is not the only thing being benchmarked). But this is a safer test, to make sure that the compiler is not optimizing any code out of the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment