Skip to content

Instantly share code, notes, and snippets.

@beefon
Last active February 3, 2020 08:21
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save beefon/0d5dbd6bffc0775bfd0fbed0785daecd to your computer and use it in GitHub Desktop.
Save beefon/0d5dbd6bffc0775bfd0fbed0785daecd to your computer and use it in GitHub Desktop.
NSOperationQueue + NSURLSession = no network activity
import Dispatch
import Foundation
// If you run this playground, you will never see `Request feedback` message in console.
let queue = OperationQueue()
// but if you uncomment the line below, it suddenly starts working great
// queue.maxConcurrentOperationCount = 63
// but if you set maxConcurrentOperationCount to 64, it stops working again
for _ in 0...999 {
queue.addOperation {
let group = DispatchGroup()
var request = URLRequest(url: URL(string: "http://example.com")!)
request.timeoutInterval = 10
let task = URLSession.shared.dataTask(with: request) { _,_,_ in
print("\(Date()) Request feedback")
group.leave()
}
print("\(Date()) Starting request")
task.resume()
group.enter()
if group.wait(wallTimeout: .now() + 30) == .timedOut {
task.cancel()
print("\(Date()) Request timeout")
}
}
}
queue.waitUntilAllOperationsAreFinished()
print("\(Date()) Queue is empty")
@vakhidbetrakhmadov
Copy link

vakhidbetrakhmadov commented Jan 31, 2020

If you do PlaygroundPage.current.needsIndefiniteExecution = true and wait long enough you will actually see all Request feedback at last when first 960 operations are done running (in this case when they timeout).

@vakhidbetrakhmadov
Copy link

vakhidbetrakhmadov commented Jan 31, 2020

It looks like OperationQueue and URLSessionDataTask here pull threads from the same threads-pool (which is apparently limited to 64).
Also looks like compiler flattens out the nested thread dispatching, for example if you run this with 0...3 range you will get:

2020-01-31 21:02:56 +0000 Starting request <NSThread: 0x6000027c4540>{number = 3, name = (null)}
2020-01-31 21:02:56 +0000 Starting request <NSThread: 0x6000027f5340>{number = 7, name = (null)}
2020-01-31 21:02:56 +0000 Starting request <NSThread: 0x6000027fcb00>{number = 4, name = (null)}
2020-01-31 21:02:56 +0000 Request feedback <NSThread: 0x6000027856c0>{number = 8, name = (null)}
2020-01-31 21:02:56 +0000 Request feedback <NSThread: 0x6000027856c0>{number = 8, name = (null)}
2020-01-31 21:02:56 +0000 Request feedback <NSThread: 0x6000027c4540>{number = 3, name = (null)}
2020-01-31 21:02:56 +0000 Queue is empty <NSThread: 0x6000027cac00>{number = 1, name = main}

The pattern will be the same for 0...N

@vakhidbetrakhmadov
Copy link

vakhidbetrakhmadov commented Jan 31, 2020

So, because of all operations being run first, and both OperationQueue and URLSessionDataTask using the same threads-pool (limited to 64), not setting maxConcurrentOperationCount (that is leaving its default value -1 that probably translates to using the entire pool) means that no URLSessionDataTask has chance to run until first 960 operations timeout.
While limiting maxConcurrentOperationCount to 63 means that there will always be at least 1 thread in threads-pull for URLSessionDataTask.

@vakhidbetrakhmadov
Copy link

vakhidbetrakhmadov commented Feb 1, 2020

Also looks like compiler flattens out the nested thread dispatching

Or even more probable than that, is that URLSessionDataTasks are being added to OperationQueue.current with lower priority, but could not confirm. OperationQueue.current === queue inside URLSessionDataTask's completion block yields false (may be completion queue is different)

@beefon
Copy link
Author

beefon commented Feb 3, 2020

Thanks Vakhid, I forgot to attach a link to the forum where Apple stuff has commented on this issue.

https://forums.swift.org/t/operationqueue-urlsession-no-network-activity-oob/33247

You were absolutely right, there is a shared thread pool in GCD for all queues, and that explains a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment