Skip to content

Instantly share code, notes, and snippets.

@eschnett
Created March 21, 2020 15:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save eschnett/4ffc00f67d62dba8aff0dc944bd137b5 to your computer and use it in GitHub Desktop.
Save eschnett/4ffc00f67d62dba8aff0dc944bd137b5 to your computer and use it in GitHub Desktop.
```Haskell
upload :: Async FilePath -> Async (Either () ())
upload files =
asyncly
$ maxThreads 10
$ S.concatMap S.fromList
|$ serially
$ S.mapM finalizeUploads
|$ asyncly
$ S.chunksOf 1000 FL.toList
|$ S.mapM uploadFile
|$ S.filterM needUpload
|$ files
where
-- at most about 10 in parallel (filesystem bound)
needUpload :: FilePath -> IO Bool
needUpload fp = return True
-- at most about 10 in parallel (network bound)
uploadFile :: FilePath -> IO FilePath
uploadFile fp = return fp
-- at most one at a time (must be called serially)
finalizeUploads :: [FilePath] -> IO [Either () ()]
finalizeUploads fps = return $ fmap (\_ -> Right ()) fps
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment