Created
March 21, 2020 15:52
-
-
Save eschnett/4ffc00f67d62dba8aff0dc944bd137b5 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
```Haskell | |
upload :: Async FilePath -> Async (Either () ()) | |
upload files = | |
asyncly | |
$ maxThreads 10 | |
$ S.concatMap S.fromList | |
|$ serially | |
$ S.mapM finalizeUploads | |
|$ asyncly | |
$ S.chunksOf 1000 FL.toList | |
|$ S.mapM uploadFile | |
|$ S.filterM needUpload | |
|$ files | |
where | |
-- at most about 10 in parallel (filesystem bound) | |
needUpload :: FilePath -> IO Bool | |
needUpload fp = return True | |
-- at most about 10 in parallel (network bound) | |
uploadFile :: FilePath -> IO FilePath | |
uploadFile fp = return fp | |
-- at most one at a time (must be called serially) | |
finalizeUploads :: [FilePath] -> IO [Either () ()] | |
finalizeUploads fps = return $ fmap (\_ -> Right ()) fps | |
``` |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment