Created
March 12, 2018 16:36
-
-
Save baldassarreFe/363ed9aedd8bac23775aaa1fdf381bbf to your computer and use it in GitHub Desktop.
Mixed-Scale Dense Network
@wohe157 It's indeed not the same as in the paper, since it's missing a crucial component in any network, which would be a nonlinearity! In the paper they briefly mention using ReLU, so if you add that, it would probably perform much better.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@EelcoHoogendoorn Indeed this example seems to show a comparison of the evaluation time for the MSDNet and a conventional CNN and I agree with you that this suggests that a conventional CNN (with a similar number of parameters) is faster, but would usually need (a lot!) more parameters than the MSDNet.
However, I don't really agree with your question about efficiency. I have tried both this example and the code from D. M. Pelt on a database of +/-1500 images of size 1x256x256 for denoising. The same network (w=1 and d=20) took 5min/epoch with this PyTorch implementation, but only 50sec/epoch with the original code.
Moreover, I'm limited to batch sizes of 8 images with this PyTorch implementation because of the GPU memory usage (6GB), while the original implementation doesn't even use 100MB and only uses my GPU for 55%. Therefore I believe that the original code can be even more optimized to perform even faster.
Lastly, I noticed that both network converge at about the same speed (in epochs) to a similar loss (at an MSE of 0.00018 in my case), but when the PyTorch network stops reducing the loss, the original network goes on to 0.00010 and better. Because of this, I'm not even sure if this PyTorch implementation is the same as the one from the paper.