SHAMANN: Shared Memory Augmented Neural Networks

Abstract

Current state-of-the-art methods for semantic segmentation use deep neural networks to learn the segmentation mask from the input image signal as an image-to-image mapping. While these methods effectively exploit global image context, the learning and computational complexities are high. We propose shared memory augmented neural network actors as a dynamically scalable alternative. Based on a decomposition of the image into a sequence of local patches, we train such actors to sequentially segment each patch. To further increase the robustness and better capture shape priors, an external memory module is shared between different actors, providing an implicit mechanism for image information exchange. Finally, the patch-wise predictions are aggregated to a complete segmentation mask. We demonstrate the benefits of the new paradigm on a challenging lung segmentation problem based on X-Ray images, as well as on two synthetic tasks based on MNIST. On the X-Ray data, our method achieves state-of-the-art accuracy with a significantly reduced model size, 3–5 times compared to reference methods. In addition, we reduce the number of failure cases by at least half.