cv-nested-unet-dense-skip-connections

Overview

Vanilla U-Net concatenates only one encoder feature into each decoder stage. UNet++ instead upsamples features from each decoder stage multiple times, concatenating them into all shallower decoder levels. This dense pattern gives every decoder output access to multi-scale features at the original resolution, improving segmentation of small and fine-grained structures.

Quick Start

from keras.layers import Conv2DTranspose, concatenate

# Deepest decoder (level 4)
deconv4 = Conv2DTranspose(filters*16, (3,3), strides=(2,2), padding='same')(convm)
# Upsample deconv4 to shallower levels for cross-depth skips
deconv4_up1 = Conv2DTranspose(filters*16, (3,3), strides=(2,2), padding='same')(deconv4)
deconv4_up2 = Conv2DTranspose(filters*16, (3,3), strides=(2,2), padding='same')(deconv4_up1)
deconv4_up3 = Conv2DTranspose(filters*16, (3,3), strides=(2,2), padding='same')(deconv4_up2)

deconv3 = Conv2DTranspose(filters*8, (3,3), strides=(2,2), padding='same')(uconv4)
deconv3_up1 = Conv2DTranspose(filters*8, (3,3), strides=(2,2), padding='same')(deconv3)
deconv3_up2 = Conv2DTranspose(filters*8, (3,3), strides=(2,2), padding='same')(deconv3_up1)

# Shallower decoders concatenate all upstream upsampled features
uconv3 = concatenate([deconv3, deconv4_up1, conv3])
uconv2 = concatenate([deconv2, deconv3_up1, deconv4_up2, conv2])
uconv1 = concatenate([deconv1, deconv2_up1, deconv3_up2, deconv4_up3, conv1])

Workflow

1. Build encoder and bottleneck as usual
2. For each decoder level, generate the primary upsampled feature
3. Additionally upsample that feature K more times to match shallower resolutions
4. At each shallower decoder level, concatenate: its own upsampled feature + all upsampled-extras from deeper levels + encoder skip
5. Apply conv blocks on the concatenated tensor as in standard U-Net

Key Decisions

• vs. plain U-Net: More parameters and memory, but richer multi-scale context at every resolution.
• Feature count: Deeper features get upsampled multiple times — keep channel counts modest to avoid OOM.
• Deep supervision: Optionally attach segmentation heads at each decoder level for auxiliary losses.

References

• Nested Unet with EfficientNet Encoder