<p>The disclosed subject matter provides techniques for inter-layer prediction using difference mode or pixel mode. In difference mode, inter-layer prediction is used to predict at least one sample of an enhancement layer from at least one (upsampled) sample of a reconstructed base layer picture. In pixel mode, no reconstructed base layer samples are used for reconstruction of the enhancement layer sample. A flag that can be part of a coding unit header in the enhancement layer can be used to distinguish between pixel mode and difference mode.</p>