Bitstream analyzing unit 111 fetches a coded block pattern and a coded quantized DCT coefficient from each block in a bitstream. Entropy decoding unit 112 decodes the coded block pattern into a block pattern and decodes the coded quantized DCT coefficient into pairs of a run length and an effectiveness factor. Dequantization unit 115 generates orthogonal transformation coefficients from the pairs of a run length and an effectiveness factor. Inverse Discrete Cosine Transform (IDCT) unit 116 generates a difference image from the orthogonal transformation coefficients. Decode controlling unit 110 instructs first selecting unit 118 to select constants "0" output from first constant generating unit 117 when the image is a "skipped" block. Image storage unit 120 stores a plurality of reference frame pictures having been decoded. Image restoring unit 119 restores an original block by adding a decoded difference image to a reference block read from the reference frame pictures stored in the image storage unit 120.
