An apparatus for restoring an image and a restoring method thereof are provided to produce an output image by extracting a channel having the highest definition and restoring the channel through a first filter. An apparatus for restoring an image comprises the followings: an image input unit(310) extracting a predetermined channel from image data; an image recovery unit(340) restoring the extracted channel by using a first filter(330); a high frequency attribute production unit(350) extracting a high frequency attribute value; and a brightness correction unit(370) amending a luminance channel by computing the restored channel, a high frequency attribute factor and the luminance channel.