The embodiments of the present invention provide a method and a device for encoding and decoding video, pertinently select an effective transform matrix to perform transformation according to the characteristic of each residual block, and therefore improve the encoding efficiency. The technical solution provided by the embodiments of the present invention is: generating the prediction residual according to the input video data; selecting a group of optimal transform matrixes from multiple transform matrixes according to the intra-frame prediction mode and the rate-distortion rule to transform encoding the prediction residual, and obtaining a transform result; generating encoding code stream according to the transform result and the index information of the selected transform matrixes.