A method and apparatus are described for tracking an object image in an image sequence (101) in which a template window associated with the object image is established from a first image in the image sequence and an edge gradient direction extracted. A search window associated with the object image is established from a second image in the image sequence and a second edge gradient direction associated with the search window extracted. The first and second edge gradient directions are weighted and correlated to provide a coordinate offset between relative positions of the object image in the search and template windows to track the object image in the image sequence.