Disclosed is a method for segmenting a plurality of objects from a two-dimensional (2D) video captured through a depth camera and an RGB/G camera. The method comprises detecting camera motion in each 2D frame of the plurality of 2D frames from the 2D video and generate a first set of 2D frames without any camera motion. The method further comprises generating a plurality of cloud points for the first set of 2D frames corresponding to each pixel associated a 2D frames in the first set of 2D frames. The method further comprises generating a 3D grid comprising a plurality of voxels. The method further comprises determining valid voxels and an invalid voxels in the 3D grid. Further, a 3D connected component labeling technique is applied on to the set of valid voxels to segment the plurality of objects in the 2D video.
