Histogram-based and region-based segmentation approaches have been widely used in image segmentation. Difficulties arise when we use these techniques, such as the selection of a proper threshold value for the histogram-based technique and the over-segmentation followed by the time-consuming merge processing for the region-based technique. To provide efficient algorithms that not only produce better segmentation results but also maintain low computational complexity, a novel top-down region dividing based approach is developed for image segmentation, which combines the advantages of both histogram-based and region-based approaches. Experimental results show that our algorithm can efficiently perform image segmentation without distorting the spatial structure of an image. Furthermore, two potential applications in medical image analysis are presented to show the advantages of using the proposed algorithm.