Intelligent video surveillance system has emerged as a very important research topic in the computer vision field in the recent years. It is well suited for a broad range of applications such as to monitor activities at traffic intersections for detecting congestions and predict the traffic flow. Object classification in the field of video surveillance is a key component of smart surveillance software. Two robust methodology and algorithms adopted for people and object classification for automated surveillance systems is proposed in this paper. First method uses background subtraction model for detecting the object motion. The background subtraction and image segmentation based on morphological transformation for tracking and object classification on highways is proposed. This algorithm uses erosion followed by dilation on various frames. Proposed algorithm in first method, segments the image by preserving important edges which improves the adaptive background mixture model and makes the system learn faster and more accurately. The system used in second method adopts the object detection method without background subtraction because of the static object detection. Segmentation is done by the bounding box registration technique. Then the classification is done with the multiclass SVM using the edge histogram as features. The edge histograms are calculated for various bin values in different environment. The result obtained demonstrates the effectiveness of the proposed approach.