This paper presents a heuristics based approach for capacitor allocation problem incorporating Point Estimate Method (PEM) based analysis to attain optimal capacitor reallocation in radial distribution systems. The proposed probabilistic method is used to achieve optimal capacitor allocation associated with the active/reactive loads forecasted error as well as energy loss reduction by taking into account the cost function coefficients. This problem is formulated as a mix integer non-linear, discrete multi-objective problem which be in need of a robust optimization tool to avoid trapping in local optima. The proposed solution based on modified harmony search (MHS) algorithm employ to search for a set of Pareto optimal value of objective functions which save in memory that called repository. Moreover, a fuzzy clustering method is utilized to control the size memory (repository) for the concerned problem. The proposed approach has been tested on IEEE 9 buses to check feasibility and robustness of the proposed approach.