Inefficiencies in waste management contribute to the increasing amount of pollution in society, leading to public demands for better waste management and classification. Waste sorting is the beginning of the waste recycling process, which can help reduce the amount of waste in the environment. However, coupled with a lack of awareness of waste sorting due to minimal public education about waste management, the waste sorting system is still carried out manually using human power. Therefore, it is necessary to have a waste classification system to encourage people to manage their waste well. This research aims to design a tool to detect types of waste and classify them into three categories; metal, paper and plastic waste. The system can recognize the shape of trash images using a deep learning method developed using Faster R-CNN with ResNet-50 and ResNet-101 as the network architecture. This research began by collecting 250 datasets of metal, paper and plastic waste which were used as training data and test data in the testing process. The training data for the training process are 80, 120 and 200 datasets respectively. Test data for each training experiment, 30 and 50. Where 20 datasets in 50 test data are taken from the dataset for the training process. In each training process, the number of steps is carried out up to 3000, 4000 and 5000 steps, each of which has a total loss parameter. Based on the test results applied to the Faster R-CNN ResNet-50 and ResNet-101 methods, it produces an average F1 Score of 63% and 77% respectively. The best F1 Score is Faster R-CNN ResNet-101.