Neural network based models are paramount in machine learning, but they are black boxes that are difficult to interpret. In this project, we propose a benchmarking study to compare the current existing techniques that extract knowledge from these models. In this way we hope to identify current potential and limitations and use these to develop new knowledge extraction methods.