Sebastian Buschjäger has published the software "Fastinference". It is a model optimizer and model compiler that generates the optimal implementation for a model and a hardware architecture. It supports classical machine learning methods like decision trees and random forests as well as modern deep learning architectures.

https://github.com/sbuschjaeger/fastinference