SilvanForge: A Schedule-Guided Retargetable Compiler for Decision Tree Inference

The proliferation of machine learning together with the rapid evolution of the hardware ecosystem has led to a surge in the demand for model inference on a variety of hardware. Decision tree based models are the most popular models on tabular data. This paper is motivated by the problems encountered when targeting inference of these models to run at peak performance on CPU and GPU targets. Existing solutions are neither portable nor achieve the best possible performance for the specific hardware they target.

This paper describes SilvanForge, a schedule-guided, retargetable compiler for decision tree based models that searches over several optimization choices and automatically generates high-performance inference routines for CPUs and GPUs. SilvanForge has two core components. The first is a scheduling language that encapsulates the optimization space, and techniques to efficiently explore this space. The second is an optimizing retargetable compiler that can generate code for any specified schedule. SilvanForge’s ability to use different data layouts, loop structures and caching strategies enables it to achieve portable performance across a range of targets.

SilvanForge generated code is an order of magnitude faster than XGBoost and about 2-5x faster on average than RAPIDS FIL and Tahoe over several batch sizes. While these systems only target NVIDIA GPUs, SilvanForge achieves competent performance on AMD GPUs as well.