{"id":16945,"date":"2016-10-17T14:15:02","date_gmt":"2016-10-17T21:15:02","guid":{"rendered":"https:\/\/blogs.technet.microsoft.com\/dataplatforminsider\/?p=16945"},"modified":"2024-01-22T22:50:36","modified_gmt":"2024-01-23T06:50:36","slug":"sql-server-as-a-machine-learning-model-management-system","status":"publish","type":"post","link":"https:\/\/www.microsoft.com\/en-us\/sql-server\/blog\/2016\/10\/17\/sql-server-as-a-machine-learning-model-management-system\/","title":{"rendered":"SQL Server as a Machine Learning Model Management System"},"content":{"rendered":"

This post was authored by Rimma Nehme, Technical Assistant, Data Group<\/em><\/p>\n

Machine Learning Model Management<\/h3>\n

If you are a data scientist, business analyst or a machine learning engineer, you need model management \u2013 a system that manages and orchestrates the entire lifecycle of your learning model. Analytical models must be trained, compared and monitored before deploying into production, requiring many steps to take place in order to operationalize a model\u2019s lifecycle. There isn\u2019t a better tool for that than SQL Server!<\/p>\n

SQL Server as an ML Model Management System<\/h3>\n

In this blog, I will describe how SQL Server can enable you to automate, simplify and accelerate machine learning model management at scale – from build, train, test and deploy all the way to monitor, retrain and redeploy or retire. SQL Server treats models just like data \u2013 storing them as serialized varbinary objects. As a result, it is pretty agnostic to the analytics engines that were used to build models, thus making it a pretty good model management tool for not only R models (because R is now built-in into SQL Server 2016) but for other runtimes as well.<\/p>\n

SELECT * FROM [dbo].[models]<\/p>\n

\"Machine<\/a><\/p>\n

Figure 1: Machine Learning model is just like data inside SQL Server.<\/strong><\/p>\n

SQL Server approach to machine learning model management is an elegant solution. While there are existing tools that provide some capabilities for managing models and deployment, using SQL Server keeps the models \u201cclose\u201d to data, thus leveraging all the capabilities of a Management System for Data to be now nearly seamlessly transferrable to machine learning models (see Figure 2). This can help simplify the process of managing models tremendously resulting in faster delivery and more accurate business insights.<\/p>\n

\"Publishing<\/a><\/p>\n

Figure 2: Pushing machine learning models inside SQL Server 2016 (on the right), you get throughput, parallelism, security, reliability, compliance certifications and manageability, all in one. It\u2019s a big win for data scientists and developers \u2013 you don\u2019t have to build the management layer separately. Furthermore, just like data in databases can be shared across multiple applications, you can now share the predictive models.\u00a0 Models and intelligence become \u201cyet another type of data\u201d, managed by the SQL Server 2016.<\/strong><\/p>\n

Why Machine Learning Model Management?<\/h3>\n

Today there is no easy way to monitor, retrain and redeploy machine learning models in a systematic way. In general, data scientists collect the data they are interested in, prepare and stage the data, apply different machine learning techniques to find a best-of-class model, and continually tweak the parameters of the algorithm to refine the outcomes. Automating and operationalizing this process is difficult. For example, a data scientist must code the model, select parameters and a runtime environment, train the model on batch data, and monitor the process to troubleshoot errors that might occur. This process is repeated iteratively on different parameters and machine learning algorithms, and after comparing the models on accuracy and performance, the model can then be deployed.<\/p>\n

Currently, there is no standard method for comparing, sharing or viewing models created by other data scientists, which results in siloed analytics work. Without a way to view models created by others, data scientists leverage their own private library of machine learning algorithms and datasets for their use cases. As models are built and trained by many data scientists, the same algorithms may be used to build similar models, particularly if a certain set of algorithms is common for a business\u2019s use cases. Over time, models begin to sprawl and duplicate unnecessarily, making it more difficult to establish a centralized library.<\/p>\n

\"Why<\/a><\/p>\n

Figure 3: Why SQL Server 2016 for machine learning model management.<\/strong><\/p>\n

In light of these challenges, there is an opportunity to improve model management.<\/p>\n

Why SQL Server 2016 for ML Model Management?<\/h3>\n

There are many benefits to using SQL Server for model management. Specifically, you can use SQL Server 2016 for the following:<\/p>\n