Let your AI model as close as possible data sources

Source: Redislabs Author: Pieter Cailliau, LucaAntiga translation: Kevin (Public number: Middleware little brother)

Brief introduction

Today we released a preview version of RedisAI, pre-integrated [tensor] werk components. RedisAI is a service tensors tasks and can perform tasks Redis depth learning modules. In this blog, we will introduce the function of this new module, and explain why we believe it will subvert machine learning (ML), the depth of learning solutions (DL) is. Produced RedisAI are two reasons: First, the high cost of migrating data to the host performs AI model and experience a great influence on real-time; secondly, Serving AI model has always been a challenge in the field of DevOps. Our purpose of constructing RedisAI is to allow the user can not move Redis multi-node data of the situation, but also a good service, update and integrate their own models.


Location data is important

In order to demonstrate operation of the machine learning, deep learning the importance of the position data model, we give an example of a robot chat. Chatbot usually recurrent neural network model (RNN), to address one (seq2seq) User Q & A scene. More advanced model uses two input vectors and two output vectors, and the digital intermediate state vector in a manner to preserve the context of the conversation. Model uses the last message as the user input, dialogue among representatives of state history, and its output is a response to user messages and a new intermediate state.

In order to support interactive user-defined, this intermediate state must be saved in a database, so Redis + RedisAI is a very good option, there will be the traditional program and RedisAI program to make a comparison.


1, the traditional program

To construct a bot or other application programs to use Flask integrated Spark. Upon receiving the message the user session, the server needs to obtain from the intermediate state Redis. After the Redis because there is no native Tensor data type can be used, it is necessary to deserialize and Recurrent Neural Network (RNN) in operation, ensure real intermediate state may be saved to Redis after re-serialized.


Taking into account RNN time complexity, data serialization / de-serialization overhead and a huge network overhead on the CPU, we need a better solution to ensure the user experience.


2, RedisAI program in RedisAI, we provide a data type called Tensor, just use a series of simple commands can operate on Tensor vector mainstream client. Meanwhile, we also provide two additional types of data to the runtime characteristics of the model: Models and Scripts.


Models and operation command device (CPU or GPU) and a rear end from the parameter definitions. RedisAI built a mainstream machine learning framework, such as TensorFlow, Pytorch, etc., and will soon be able to support ONNX Runtime Framework, while increasing support for the traditional machine learning model. However, it is great, its back-end Model command is not aware of:

AI.MODELRUN model_key INPUTS input_key1 …  OUTPUTS output_key1 ..

This allows the user to select the back-end coupling off (usually determined by data experts) and application services solution, the replacement model only need to set a new key can be very simple. RedisAI management model process all requests in the queue, and executes in a separate thread so that the security response Redis still other normal requests. Scripts command can be executed on a CPU or GPU, and allows the user to operate Tensors TorchScript vector, TorchScript class Python is a vector operable Tensors custom language. This may help users to preprocess the data prior to execution model can also be used in processing the results of the scenarios, for example to improve performance by integrating different models.

We plan for the future DAG supports batch execute commands through the command, which allows users to perform multiple RedisAI batch command in an atomic operation. Such as running different instances of a model on a different device, do the average forecast of the results by the script. Use DAG command calculation can be performed in parallel, and then perform the polymerization operation. If you need the full amount and deeper list of features, visit The new architecture can be simplified as:



Service can be simpler model

In a production environment, use Jupyter notebooks to write code and deploy applications are not the optimal solution in Flask. How to determine the user’s own resources is the best of it? If the user host is down, the intermediate state of the bot what happens? User-create the wheel may be repeated to achieve the existing Redis functions to solve the problem. In addition, due to the complexity of the combination regimen often than expected, stubbornly adhere to the original solution will be very challenging. RedisAI by Redis enterprise-class data storage solutions, supporting data type Tensors, Models and Scripts such as deep learning needs, to achieve a good depth of integration Redis and AI models. If you need to expand the computing power of the model, simply for Redis cluster expansion can, so users can add as many models in a production environment, thereby reducing infrastructure costs and total cost. Finally, RedisAI well adapted to the existing ecological Redis allows users to perform scripts to pre-and post-processing of user data, the data structure can be used RedisGear do the right conversion, can be used to keep data up to date RedisGraph state.


Conclusions and follow-up plans

1, in the short term, we want to use RedisAI in the case supports three major back-end (Tensorflow, Pytorch and ONNX Runtime), and stabilized as soon as possible and reaches a steady state. 2, we hope that these can be dynamically loaded the back end, users can customize the designated back-end loaded. For example, this would allow the user to use edges with Tensorflow Lite treated cases. 3, plans to achieve the automatic scheduling function can be achieved automatically merge different queues in the same model. 4, run statistical models RedisAI will be used to measure the performance of the model.

5, to complete the DAG characteristics explained hereinbefore.


More high-quality IT middleware technology / original / translation of the article / data / dry goods, please pay attention to “middleware little brother” public number!

Leave a Reply