Multi-task Collaborative Intelligence

In Collaborative Intelligence (CI), a Deep Neural Network (DNN) model is split into two parts. The front-end of the model, composed of the initial layers of the model, is deployed on an edge device, while the second part, comprising the rest of the model, is hosted in the cloud. The data volume in the deep feature tensors obtained from the intermediate layers of the model can be inflated compared to the size of the DNN's input. This is especially evident with multi-stream models where multiple feature tensors are obtained at the split point. To efficiently use the bandwidth available for transferring deep features to the cloud, data reduction and compression methods are applied to the features at the split point. In this thesis, bit allocation for CI with an emphasis on multi-task systems is studied. The goal is to provide an efficient way to utilize the bandwidth such that the accuracy loss in target tasks caused by the compression of transferred features is minimized. For multi-stream CI systems, closed-form bit allocation solutions for single-task systems and scalarized multi-task systems are obtained by modeling task distortion as a function of rate using convex surfaces. In addition, analytical characterization of the full Pareto set for 2-stream multi-task systems, as well as the bounds on the Pareto set for 3-stream 2-task systems are provided. For single-stream models, mutual information between a feature and a model's desired output is used as a measure of the feature's importance for that task. Then, importance ranking is used to perform bit allocation among the features to be transferred to the cloud. The proposed bit allocation method based on importance ranking is used to obtain an information-theoretic privacy model called "Privacy fan" to achieve privacy-friendly inference in CI systems. We show through extensive experiments that our proposed methods improve the performance of CI systems qualitatively and quantitatively for a broad range of applications and models.