Crowdsourced live video transcoding based on spark streaming

New generation of video broadcast services, such as Twitch.tv and YouTube live events, attract attention from both industry and academia. As for such multimedia system, live video streams are crowdsourced from amateur users and broadcast to the worldwide audience over the Internet. To support its dynamic and heterogeneous viewers, massive source video streams need to be transcoded into multiple versions per stream. Huge heterogeneous and dynamic video sources come to the transcoding system in real-time and need to be processed on the fly in a scalable, efficient and fault-tolerant way with low latency to provide satisfactory user streaming experience. Hence, we build a crowdsourced live video transcoding system based on Spark Streaming, a distributed stream processing platform. To reduce the transcoding latency and optimize the computation resources utilization, we propose an online data placement and task scheduling algorithm based on classification-based transcoding time complexity prediction.