In this paper, the implementation of a DSP-based video decoder compliant with the H.264/SVC standard (14496-10 Annex G) is presented. A PC-based decoder implementation has been ported to a commercial DSP. Performance optimizations have been carried out improving the initial version performance about 40% and reaching real time for CIF sequences. Moreover, the performance has been characterized using H.264/SVC sequences with different kinds of scalabilities and different bitrates. This decoder will be the core of a multimedia terminal that will trade off energy against quality of experience.