In this paper, the implementation of a DSP based video decoder compliant with the H.264/SVC standard (14496-10 Annex G) is presented. A PC-based decoder implementation has been ported to a commercial DSP. Performance optimizations have been carried out improving the initial version performance about 32% and reaching real time for CIF sequences. Moreover, conformance tests have been done using different H.264/SVC streams. This decoder will be the core of a multimedia terminal that will trade off energy against quality of experience.