video with audio playback

Post your cool example code here.
User avatar
hoek67
Posts: 59
Joined: Thu Mar 28, 2019 1:11 am
Location: Mount Beauty, Victoria, Australia
OS: Windows 10, Linux
IDE: STM32duino via Visual Micro in VS 2017
Core: STM official (I think)
Board: STM32F407VET6
Contact:

Re: video with audio playback

Post by hoek67 » Mon Apr 01, 2019 12:22 am

After my last post... and before bed (was very late) I put the program into test mode where sound is loaded... but not played and frames are played as fast a possible. Weeeeelllllllllll... DMA had a huge difference. Judging by the time it took to play the clip I had looking at least a 3x to 4x increase in potential frame rate. Was playing at 23.98fps before and without delays probably 2x. With DMA it's chewing up at least 300+ frames a second as the 100's frame digit is non-stop ticking over.

I'll grab just the needed files and zip/attach them a bit later with a link to original site and a few hints that delayed me. This should allow anyone to implement SPI with DMA. (He had a branch for F2 and F3 as well off memory)

Only thing I have left is to make it honour SPISettings when I pass it in and it's different to the current mode, the current SPI implementation calls the initialize again to reset the SPISettings. BIG problem is it's also doing it after every transfer... not just when settings change.

ag123
Posts: 239
Joined: Thu Mar 07, 2019 6:15 am
OS: linux
IDE: eclipse, arduino 1.8.5
Core: Roger's
Board: Maple mini, Bluepill

Re: video with audio playback

Post by ag123 » Tue Apr 02, 2019 12:33 pm

psst, upload a video of it in action :D

User avatar
MoDu
Posts: 70
Joined: Tue Mar 05, 2019 1:56 pm
Location: Lisbon
OS: Windows
IDE: Visual Studio
Core: Roger's
Board: Maple mini

Re: video with audio playback

Post by MoDu » Fri Apr 05, 2019 2:03 pm

Hoek, have you, by any chance, looked into hardware codecs? I'm having crazy ideas of streaming a low resolution, low bit-rate video with a modern codec. I've actually found h265 encoders but those are still WAY too big and expensive :p
As for h264 encoders, I could literally stuff a Raspberry Pi just to encode video, but I would love find a small, dedicated IC, instead of Linux machine.

With either modern codec, you could get hours, if not days of video on a Sd Card.

EDIT: Related paper I found.

User avatar
hoek67
Posts: 59
Joined: Thu Mar 28, 2019 1:11 am
Location: Mount Beauty, Victoria, Australia
OS: Windows 10, Linux
IDE: STM32duino via Visual Micro in VS 2017
Core: STM official (I think)
Board: STM32F407VET6
Contact:

Re: video with audio playback

Post by hoek67 » Fri May 03, 2019 4:40 pm

Did a bit of encoding years ago and was a royal pain, even with MB's of memory. Very FP intensive and usually needs to convert color spaces etc.

MJPG maybe if can get hold of an integer only implementation but more than likely will use more resources than what's available. Even MP3 decode only integer uses 32K program space for tables and code and requires additional buffers.

Given a 128 * 128 screen I've had no problems playing back 1,4,8 and 16 BPP with 2 channel 16 bit audio at 22050 in a test which is more than decent for small instructional clips etc. Even better when you consider a micro SD can hold 32GB if needed.

I tried with some simple run length decoding ages ago but found it was taking longer to decode than load uncompressed and just throw straight to the output device. Might gain a bit if you can decode 1 scan line while DMA is transferring the previous one.

Post Reply