Cross platform video playback in unreal engine [en]

In this guide an explanation will be given about the unreal engine media framework and different video players, codecs and more that are supported by it. If a step by step guide how to implement a cross platform video player with BluePrints is preferred press: TL;DR

Unreal mediaframework

Video playback in Unreal engine can be difficult with all the different kinds of videocodecs and different internal video players Unreal supports. By looking into the documentation MediaFramework techreference a group of video players can be identified (mobile players have been omitted).

  • AvfMedia
  • WmfMedia

This list also includes players for Android, PS4, Xbox One what it however completely omits is the ImgMedia player. Where AvfMedia only works on Mac and iOS and WmfMedia only works on Windows. The ImgMedia player works on Linux, Mac and Windows; even more so ImgMedia is the ONLY available video player for Linux within the unreal engine.

So what kind of videocodes does ImgMedia support; the answer is: NONE. ImgMedia reads sequences of jpg, png, bmp or exr images from a pre-defined folder at a framerate which manually has to be entered. Suffice to say video player support on Linux from unreal is seriously lacking. Compared to other video players in the unreal engine the ImgMedia player works a little different and requires other assets to make it functional. It is also important to note that the ImgMedia player by itself does not support audio, audio has to be played from another source alongside the videoplayer; additional steps may be required to keep the video and audio in sync.
Image Sequence Example

Example of creating a ImgMediaSource asset in unreal and assigning the appropriate variables

The ImgMedia player requires a number of assets to properly operate, All these assets can be easily created and configured from within the unreal editor.

  • Media Player
  • Media Texture
  • ImgMediaSource
  • Sound Wave
  • Sound Concurrency
  • Widget Blueprint

Explaining asset functions

The Media Player will be responsible for loading source material and starting and stopping playback. The Media Player will be linked to the Media Texture, this allows the output of a Media Player to be mapped to a Texture which can be assigned to all kinds of materials within unreal. The source the Media Player will be loading is of type ImgMediaSource; the asset ImgMediaSource is used to configure the used video player(ImgMedia), the framerate and the location of the image sequence. The Sound Wave is a 16bit PCM WAV file of the audio that is to be played alongside the video, The Sound Wave will be need to assigned an Sound Concurrency to prevent it from playing multiple times simultaneously. The last important part is a Widget Blueprint this will be responsible for creating the logic that can play the video and audio and make sure it appears in the viewport.

Creating cross platform player in BluePrints

Make sure you have the unreal editor and the project you want to have this video player in open. The first step is to create all the required assets. Within the content browser press the right mouse button and from within the media submenu select “Media Player”, Within the appearing dialog select be sure to select “Video output Media Texture Asset” and click OK. With this step the Media Player & Media Texture will be created and properly configured.
asset media and media texture

  1. Create an ImgMediaSource by pressing “Add New” in the content browser and selecting ImgMediaSource under the Media sub menu
  2. If you do not have an image sequence already learn how to convert a video file into one
  3. Please the folder with the image sequences files into the project folder and make sure it is within the Content/Movies/ directory
  4. Unreal editor will ask to import newly found files it is extremely important to click DON’T IMPORT
  5. Open the created ImgMediaSource asset and set the Sequence Path parameter to the image sequence folder, for example Content/Movies/MyImageSequence
  6. Still within the ImgMediaSource asset set the Frames per second override to the exact framerate of the original video, be as precise as possible as this will limit audio and video desync issues
  7. Lastly in the ImgMediaSource asset make sure that the Linux, Mac & Windows video player overrides parameters are set to ImgMedia
  8. Now it is time to import the 16bit pcm wav file which represents our video audio, if you do not have such a wav file ready do not worry: Simply open the original video file in audacity and under ‘file’ in the top menu select the export and export as wav option.
  9. Within the unreal editor have the content browser open and select import, select the wav file and a new Sound Wave asset will be created.
  10. In the content browser right press or select ‘Add New’ and under sounds select the Sound Concurrency asset.
  11. Open this new asset and set the Max Count parameter to ‘1’ and the Resolution Rule to ‘Stop Oldest’.
  12. Create the final asset within the content browser under the ‘User interface’ sub menu and select ‘Widget Blueprint’
  13. Open the Widget Blueprint and make sure you are on the designer view(top right).
  14. Add an Image component by selecting it from the Palette(top left).
  15. Size and Anchor the Image as to how you want to video to appear in the viewport.
  16. Select the graph tab in the top right corner and add all the required Variables(top left).variables required by Widget Blueprint
  17. Player is of variable type ‘Media Player’, playerTexture of type ‘Media Texture’, video of type ‘Media Source’, music of type ‘Sound Wave’ and musicConcurrency of tope ‘Sound Concurrency’. In my example the Image is called videoimagecanvas which will later be visible in the blueprint design.
  18. The final step is to copy this design into the graph editor, the video and audio will start playing whenever ‘Set Brush’ is called.
  19. Optionally an event can be bound to the OnEndReached multicastDelegate to perform actions once the video finishes.

Create an image sequence from video files

Geef een reactie

Het e-mailadres wordt niet gepubliceerd. Vereiste velden zijn gemarkeerd met *

*