techarthub

A little while ago I was attached to a project set in a loft apartment with an expensive TV as its centerpiece. This TV had to both play a video file and also use it to dynamically light the environment in as realistic a manner as possible. This was all the while having a minimal impact on performance because (as usual) the budget was already tapped. It was a fun exercise, and this is the effect I came up with. This tutorial will teach you how to create a dynamically lit TV screen effect in Unreal Engine 4 using light functions. Versatile and easy on system resources, the effect uses a scrolling ‘movie barcode’ to approximate light emitting from a video display. We’ll be starting from scratch in an entirely new project, so the only thing you’ll need to prepare beforehand is the video file you’d like your TV to display. If you’d like to skip straight to the end, the result of this tutorial is also available for download on my GitHub. Feel free to pick it apart and use it in whatever manner serves you and your project. Project overview This effect will be made entirely within the engine using Blueprint and the Material Editor, and will work in Unreal Engine 4.20 and above (we’ll need Rect Lights which were introduced in 4.20). https://www.youtube.com/watch?v=_vsvy6uVC8g The dynamic light effect is achieved by panning through a ‘movie barcode’ texture in time with the video, and feeding that information into one or more Rect Lights via a Light Function. The barcode image that's used in the above video to light the environment. I'm quite proud of this effect and I'm keen to share it with you, so let’s get started! Adding your video file to the project The first thing we’ll need to do to get your video playing in the Editor is to copy it into your project directory. This has to be done via your file explorer, and not within the engine. Unreal can be a little finicky about the location of media files, so I highly recommend you copy the file into your Content/Movies folder. If it doesn’t exist, create it! The engine’s Media Framework should be able to read any video file that Windows Media Player can also read, but if you run into trouble the official documentation is a good place to start looking for more information. In this example I am using an mp4 file, which is usually a safe bet. Setting up the Media Player Once your video file is copied into the project, jump into the editor and create a new Media > File Media Source asset via the Content Browser. This will act as a reference to your video file so other assets in the Content Browser can read from it. Within the File Media Source asset, click the ‘…’ button and guide it to your video file. It should automatically start within the Movies folder so it shouldn’t be hard to find. Back in the Content Brower, reenter the Create Asset menu and select Media > Media Player. Upon creation it'll ask if you want to create an additional Video output MediaTexture asset - check that box and hit OK. These three media assets are all we’ll need for video playback: A FileMedia Source that will dictate which file to play, a Media Player which acts as a container for a specific instance of that source, and a Media Texture to which the player will output. I named mine TV_FileMediaSource, TV_MediaPlayer, and TV_MediaTexture respectively. Side note: We’re done with media assets now, but at this stage if you’d like to double check that everything is working correctly, open up your player asset and double-click your file source from the list. This will automatically open and start playing the file, and you’ll be able to see that reflected in both the player and the texture. This works in the editor, so you should see your texture changing in the Content Browser. Pretty neat! Creating the TV screen Material Now we have our Media Texture we’ll need to add it to a material so we can apply it to geometry in the world. Use the Content Browser to create a new Material asset. I called mine M_TVScreen. There isn’t anything fancy going on in this one. I am using a fresnel node with an exponent of 0.5 to softly lerp between our Media Texture and black, which is just an easy way to reproduce that darkening effect you might see when not looking at a digital display from its optimal viewing angle. I also set the material's roughness to 0 so when the screen is darker it’ll reflect the environment. Jumping into Blueprint Once again from the Content Browser, select Blueprints > Blueprint Class from the Create Asset menu. What we want is an empty actor that we can place in the world, so when prompted to pick a parent class select ‘Actor’ from the top of the Common Classes menu. Name your new Blueprint (I called mine BP_TV) and open it up in the Blueprint Editor. The first thing our TV needs is a screen, so we're going to select Plane from the add menu to add a Plane Component. It will come in facing upwards, so for starters we'll rotate the plane by 90 degrees in the X axis to get our TV screen sitting vertically. You'll also need to scale it to match the aspect ratio of your video file. If, like mine, your video uses the standard 16:9 aspect ratio, scale your mesh in Y by 0.565 to get the right height. My plane settings. Apply your TV_Screen material, and if you're unsure you've done everything correctly you can double check the scale and rotation of the plane by running your Media Player and having a look in the viewport. Remember to make sure your video isn't playing upside down! If you want your video to play with audio, you’ll also need to assign your Media Player asset to a new Media Sound component. The audio will automatically start playing and sync to the player. The last thing we need to do (for now) in this Blueprint is set up the Media Player to automatically load and play when the game is run. To do this we’ll need a reference to the Media Player itself, so make a new Media Player variable and point it to the player asset. Now we can set up a little bit of logic to make it play. The following short graph will fire off when the game begins, and will first load, then play the Media Player you have referenced. Don’t forget to assign the Media Source reference in the Open Source node to your File Media Source asset. The Movie Barcode A Movie Barcode, popularized by moviebarcode.tumblr, is a generated image that consists of a series of vertical bands of color. Arranged sequentially, each of these bands (usually one pixel wide) represents a frame of a video file. Smoothed (top) and unsmoothed versions of a movie barcode generated from the Indie Game: The Movie trailer on YouTube. You could theoretically use any image editing software to make your barcode, but I highly recommend using Melvyn Laily’s Movie Barcode Generator, which comes with a lot of configurable parameters to generate the image for you. It's such a cool little program. Once you've downloaded the generator the first thing you'll need to do is guide it to both your video file and where you’d like to output the resulting image. Then you'll need to decide on how big you'd like to make your barcode. This is really important, as the width of your image will define the detail and accuracy of your TV lighting - if your video file has a lot of abrupt changes in color or intensity then you’ll need to use a higher resolution to keep things in sync. Luckily for us the texture’s height doesn’t matter, and creating a 4k+ texture that’s only 1-pixel high isn’t likely to break our texture budget. I've decided to make mine 4096, because 8192 seemed excessive. I was sorely tempted, though. These are the image settings I have used. If you want your barcode to be perfectly accurate you might consider making your barcode's width the same as the number of frames in the video, so each frame is represented. The barcode version we’re looking for is Normal (smoothed), although it doesn’t matter too much if your pixel height is 1 since there is nothing to smooth. I highly recommend playing around and checking out the different effects you can create. When you’re happy with your barcode, jump back into Unreal and import it as a texture. Unreal will treat it as an uncompressed B8G8R8A8 texture - but as it's only 1 pixel high the size should be negligible. Side note: Now you've generated your barcode you should be able to get a really good idea of the kind of light and color you’ll see from the final effect. If you’re not seeing much contrast in your barcode, the lighting will be a lot more subtle. If the generator isn't giving you the results you want, you can always non-destructively adjust the image in your Texture Properties Editor. Creating the TV light Material It's almost time to create our light function, but before we do we'll need to quickly create a Material Parameter Collection asset. An MPC provides a place to store common values that any material can reference, and we'll be using it to communicate between our material and TV Blueprint. I called mine MPC_TV, because I am an original thinker. Inside the Material Parameter Collection I’m just going to create a single scalar parameter called Alpha. Our Blueprint is going to write to this value, and the light function is going to read from it to find out how far into the video we are. Alpha will be normalized, so 0 will always be the start of the video, and 1 always at the end. Here is our light function material. I've called mine M_TV_Light. Click here for a higher resolution version! This material will divide the UV coordinates of your plane by a scalar value which you should set to the width of your texture in pixels (if this value is different to your barcodes width, the animated lighting won't sync to your video). This bit of math will result in a very zoomed in view of your barcode where one pixel fills the entire space. From here we are adding the Alpha value from our MPC to the U coordinate which will let us scroll the texture horizontally. The Alpha value now dictates where in the barcode the playhead is sitting. As it goes from 0 to 1, so will the barcode scroll from start to finish. In the material's properties don't forget to change the Domain parameter to Light Function or you may get weird results. This last section of the graph is optional and only important if you want your dynamic TV light to be in color. If you're using a greyscale (or close to greyscale) barcode you can plug your texture sampler straight into Emissive Color and skip to the next section. These three nodes are static parameters that can be toggled on/off within instances of this Material. We’re using them to work around a Light Function limitation that complicates our effect a little bit - Light Functions can only affect a light's Intensity value, not color. To work around this problem we’re going to be splitting the three color channels of our barcode into three different Material Instances, and use three different lights to recombine them to create our final colored lighting effect. To do this we’ll need to create three new instances of M_TV_Light, one for each channel. Use the Switch Parameters to isolate a single channel in each instance. That's all the assets we'll need. We're finally ready to put it all together and create some lights. Adding Rect Light components Back in the BP_TV Blueprint create a Rect Light component for each color channel (unless you're using a greyscale barcode, in which case you'll only need to create one). The color of each of these lights will need to be 100% of their value (either red, green, or blue) to blend correctly. ColorRGBHexRed1,0,0FF0000FFGreen0,1,000FF00FFBlue0,0,10000FFFF Play around with the Intensity, Attenuation Radius, and Source Width/Height to get the best results for your environment. Remember these can be changed later (and at runtime!) These are the settings I landed on for my Rect Lights. Don’t forget to apply the appropriate Light Function Material Instance! Casting shadows You might have noticed I've unchecked Cast Shadows on my Rect Lights. This is because realtime shadows are expensive, and having three shadow casting Rec Lights in the same place is absolutely overkill. If you want your TV screen to cast shadows (and who wouldn't?) I would recommend either: Checking Cast Shadows on just one of your colored Rect Lights. The big drawback here is that depending on the colors in your barcode, this may mean that shadows are sometimes not cast if that color isn't present.Create a fourth Rect Light/Material Instance (with no masking) for shadow casting purposes. This is adding an additional light (with its associated cost). Remember to make the new light's color white. Hooking the Media Player up to your Materials All we need to do now is get our Media Player communicating to the Material Parameter Collection we created. This is achieved with the following logic in your BP_TV Event Graph. I had to squish it together a little to make it fit. Each Tick this code will check to see if your Media Player is playing, and if it is it will then find its normalized position in the timeline and feed that into our MPC so our instances can update their barcode position. Side note: I am firing this code every tick because I’m lazy. In an actual production, for performance reasons I would recommend not using tick if you can at all help it. Even just firing every second frame will half this effect's performance impact. Hit play, and you should now see the result of your finished TV Lighting setup. Nice one! Final thoughts It’s important to remember that this implementation of a dynamic TV light is only going to give you an approximation of the color on screen, and can only support one tone per pixel. As the Movie Barcode Generator will average the color values of each sample frame, it may not always give you the results you’re imagining. For example, if one half of the frame is red and the other is blue, the output is going to be purple. That said, it’s a cheap effect and it looks pretty cool. I hope you find it useful! That’s about it for this tutorial, thanks for following along!

Recent Posts