Shaders are bits of code that communicate directly with the graphics card in order to create visuals. Don’t be afraid of the “code” part, there are a lot that are ready to use! From version 1.10, HeavyM even has its own library with about a hundred of them. 

In this tutorial, we’ll explain how to use the shaders library in HeavyM and we’ll give you resources if you want to find other shaders or even create them yourself!

 

Using shaders in HeavyM

Adding a shader in a player

In order to add a shader in HeavyM, you first have to add a player by drag & dropping the corresponding icon from the toolbar to your work area. A rectangular shape should appear in your work area and a parameters panel on the left.

By default, the player mode is set on media (image, GIF or video). To set the player in shader mode, you have to click on the corresponding icon in the settings section of your player.

 

The panel opens up when you click on a player and you can keep it always open by clicking on the pin icon.

You can then open your shaders library with the + icon. You can see that HeavyM provides you with about a hundred ready-to-use shaders. Simply select the one you want and click on “Open”. (You can also double-click on a shader directly.)

 

HeavyM shaders library
Hover over the thumbnails to preview the shaders

Then, from the players panel, you can modify the settings of your shader as you wish! 

Note: don’t forget to hit play and to choose a display setting for your player in order to see it in the projection! To know more about the general operation of players, read our tutorial on the subject.

 

HeavyM shader uniform values
BitStreamer.fs has a lot of uniform values!

The parameters of a shader are called uniform values and they differ with the shader you choose. You’ll find a great variety of settings, like color, speed or position variables, etc. Some shaders also use a source as input: you can use an image, a video, a webcam stream or even a syphon/spout stream (only in HeavyM Live for this last option). In this case, the shader is applied on the input source and you can see the effects live!

 

Managing the shaders library

As you’ve seen previously, you already have a lot of ready-to-use shaders. Those have been provided by the creators of the ISF format: VidVox. In addition to these, the library also allows you to store your creations or finds. It has been introduced in version 1.10 and can contain ISF shaders, as well as “classic” GLSL shaders whose variables have been adapted for HeavyM (those are just formats, more info on how to get them later in this tutorial.)

Your personal collection is located in the second tab of the library, titled “My Shaders”. To add a shader, simply click on  and browse the files on your computer. The files have to be .fs for ISF or .frag for classic GLSL.

When you add a shader, it is copied in the HeavyM library folder on your computerIt is then ready to be used in HeavyM like the other shaders. You can manage your collection as you wish, delete some shaders, add an image or GIF that will serve as a thumbnail for your shader so that it’s easier to navigate the list.

Technical note 1: some ISF shaders also contain a .vs that will be copied automatically if it’s located in the same file and bears the same name.

Technical note 2: HeavyM doesn’t support shaders with sound input for now. You can also encounter compatibility issues with some ISF v1 shaders or ones using a persistent buffer and multipass, especially on Mac if your graphics card is not dedicated. Don’t hesitate to update your graphics drivers, it can solve some problems. The ISF website also allows you to convert ISF v1 shaders to ISF v2.

Technical note 3: HeavyM can detect and display some types of errors in shaders, they are displayed in a pop-up and give you indications of the problems in the code.

 

Where do I find compatible shaders?

A lot of sources will help you find new shaders to use in HeavyM without having to create them yourself. Indeed, there’s a really active community of artists who share their work. Here are some resources and tips on how to find and import shaders in HeavyM: 

First off, know that ISF shaders, which tend to become more popular, are very easy to import: if you find a .fs file, you can directly add it and use it in HeavyM as described before. On the other hand, a common .frag, if it hasn’t created specifically for HeavyM, won’t be directly compatible with the software. You can adapt it yourself by hand (the method is described in the last part of this tutorial), or more simply use a .frag → .fs converter. (Warning, if you use a HeavyM version older than 1.10, ISF are not supported, therefore, you need to refer to the “manual” method.)

Here are some useful links:

interactiveshaderformat.com homepage
interactiveshaderformat.com homepage

Note: depending on how you intend to use it, don’t forget to check the license chosen by the creator! It usually is stated at the beginning of the shader’s code.

Of course, you can also create your own shaders! If you’re interested, the next part of this tutorial will give you more info on the technical aspects of shaders and how to get started with content creation.

 

 

Understanding shaders and creating your own content

By the way, what’s a shader exactly?

A shader is a program that communicates directly with your graphics card to render an image. You can find a lot of tutorials and explanations about shaders on the web. 

In HeavyM, we use fragment shaders. To keep it simple, let’s say it’s a little program that renders the color of each pixel of the image you want to display. To make those programs work, you must have a graphics card that supports OpenGL 2.0.

Regarding the difference between the formats, ISF shaders are merely GLSL fragment shaders that have been slightly modified in order to be understood more easily by interactive applications and thus create a standard. Basically, a part of the code of an ISF describes how to interpret it, that’s why you don’t need to convert it to use it in HeavyM (more details on this by the creators of the format here).

 

Creating your own ISF shader

If you want to create your own shaders, we advise you to write in ISF format directly, so that they are compatible with the most apps. However, if you want to create one specifically for an older version of HeavyM than 1.10 (that only supports .frag), you’ll have to refer to the next part, to understand how to declare uniform values in this case.

We’re not going to teach you how to code here, but here is some advice on how to get started:

You can start by checking out the shaders on the ISF website and try to edit them to see what happens. For instance, you can begin with Solid Color, a shader that simply displays a color. Click on the “FS” on the upper-left corner of the page to see the code and try to change things, you’ll see your modifications applied on the preview live. Then you can go see Linear Gradient. You can try to edit it too and evolve towards more and more complex shaders. 

Then, you’ll be able to start creating your own shaders by going to the homepage and clicking on “create your own” and thus get access to a basic ISF template. Note: there are other editors with a live preview, online or as apps, like ISF Editor (only on Mac for now).

There, now if you want more technical details on how the code of an ISF works, you can also read the next part about .frag in HeavyM. The only difference is the part concerning uniform values, but the principle behind the code stays the same!

 

Creating or converting a .frag shader compatible with HeavyM

As we’ve been saying in this tutorial, ISF compatibility has been integrated in HeavyM 1.10. If you use an older version, you can only use .frag shader and you have to “convert” them in order to make them compatible in HeavyM. You can start by testing the shaders the team has adapted for you, that you can use in HeavyM directly:

DOWNLOAD SHADERS

To use other shaders, you don’t need to understand everything but it’s good to get the basics. To recall specifications stated in a previous part, HeavyM only supports fragment shaders and to make them work, you need at least a graphics card with OpenGL 2.0.

 

Output color

void main( void )
{
      gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
}

This is a short example that puts the red color on each pixel. You can already try this code in HeavyM. Copy/Paste this code in a text editor and save it with the .frag extension. Then, load it into a HeavyM player as described before.

Now, if you click on Play and Activate on Top in your players panel, your player will display a red image.

 

 

Explanation: each pixel is rendered by this code. gl_fragColor is the name given to the color we get. We assign it an RGBA vector with some values between 0 and 1 (in our example, we have red, which is : R = 1, G = 0, B = 0, A = 1).

Now you can try to change those values in order to get other colors.

 

Input position

We are are going to see how we can have different colors on different pixels. We can modify the color of the pixel depending on its position. This position is called gl_FragCoord, and if we want to get the X or Y position of the pixel, we have to use gl_FragCoord.x and gl_FragCoord.y.

A simple effect is to vary the RGB values based on the position of the pixel, in order to get a gradient:

void main( void )
{
      float red = gl_FragCoord.y / 1000.0;
      float blue = gl_FragCoord.x / 1000.0;
      gl_FragColor = vec4(red, 0.0, blue, 1.0);
}

In this code, we declare a number red, equal to the Y position of the pixel divided by 1000. We do the same for the blue, equal to the X position divided by 1000.
Therefore, the pixel located at (0,0), on the top left side, gets red and blue values equal to 0, so it will be black, whereas the pixel located on the bottom right gets red and blue values equal to 1, so it will be purple (red + blue).

Here, we divide by 1000 because we want some values between 0 and 1 (it depends on the resolution of your shader).

 

You can then apply a lot of mathematical transformations to your colors to get some visual effects. To understand this part, we advise you to read some tutorials dedicated to fragment shaders on the web.

 

Uniform Values

However, all of this only allows you to display a fixed image. In a video mapping, it’s nice to have an animated image, so we have to take the time into account. In order to do this, we can send some values to a shader at each frame: those values are called Uniform values.

In HeavyM, we have decided to send 6 values:

  • HM_x, HM_y, HM_z and HM_speed : just integer numbers between 0 and 100 that you can modify from the interface.
  • HM_resolution: a vector (x,y) corresponding to your shader’s resolution.
  • HM_time: a float corresponding to the elapsed time since the projection was launched.

 

uniform float HM_x;
uniform float HM_y;
uniform float HM_z;
uniform float HM_speed;
uniform float HM_time;
uniform vec2 HM_resolution;

void main( void )
{
      float red = (gl_FragCoord.y / HM_resolution.y);
      float blue = (gl_FragCoord.x / HM_resolution.x);
      float green = (sin(HM_time) + 1.0) * 0.5;
      gl_FragColor = vec4(red, green, blue, 1.0);
}

First of all, we declare those uniform values, to tell the shader we are going to use them.
We can then use those values in the shader. In the example above, the green value is equal to the sinus of the time (so we have a value between -1 and 1), to which we add 1 and then multiply by 0.5, in order to have a final value between 0 and 1, that varies with HM_time.

 

 

Here is another example of a code which draws a circle that changes color based on time. The circle radius depends on the HM_x variable, that you can modify in the software.

uniform float HM_x;
uniform float HM_y;
uniform float HM_z;
uniform float HM_speed;
uniform float HM_time;
uniform vec2 HM_resolution;

void main(void)
{
         float x = (gl_FragCoord.x / HM_resolution.x) – 0.5;
         float y = (gl_FragCoord.y / HM_resolution.y) – 0.5;

         float rayon = HM_x / 100.0;

      float d = sqrt(x*x + y*y);
      float red;
      float mov = 0.5 * sin(HM_time);

      if( d < rayon)
      {
         red = 0.7 – 2.0 * d + mov;
      }
      else
      {
         red = 0.6 * d – mov;
      }
      gl_FragColor = vec4(red, 0.0, 0.4, 1.0);
}

 

 

Modifying an existing shader

If you want to be able to use shaders found on websites like teractiveshaderformat.com or glslsandbox.com, you have to understand what’s happening inside a little bit.

The first things to modify are the Uniforms values. They are chosen by the person who executes the shader. In HeavyM, we chose to put 6 uniform values and to name them in a certain way (see above). You have to adapt your shader, depending on those names, to get your values correctly from HeavyM.

Let’s take an example: http://glslsandbox.com/e#40449.0.

#ifdef GL_ES
precision mediump float;
#endif

#extension GL_OES_standard_derivatives : enable

uniform float time;
uniform vec2 resolution;

void main( void ) {

      vec2 position = ( gl_FragCoord.xy * 2.0 – resolution) / min(resolution.x, resolution.y);
      vec3 destColor = vec3(1.0, 0.0, 0.8 );
      float f = 0.0;

      for(float i = 0.0; i < 50.0; i++){

         float s = sin(time + i ) ;
         float c = cos(time + i );
         f += 0.003 / abs(length(8.0* position *f – vec2(c, s)) -0.4);
      }

      gl_FragColor = vec4(vec3(destColor * f), 1.0);
}

In this code, glslsandbox calls those uniform values time and resolution. So you have to replace all the occurences of time by HM_time and all the occurences of resolution par HM_resolution.

#ifdef GL_ES
precision mediump float;
#endif

#extension GL_OES_standard_derivatives : enable

uniform float HM_time;
uniform vec2 HM_resolution;

void main( void ) {

      vec2 position = ( gl_FragCoord.xy * 2.0 – HM_resolution) / min(HM_resolution.x, HM_resolution.y);
      vec3 destColor = vec3(1.0, 0.0, 0.8 );
      float f = 0.0;

      for(float i = 0.0; i < 50.0; i++){

         float s = sin(HM_time + i ) ;
         float c = cos(HM_time + i );
         f += 0.003 / abs(length(8.0* position *f – vec2(c, s)) -0.4);
      }

      gl_FragColor = vec4(vec3(destColor * f), 1.0);
}