How to implement 2D raycasting light effect in GLSL

1. Background map

   Ok I created a test RGBA map as 2 images one containing RGB (on the left) and second with the alpha channel (on the right) so you can see them both. Of coarse they are combined to form single RGBA texture.

   

   I blurred them both a bit to obtain better visual effects on the edges.

2. Ray casting

   As this should run in GLSL we need to cast the rays somewhere. I decided to do it in fragment shader. So the algo is like this:

   1. On GL side pass uniforms needed for shaders Here goes mouse position as texture coordinate, max resolution of texture and light transmition strength.
   2. On GL side draw quad covering whole screen with texture of background (o blending)
   3. On Vertex shader just pass the texture and fragment coordinates needed

   4. On Fragment shader per each fragment:

      • cast ray from mouse position to actual fragment position (in texture coordinates)
      • cumulate/integrate the light properties during the ray travel
      • stop if light strength near zero or target fragment position reached.

Vertex shader

// Vertex
#version 420 core
layout(location=0) in vec2 pos;     // glVertex2f <-1,+1>
layout(location=8) in vec2 txr;     // glTexCoord2f  Unit0 <0,1>
out smooth vec2 t1;                 // texture end point <0,1>
void main()
    {
    t1=txr;
    gl_Position=vec4(pos,0.0,1.0);
    }

Fragment shader

// Fragment
#version 420 core
uniform float transmit=0.99;// light transmition coeficient <0,1>
uniform int txrsiz=512;     // max texture size [pixels]
uniform sampler2D txrmap;   // texture unit for light map
uniform vec2 t0;            // texture start point (mouse position) <0,1>
in smooth vec2 t1;          // texture end point, direction <0,1>
out vec4 col;
void main()
    {
    int i;
    vec2 t,dt;
    vec4 c0,c1;
    dt=normalize(t1-t0)/float(txrsiz);
    c0=vec4(1.0,1.0,1.0,1.0);   // light ray strength
    t=t0;
    if (dot(t1-t,dt)>0.0)
     for (i=0;i<txrsiz;i++)
        {
        c1=texture2D(txrmap,t);
        c0.rgb*=((c1.a)*(c1.rgb))+((1.0f-c1.a)*transmit);
        if (dot(t1-t,dt)<=0.000f) break;
        if (c0.r+c0.g+c0.b<=0.001f) break;
        t+=dt;
        }
    col=0.90*c0+0.10*texture2D(txrmap,t1);  // render with ambient light
//  col=c0;                                 // render without ambient light
    }

And Finally the result:

Animated 256 colors GIF:

The colors in GIF are slightly distorted due to 8 bit truncation. Also if the animation stops refresh page or open in decend gfx viewer instead.