As such I sat down and had a look at the code. It was a hacky mess of naughty tricks that confused me as to how I'd even gotten it to work in the first place. If I recall, I sort of threw code at it until something akin to shadows edged onto my screen and I called it a day.
"But that's not good enough!" I cried out. Which caused some people nearby to give me funny looks. This weekend I decided to build the shadow engine properly and I pretty much scrapped a vast majority of the code. It's still not perfect or completely finished (there are a lot of possible effects and optimisations (see previous post) I'd like to add to it). It's also superior to my previous attempt with a nice array of features.
How to muck around with light:
The top left button will reset the scene, positioning the lights and objects randomly. You can click and drag the objects around the scene. Dragging them through a light or another object will glitch out the shadows (I haven't done light/object intersection testing or object collision testing). Objects will shadow each other, too.
Clicking on the light buttons labeled "light x" will enable control of that light. With a light selected, hold the spacebar and the light will move towards the mouse. Select the light again to turn off the controls. Change the light colour with the red/green/blue ratio sliders - the objects are white so you can see how the light colours them, lights of different colours will blend together (disco lights!).
Play with the intensity, etc. to see the effect. Glows are affected by the intensity and falloff of the light itself but have a separate radius. A high falloff level will give a hard edge at the radius to the light. You can turn off the light or the glow. Turning off lights will speed up the simulation. If multiple lights cast onto the same object, it'll slow down the sim quite a bit. So it's something I'll be avoiding in the actual game.
When light enters an area, the photons bounce around against objects. Effectively, this causes the objects to "light" each other. This is why, when there is only a single light source, you can still see in the shadowy areas of a room. The light bounces around even into the shadowed area and lights it up a bit as well. The effect is called radiosity. Besides just lighting areas, it also creates a colour-bleed between various objects. If you have a green room, objects within that room will be tinged green due to the light bouncing off the green walls (and the bits of the wall near certain non-white objects will tint towards those colours a little).
I've simulated this above on a washing machine in a room with some slightly weird perspective. In the first image, there is no colour bleed and the shadow is solid black. It doesn't look very natural at all. The second image has some colour bleeding onto the (white) machine and the wall and floor bleed colour onto each other (although it may be hard to see the wall/floor bleeding). This makes the image feel as though it is more inline with reality. Except for the alien geometries.**
Colour-bleed is a pretty complex and intense (to the CPU) thing to do in real time. So I've chucked that concept and instead stuck with a slight lightening of the shadows instead, depending on the intensity of the light around them (and the density of the shadow as larger or less translucent objects have more dense shadows). This is more easily noticed if you only have a single light affecting an object (simply turn off all the other lights) and adjust the lights intensity while taking note of the shadow density. I've also included something called ambient lighting, which makes the shadow layer more or less opaque, simulating a higher or lower level of radiosity in a pretty rudimentary way.
(it's the top slider in the prototype, feel free to drag it around. Things look much better when it's darker, though).
There's lots still to do in the shadow engine but I feel I've done enough to move onto other things for a bit before I get too carried away on just that one feature. Object shadows are currently limited to non-concave objects, but it shouldn't be too bad to adjust that by grouping shadows together. The shadow that an object casts and the object itself are unrelated elements so you can have very complex visuals with fairly simple shadows.
At the moment, I've only allowed it to do big veiling sort of shadows (blocking out light completely behind an object) which is fine for large objects but it would be nice to have smaller objects (like a thin pole, or thrown knife) to only cast a small shadow of themselves nearby without the veiling.
And for extra fun, a sun or global light system would be cool for outdoors. At the moment, shadows expand out from the source light, but a sun system will keep the shadow angles the same for each object.
* line blatantly stolen from the Scryed anime.
** Cthulhu reference again. I should probably stop that.