I know something about shaders, I have written some very basic shaders. This is the most advanced shader I have written so far.

It's been several days since I conceived the idea of trying to recreate this shader (image below), but I still can't wrap my head around on how to approach it.

I figured out I can hide part of the object by making the texture fully transparent and then, after player moves the brush across the object, project the brush on to the object and reveal that part of the texture.

Shader "Custom/Masked Transparent" {
    Properties {
        _Color ("Main Color", Color) = (1,1,1,1)
        _MainTex ("Base (RGB) Trans (A)", 2D) = "white" {}
        _MaskTex("Mask Texture", 2D) = "white" {}
    }

    SubShader {
        Tags {"Queue"="Transparent" "IgnoreProjector"="True" "RenderType"="Transparent"}
        LOD 200

        CGPROGRAM
        #pragma surface surf Lambert alpha:fade

        sampler2D _MainTex;
        sampler2D _MaskTex;
        fixed4 _Color;

        struct Input {
            float2 uv_MainTex;
        };

        void surf (Input IN, inout SurfaceOutput o) {
            fixed4 main_color = tex2D(_MainTex, IN.uv_MainTex);
            fixed4 mask_color = tex2D(_MaskTex, IN.uv_MainTex);
            o.Albedo = main_color.rgb;
            o.Alpha = mask_color.r;
        }
        ENDCG
    }
}

But this implementation lacks the perceived height and liquidity you can observe in the original shader . How do I implement the effect of smearing acrylic paint? Any advice, resources, tutorials and maybe there's ready to use assets - anything is welcome! Thanks!

Looks like verlet impelmentation of a grid wit a texture and a normal map

Hmm… Well, it looks to me that the “ripples” in the main body of the “paint” are static--I would guess, then, that they're handled via a basic normal-map with some lighting applied.

The little “splats" that are visible when painting is begun I might guess are actually separate bits of geometry that render under the main image, and that shrink over time and then disappear.

(The shrinking I'm not confident of, so I'll leave that for another poster. My guess is that it involves a fuzzy “brush” and the control-image being edited over time to become less “fuzzy”, with normals being calculated on the fly in the shader.)

That's a fluid simulation topic then, not really about shaders or lighting.

Personally i've started with this project, which was hard to port because code is incomplete for Windows. But there is a nice and small 2D example on the page: https://github.com/yuanming-hu/taichi_mpm​
MPM is in general faster than the well known SPH method, but it's math is harder. For SPH there are many tutorials to find.
Beside a simulation framework, you also need a material model, in this case something very viscous like honey. The project above has a jelly material which i think would work with some tweaking.

If you're new to the subject, this will take a lot of time and serious effort!

After that, all you have is particles. (A desktop CPU can simulate 10k-100k particles in real time, GPU compute would be faster.)You then need to create a surface from the particles, which can be done with isosurface meshing methods like marching cubes, or volume rendering / raytracing methods, after turning particles into a density volume.

There are also fluid simulation methods which work on surface meshes directly, not requiring particles or volumes. See ‘vortex sheets’ or ‘Surface-Only Liquids’, for example.Your image shows meshing artifacts like long sliver triangles, which hints such method is used, but i can't tell.

I assume all this is quite overkill for what you want to invest. Ideally you can use a limited but low complexity 2D simulation. That's mostly used for terrain / erosion simulations, e.g. this: https://github.com/LanLou123/Webgl-Erosion

The project contains an implementation of a shallow water simulation model, which also is used in games to get waves and ripples on a surface, e.g. puddles. Fast and simple. I guess this could work by making the water more viscous.

Programmer71 said:
Looks like verlet impelmentation of a grid wit a texture and a normal map

That's another / complementary option. You could make a grid mesh representing a height field, and brush strokes allow to deform / smear the grid.
That's simple and practical as long as the mesh topology remains static, avoiding the dynamic meshing complexity of surface simulation methods.

Thanks for all the answers! I'm not familiar with either of those terms (verlet, MPM, SPH, vortex sheets and Surface-Only Liquids) so I guess it's time for me to google. The shader is from a small hyper-casual mobile game. I doubt they used a full blown particle based fluid simulation for this effect, it would've been an overkill. I'll try to do my research and learn more about verlet and Surface-Only Liquids.

blablaalb said:
I'll try to do my research and learn more about verlet and Surface-Only Liquids.

Maybe you can look up cloth simulation to learn about these. Cloth is usually a mesh (so ‘surface only’), with distance constraints from vertices to their adjacent vertices. Mesh edges can be used to represent those constraints.

Verlet is a simple integration method replacing the concept of velocity with the difference of the previous position and the current position.

This paper explains it very well: https://www.cs.cmu.edu/afs/cs/academic/class/15462-s13/www/lec_slides/Jakobsen.pdf​

This way you can make some grid patch behaving similar to cloth. You can drag it, and it tries to get back to it's initial state. Notice: That's not what fluid does, which does not remember an initial state and trying to get back to that. It can change it's shape completely, unlike cloth.

Thus i'm not sure how useful this is for your case. I guess, you would need some additional form of energy (representing volume / density / height) which can move from one polygon to the next, to compensate the limitation.
But once you have the ability to move matter around freely (preserving volume is usually the problem), the underlying deformable cloth mesh is no longer needed at all, eventually.

But not sure. If i had to do this, i would not know how yet either. I would try some approaches in hope something turns out working.

blablaalb said:
I doubt they used a full blown particle based fluid simulation for this effect

There is only one proof they use simulation at all: The shrinking splat of paint at the beginning of the animation.
After that, at looks much more like a simple painting application, creating a bumpy structure. We don't see brush strokes in the paint, so obviously their approach is limited, more aiming to fake it than to simulate it.

Maybe you don't need any simulation at all. Something like creating a vector field from paint strokes and creating procedural bump texture aligning to the strokes might be even more impressive.

Searching for ‘paint simulation’ on YouTube gives lots of results, coming closer to the topic maybe.

JoeJ said:
There is only one proof they use simulation at all: The shrinking splat of paint at the beginning of the animation. After that, at looks much more like a simple painting application, creating a bumpy structure. We don't see brush strokes in the paint, so obviously their approach is limited, more aiming to fake it than to simulate it.

I'm still inclined to suspect that it might be done with some sort of image-based approach.

I'm imagining a system in which the user “paints” a fuzzy brush-stroke onto a texture (fuzzier at the start of the stroke, to give that initial “blob” of paint), which is used as the input to the shader. The shader then thresholds that fuzziness to determine the edges of the paint. It also outputs not only the full rendering, but also the input texture, modified via something like a smoothstep operation to reduce its fuzziness. On the next frame, this modified texture is used as the texture to be painted on.

That said, as I noted above, I am not confident in the feasibility of this, and do stand to be corrected!

Thaumaturge said:
I'm still inclined to suspect that it might be done with some sort of image-based approach.

Well, we could say any 2D fluid sim using Eulerian grid is image based too, so that would not rule out the use of simulations anyway.

I have no doubt they do simulation because of the shrinking blob. That's basically a form of surface tension, trying to minimize surface area by concentrating volume to one spot. This can be done either with simulation, or by expressing it as an iterative optimization problem.
The meshing artifacts show they use a mesh based approach in some form at least, which wonders me. Maybe they needed ability to paint on complex topology, not just on a flat surface.

The question is: Is such dynamic motion of paint desired and needed?

Thaumaturge said:
I'm imagining a system in which the user “paints” a fuzzy brush-stroke onto a texture (fuzzier at the start of the stroke, to give that initial “blob” of paint), which is used as the input to the shader. The shader then thresholds that fuzziness to determine the edges of the paint. It also outputs not only the full rendering, but also the input texture, modified via something like a smoothstep operation to reduce its fuzziness. On the next frame, this modified texture is used as the texture to be painted on.

To get some painting, i'd see another method: Make a path of points from the user stroke, and generate a quad mesh by extruding this path. (Artifacts on sharp corners expected.)
The quad mesh can be easily textured to add the bumpy texture of various brushes. So the texture would have height of paint, plus some alpha to fade it out.

Then this mesh could be rendered to a texture, adding one brush stroke on top of the former strokes. This gives us a bump map which we can convert to normal map.

Pretty much the same idea as yours, but some graphics programmers might prefer to utilize the rasterization functionality to model the stroke.

If we want the paint to flow around, simulation could be implemented on top, e.g. so one color diffuses into another.

Though, if we want to paint on complex 3D models, we have no simple mapping from the 3D surface to a planar texture domain, and everything becomes much harder.
Even if we have nice UV charts on the model, there will be no adjacency information across the chart boundaries, so simulation breaks, and texturing has seams.
This can be solved with seamless UV maps, but having worked this out, i would say that's much too difficult.

Alternatives then become 3D particles, making a paint mesh on top the model, working on the mesh directly (using vertices / polys instead texels), or various approaches of volumetric texturing.
But all of that is much more expensive and complex, so i would hope a 2D approach is good enough.

JoeJ said:
Well, we could say any 2D fluid sim using Eulerian grid is image based too, so that would not rule out the use of simulations anyway.

Hah, okay, fair enough!

JoeJ said:
I have no doubt they do simulation because of the shrinking blob. That's basically a form of surface tension, trying to minimize surface area by concentrating volume to one spot.

It could also be done by starting with a fuzzy source-image, and using something like:

reRenderedSourceImagePixel = smoothstep(min(0.9989, 0.999 * dt), 0.999, inputPixel);

Or so I would think.

That should result in an image in which the lighter colour (which I'm presuming for the sake of example to be the underlying “painting” colour) gives way to the darker colour over time--but never so much that pure white is overtaken.

As a result, a fuzzy blob slowly shrinks in on itself, I would imagine.

JoeJ said:
Pretty much the same idea as yours, but some graphics programmers might prefer to utilize the rasterization functionality to model the stroke.

That's fair; something like that could work too, I daresay.

JoeJ said:
Though, if we want to paint on complex 3D models, we have no simple mapping from the 3D surface to a planar texture domain, and everything becomes much harder.

Oh yes, that's a much harder problem. What we have here, I'm glad to say, is significantly simpler! Just a nice, uncomplicated rectangle to paint.

blablaalb said:
This is the most advanced shader I have written so far.

sidenote: I don`t think that`s too much relevant if it does not relate to the reason/problem why you`ve created this post (like it`s not related to what you`re currently trying to achieve ). You should focus on explaining the problem you`re trying to solve. If it bears relevance to the issue for which you`re seeking help than you should copy past that code in this thread.