+74
Under review

User defined nodes to encapsulate frequently used functions

elliot bentine 11 years ago updated by Demian Ford 2 years ago 23

I'd like to be able to group collections of nodes I often use and save them into a new user defined node. Eg, dot product of the surface normal with the view direction for Fresnel-type lighting, so that in future shaders I can simply drag and drop the custom node in.


If we had a way of sharing these nodes, with the basic operations the testing community could also help implement missing nodes (panner, rotator, dot, etc) without having access to the source code.

Answer

Answer
Planned
Node nesting is a planned feature :)

Fantastic! Many thanks

Answer
Planned
Node nesting is a planned feature :)

+2
As a temporary solution, this might be useful: http://forum.unity3d.com/threads/242835-Shader-Forge-Extensions
It would also be nice to be able to group/expand certain nodes in a huge node network to keep it more readable.
+1
I love this idea and could really use it. A few ideas on how I personally would like it to work:
  1. The nested nodes should act a little like a unity prefab. You should be able to reuse it and if you make changes to it, there should be a way to apply those changes back to the "prefab." I wouldn't necessarily want it to auto-update other shaders, but it should be easy to import/apply the updated prefab in other shaders that use it, rather than deleting and re-adding. This implies some form of library either in the project or saved globally to share between projects.
  2. I agree with Kajfa. The nested nodes could be visually grouped into a container that would be zoomed out by default with regular sized inputs/outputs on the container. that way you can kinda see what's going on, but you can also zoom in to edit.
  3. I think this is obvious, but I would like to be able to name them and have them show up in the list of nodes on the right, as well as in the shortcut menus (when you press a single letter), maybe at the bottom.
Unrelated, but huge congrats on the award at Unite!
+2
This is coming in Shader Forge 2
+1
I want this so much. When can we expect Shader Forge 2?
Under review

Now that SF2 has been somewhat cancelled, I may look into it for SF1.

It would however be limited in the following ways:


  • Nested nodes would only have a single output
  • Nested nodes would not be able to contain any properties
  • Nested nodes would probably not support nested nodes in themselves
  • If you changed a nested node group, you would likely have to re-open all the shaders using them to force a recompile.

Are these limitations too limiting, in a way that defeats the purpose of having nested nodes, or do you think it would still be worth it? As in, what's the primary use case for you?

Any statement somewhere as to why SF2 is somewhat canceled?


As for your question: Single in- and output would be pretty limiting, as for a lot of things you would need to feed some extra pre-calculated information into the equation of the nested node. And things calculated inside a nested node, sometimes need their way out as well.


Without properties would be okay, a lot is solved already if you can have multiple inputs as you can just feed the property-data into the nested node that way.


What kind of implication do you forsee when you say you probably wont support nested nodes in nested nodes (this sentence also learns nested nodes isn't a good name either :P We're nesting nested nodes)?


What if you generate include-files out of nested nodes and basically just call a function from the main shader into the include-file? I'd foresee that Unity would recompile it all for you UNLESS you decide to modify the in- and outputs. In your defense, I think even UE4 draws the line there.


If I were to make this: the most flexible way would be to take your Code-node, and instead of code, allow one to assign a sub-shader/nested node/include-file to it and save that node as-is. Then its the nested nodes responsibility to make the inputs/outputs match that of the one defined, and even after it changes, you won't mess up the wiring of all nodes, unless one physically updates/regenerates the node.

Oh, they would always support multiple inputs, just not multiple outputs :)


The tricky part about nested nodes is that I have to add check to ensure not nesting itself, into itself, and that I have to recursively traverse and unfold all the nodes when compiling, which may be heavy. Not saying it's impossible, just not sure how important it is.


The problem with generating an include file is that nested nodes shouldn't simply be operators. I mean, I could limit it that way, in which case it would work, but what if you want to use the normal direction node in a nested node? In that case, you need all the other things a shader needs in order to read that data.

You will still get nested loops of functionA calls functionB which calls functionA again, this is where the stackoverflow exception is for :P You don't have to prevent it at all, just notify the user that stack is getting rather full and that you quit the code generation (I'm pretty sure even GPU's stop processing shaders if they keep calling recursively).


I'd say anything more complex than the standard nodes like add/subtract/oneminus, etc. should be fed to the nested node by means of an input, not as an actual node inside them. It's been a while since I've used UDK/Unreal, but I believe that's how they restrict it. It would make sense, too. Nested nodes don't live on their own, they're not shaders, they're a part of one.

In which case they should rather be called something like custom functions/operators. Might be possible to do this, but it seems to have quite a slim use-case coverage.

I see nested nodes as a way of taking a (huge) set of nodes, call them boiler-plate nodes if you will, and change them to a single node instead. Looking at my clip library I mostly have clips for very repetitive node structures that a bunch of shaders all have in common, and thats what nested nodes should provide IMO. A single node that acts as a bunch of them, to keep the tree small and easily interpretable.

I agree - this is also the majority of the ways I've seen them used in projects. While it would be really nice if artists used them to share common settings/values and enforce certain consistencies across a project, I've never seen that actually happen in a live project with a significant number of people on it.

Agreed. Nested nodes in nested nodes is kind of a special case that - probably - not many artists will utilize. No need for endless recursion where I work. I rather prefer a goto set of functions that I repeatedly use in a bunch of shader graphs. Some examples:
- Triplanar mapping
- Unpacking custom normal maps
- Clipping plane gradient

- Circular gradient

- Worldspace Top-Down mapping

- Vertex Colors masks for custom coloring


Code nodes inside nested nodes would be nice, too.

So, I don't use shader forge very often anymore, so take it with a grain of salt. Anyway, my opinion is that not allowing nested nodes to nest is likely ok, but annoying as it means anyone creating a nested node is going to have to know that nested nodes are second class citizens and need to be handled specially. Multiple output support is nice for things like parallax occlusion mapping, where you can supply another output for shadow calculations and expect that the compiler will optimize it out when not in use. Not having properties seems ok, as long as the input properties can still be optimized to static values by the compiler.


Personally, I'd personally prefer to approach this from a higher level; nested nodes are just a way to create a function library - in most cases, that library gets shared and used over and over, and doesn't change that often. While you might have some truly unique, project specific nodes, the number is likely pretty small. Thus the recompile issue is not huge IMO. Additionally, any system which allows us to add custom nodes would potentially solve the same issue. The code node is currently cumbersome, not easily re-usable, and has many limitations. I don't personally care if the way to add new nodes is via the node structure or another technique, and while I suspect some of the community would be sad about another solution, for most users it's just a way to get more nodes that encapsulate common tricks. If an API could be exposed that would allow people to add nodes in another way (c#/CG files, etc) and had less restrictions, then that would be preferable.


I think the problem with allowing users to create their own nodes the same way that I do, is simply that it still doesn't cover the very common use-case of nested nodes, where you want to have a set of visuals/behaviours shared between shaders. Sure, it allows you to make custom operators, but, most of those should already be provided by SF, no?

+2

Some nodes I used to have in my shader graph that SF doesn't support, that I would add, include:


GradientNoise 1D/2D/3D

WorleyNoise 1D/2D/3D

FBM functions

ParallaxOcclusionMapping (and it's various variants)

Flow Mapping

SDF functions

Curve/Gradient nodes

Masking nodes (isolate signals, etc, via simple nodes instead of math)

etc..


For me, nodes in nodes and the code node have always been ways to add large functions which are not available in SF by default and are too unwieldy to work with as a unique part of each graph that uses them. Our old shader graph didn't have nested node support (nor were the artist crying for it)- but it was very easy (for coders) to add nodes to, and our artist would just ask show us a graph when they wanted it encapsulated into a node and we'd do it for them.

Take a look at how Blender groups nodes. It's very efficient, and effective. It's also easy to simply share the node group with someone else.

In the meantime, how about a simpler implementation where you can simply select a group of nodes and save them along with their connections (amongst themselves). Then when you're in a shader, you could browse the list of saved nodes and load a saved group and Shader Forge would recreate all those nodes in your current canvas like a copy/paste. Not as tidy I guess, but still super helpful when you have commonly used node trees and don't want to manually recreate them when making a new shader.

Don't worry about updating prefab instances of the node groups or whatever (user can simply load updated group and swap it out with old one themselves), or number of outputs, or stuff like that. Just plop the nodes in there.

I've always dreamt of winning BIG in the lottery. I surfed the internet for help to win the lottery massively and I was directed to the DR Amber website. I had a chat with him about what I needed and he assured me that his spell will make me become a lottery winner. He got back to me after 3 days with the numbers that I needed to play the lottery and told me to believe in myself. I felt enveloped by the control of the spell. I took his words and played the Mega Millions Lottery. Two days later, I was sent an email that I've won the sum of 107 Million Dollars. I was shocked and couldn't believe what my eyes were seeing. Immediately I was given the cash prize, I stopped working. I want to say a very big thank you to DR Amber for helping people like us who really need help. Now I know that there's absolutely nothing too hard for DR Amber to do. To get in touch with him, email : [ amberlottotemple@yahoo.com ] or click his webpage URL___ amberlottotemple.com