Advertisement

UE4 Level Blueprints: A case for Node Based Programming?

Started by August 04, 2015 05:52 PM
21 comments, last by SmkViper 9 years, 2 months ago

"Node Based Programming" ( as described ) has existed for a LONG time - it hasn't 'caught on' mainly due to how inflexable it actually is.

There isn't 1-to-1 association between 'node-based programming' and 'visual programming'. Node-based and flow-based toolkits are really very common in the enterprise world, minus the fancy visual editors.

For example, an enterprise service bus architecture such as Apache Camel, Microsoft's Azure Service Bus, or Spring Integration is effectively a flow-based routing engine, with DSLs for defining routing logic, that connects generic beans together into a service.

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

I've been using UE4 for about 6 months now. Here's my opinion on blueprints:

Pros:

-It is very fast for iterating quickly on a game system. You don't need to wait 2-5 minutes between compiles.

-It's fantastic for people who don't program as much (designers, artists) because it makes gameplay mechanics a lot more accessible to them.

-You can't shoot yourself in the foot as easily with things like linker errors and cyclical includes

-You can write lower level C++ code which creates a high level node other people can use

-You can see program execution traveling along the node wires

-It works very consistently with other node based systems

Cons:

-You have to manage the wires which connect nodes. If you don't, you get spaghetti! Any time spent managing these wires is an overhead cost you have to pay for.

-The node commenting system is inferior to code commenting. You can't comment out sections of nodes like you can with code.

-It's very hard to reuse. You can't copy and paste large sections of nodes like you can with code.

-Something which can be done in one or two lines of compact code takes several minutes to accomplish with blueprint nodes

-Debugging is a pain in the ass. You can't set watches on variables or dig into an objects variables. Some variables are even out of scope when they shouldn't be.

-If you've never done any programming before, you're going to have a steep learning curve.

-If your blueprint crashes the editor, you lose any unsaved work and need to relaunch the editor and reopen your workspaces (opening lots of tabs)

-Infinite loops in your construction script (like a constructor)

-I don't know how to measure how much memory a blueprint uses or how fast it runs.

Advertisement


These upsides all sound like tool/language features, not features inherent to visual scripting in general. There's nothing stopping anyone from coming up with a programming language that allows you to use arbitrary non-default function parameters (in fact, C# supports named parameter syntax which could easily turn into something like this). Having a textual programming language intellisense that only shows functions that return a value of a given type doesn't seem too far-fetched, either.

Yeah, if IDE-features for other languages would advance that far, I'd totally love that. However if we are taking C++ out of the way, there is rarely any text-based language that supports anything like that, at least from what I know (maybe I missed the right IDE). Ruby, Python, Lua, Angelscript, all those had barely even the most basic of intellisense-like features you have in C++, so having a (visual) scripting language that does all that for you, is a huge step up in my opinion. Its also true that those are not exactly visual scripting features per se, but I consider the IDE to be an integral part of a visual scripting language moreso than for a textual language, since you can't even do anything without an IDE in the first place while you can still code even in notepad... but yeah, its true that those are mostly features due to Unreals great toolset, which is where my post was headed in the first place.


"Node Based Programming" ( as described ) has existed for a LONG time - it hasn't 'caught on' mainly due to how inflexable it actually is.

IDK, correct me if I'm wrong since I'm no expert on the matter, but I've never seen anything like Unreals implementation before, so this could be the reason for that. What is so unflexible about a sophisticated visual scripting system? Unreal has inheritance (even from C++), access restriction, overridable methods (even from C++), enums, structs, RTTI, encapsulation in methods, const/static methods... I really don't know if there ever has been a generic-purpose visual scripting language with such a huge (complete?) featureset than that before.


-The node commenting system is inferior to code commenting. You can't comment out sections of nodes like you can with code.

Well, you can do that even easier - you can just cut the connection to a block of nodes. Unless of course you want to "comment out" a block ouf nodes within a large block of nodes, in which case you can still just draw the wire from the first node-to-be-commented out to the first node after that block. But I agree, it would be nice to have a feature to just tell a bunch of nodes "be gone for now!", but thats certainly in the range of possibilites for the feature.


-It's very hard to reuse. You can't copy and paste large sections of nodes like you can with code.

Why not? You can zoom out to quite huge degrees, and group-select as much as you want, as well as copy & paste without many limitations. In our Unreal projects at university, I used to copy around the whole call-graph of a quite complicated HUD-class to another one, without problems. The only issue I can imagine is with class methods not being copied due to not being in the target class, which could certainly be solved. Other than that, why would you even copy & paste large amounts of code anyways? Its much better (and perfectly duable in blueprints) to generalize that code in like global methods or in a base class (like I did at some point with my copy pasted code once I was able to identify the common shared code). To be fair, it tends to be a bit harder to identifiy shared code on certain occasions in the visual scripting system.


-Something which can be done in one or two lines of compact code takes several minutes to accomplish with blueprint nodes

Can you give an example of that, because I'm honstely curious. I've had quite some speeddowns due to complicated math routines that are much easier done in code than in blueprints, but I've never had the situation where I'd have said "phew, that took a while. If I wrote that in code I would only take a line, but here I spent 2 minutes placing and connecting nodes".


-I don't know how to measure how much memory a blueprint uses or how fast it runs.

I'm not sure anymore, but doesn't Unreals profiler also show the timings of the blueprints? It at the very least shows the execution time off all blueprints globally, as I said I'm not sure anymore about single blueprints but I quess there would have been an option.

-The node commenting system is inferior to code commenting. You can't comment out sections of nodes like you can with code.


Well, you can do that even easier - you can just cut the connection to a block of nodes. Unless of course you want to "comment out" a block ouf nodes within a large block of nodes, in which case you can still just draw the wire from the first node-to-be-commented out to the first node after that block. But I agree, it would be nice to have a feature to just tell a bunch of nodes "be gone for now!", but thats certainly in the range of possibilites for the feature.


Yeah, that's what I do for the time being. But its bad. Why would I want to disconnect wires and then have to reconnect them later? What if I forget where those wires connect to after having them disconnected for a few days/weeks/months? In visual studio, you can comment out a huge section of code by using "Ctrl+k + Ctrl+c" and its instantly commented or uncommented. In the blueprints, there should be some sort of way to just disable a bunch of blueprints and the wires should be greyed out and the flow of execution should somehow skip past the gray wires without executing anything. I'm not going to hold my breath waiting for that to happen though.

-It's very hard to reuse. You can't copy and paste large sections of nodes like you can with code.


Why not? You can zoom out to quite huge degrees, and group-select as much as you want, as well as copy & paste without many limitations. In our Unreal projects at university, I used to copy around the whole call-graph of a quite complicated HUD-class to another one, without problems. The only issue I can imagine is with class methods not being copied due to not being in the target class, which could certainly be solved. Other than that, why would you even copy & paste large amounts of code anyways? Its much better (and perfectly duable in blueprints) to generalize that code in like global methods or in a base class (like I did at some point with my copy pasted code once I was able to identify the common shared code). To be fair, it tends to be a bit harder to identifiy shared code on certain occasions in the visual scripting system.

If you have several projects, there may be a time when you want to take the generalized functionality of one project and insert it into another. It's a pain in the ass to copy/paste blueprints and nodes because a lot of the stuff contains project specific stuff.

Another use case is where you create a specific type of object (in my case, a wooden door). It works. It's great. Now I want an iron door. Oh wait, both of those are doors! So I should create a generic "door" object and have both wooden and iron doors inherit from it. Now, can I just copy/paste all of my wooden door code into the parent class? Nope! You have to go through and manually recreate every. single. function, variable, event, and delegate. In code, it would be a simple class rename and creating two new classes for each door type and implementing their unique behaviors. A 5 minute job gets stretched to 5 hours, depending on your node complexity.

-Something which can be done in one or two lines of compact code takes several minutes to accomplish with blueprint nodes


Can you give an example of that, because I'm honstely curious. I've had quite some speeddowns due to complicated math routines that are much easier done in code than in blueprints, but I've never had the situation where I'd have said "phew, that took a while. If I wrote that in code I would only take a line, but here I spent 2 minutes placing and connecting nodes".

Yeah, this is rather trivial to demonstrate. Here is a few lines of code from one of my base creature classes:


bool ABaseCreature::ShouldExplode(FCExplosion Explosion, float &OutImpulse)
{
	if (Explosion.Radius <= 0)
	{
		OutImpulse = 0;
		return false;
	}

	float Dist = FVector::Dist(GetActorLocation(), Explosion.Origin);
	float alpha = 1 - FMath::Clamp<float>(Dist / Explosion.Radius, 0, 1);

	OutImpulse = FMath::Lerp<float>(0, Explosion.Force, alpha);

	return (OutImpulse >= 1500 || (Health * 100) <= OutImpulse);
}
Now, try to imagine what a mess that would be using blueprint nodes.
You've got a function which gives an an explosion struct as input. To do this in blueprints:
1. you have to split the explosion struct.
2. Add a <= node which takes float values and compare it against 0
3. Use the node from 2 and hook it up to a branch node
4. Feed the "true" part of the branch into a temporary "return" value, because a blueprint can't return multiple values.
5. Create a temporary "float" value in the functions local values.
6. Create a "get actor location" node to get a vector
7. get the explosion origin vector wire from the input and feed that into a "vector distance" node, and set the local variable node to the output (four nodes to do 1 line here)
8. Same thing for the float alpha variable, but even more nodes.
...
And then you've got the return values. Each operator and comparison is a node. I count four operators, so thats four nodes for a line of code.

-I don't know how to measure how much memory a blueprint uses or how fast it runs.


I'm not sure anymore, but doesn't Unreals profiler also show the timings of the blueprints? It at the very least shows the execution time off all blueprints globally, as I said I'm not sure anymore about single blueprints but I quess there would have been an option.

This is more ignorance on my part than a beef with the node system.

On the flipside, the only way I can even view Blueprints (let alone edit them) is in the Unreal editor, which is a pretty heavy application. And if I wanted to write some quick tools or batch scripts to process some blueprints - how do I do that? I'm going to have to go figure out Unreal's format, parse it, make my changes, and then write it back. Not impossible, but I won't really have the libraries or tools without bringing in at least a subset of the Unreal engine into my codebase.


Thats more an issue of how Unreal handles its assets. It has binary format for everything including stuff like textures, etc... unlike Unity which I belive stores textures in a separate file (this makes working with unreal partially really annoying once you want to modify a texture in an external application). Visual scripting on the core is not very different than regular scripts. I have my own visual scripts represented as XML, other might do JSON, or you could even potentially have the script as just a plain-text script file, depending on how you load and compile your visual scripts. Only difference is that you have to store the position of the nodes (and possibly some side-attributes like count of output-nodes for a sequence-command), but other than that there is nothing that hinders you from storing a visual script in a text file and processing it with external tools (which also gives you merging abilities back).


It's not Unreal-specific. My point was that "visual scripting" has no defined base data format. You use XML - someone else uses JSON - and then there's all the binary formats.

At least with a text-based language you have a basic format that a large range of tools can understand - you just have to layer on the language specifics with some modifications. At worst, you open it up in notepad and get no syntax coloring or auto-complete, but at least you can still see what it does in a easily understood manner (assuming the coder *wrote* them well smile.png )

Try doing that across all "visual" scripting languages...

It's not Unreal-specific. My point was that "visual scripting" has no defined base data format. You use XML - someone else uses JSON - and then there's all the binary formats.

While that is true, in a way this is not different to regular scripts/code - every language has its own syntax, in the case of visual scripting this is defined partially defined via the file format. It might be a semantic issue, but I don't see that much of a difference - you cannot just write tools to operate on all available textual languages, due to different syntax (C-derivates using {}, python using intendation and Ruby using XXX - end for every block). Sure, you can always just view textual languages in a notepad, and I get that it limits the use of visual scripts to a certain range of language-intern tools. I personally don't see that as much of an issue (after all thats why its called VISUAL programming biggrin.png ), but I give you that if visual programming ever where to take over in large scale, there would have to be some sort of standard to be set in order to be more widely usable.


Yeah, that's what I do for the time being. But its bad. Why would I want to disconnect wires and then have to reconnect them later? What if I forget where those wires connect to after having them disconnected for a few days/weeks/months? In visual studio, you can comment out a huge section of code by using "Ctrl+k + Ctrl+c" and its instantly commented or uncommented. In the blueprints, there should be some sort of way to just disable a bunch of blueprints and the wires should be greyed out and the flow of execution should somehow skip past the gray wires without executing anything. I'm not going to hold my breath waiting for that to happen though.

I don't see it as that bad - sure, I havn't had a situation where I had wired disconnected for a period of months, but usually if I do this blocks are already positioned in a fashion that makes it stick in the eye where the wire originally had been. But again, having such a feature would be a nice addition. I'll certainly put it on my own feature-list because I like it. Since I don't use Unreal anymore ATM, I won't bring it up to them, but if you are using it regularely, why not bring it up to Epic Games attention? Maybe if enough users like the idea they will implement it based on that.


If you have several projects, there may be a time when you want to take the generalized functionality of one project and insert it into another. It's a pain in the ass to copy/paste blueprints and nodes because a lot of the stuff contains project specific stuff.

Ok, I can't comment on that, such until now I always ever had one project to work on at the same time, and didn't have to carry code over to the other. Unless I'm mistaken the problem is that copy/pasting will remove all nodes that are not known at this time in the other project, am I right? Which would mean that this could again be solved if the editor would simply not remove those nodes but keep them (and simply issue a compile error as it is already capable off).


Another use case is where you create a specific type of object (in my case, a wooden door). It works. It's great. Now I want an iron door. Oh wait, both of those are doors! So I should create a generic "door" object and have both wooden and iron doors inherit from it. Now, can I just copy/paste all of my wooden door code into the parent class? Nope! You have to go through and manually recreate every. single. function, variable, event, and delegate. In code, it would be a simple class rename and creating two new classes for each door type and implementing their unique behaviors. A 5 minute job gets stretched to 5 hours, depending on your node complexity.

Again, I might not have the full picture of this example, but can't you just rename the WoodenDoor-class to BaseDoor, and create two classes the inherit from that? Or copy the WoodenDoor-blueprint, and then rename it? We certainly had to do this kind of stuff both on C++ and Blueprint level a few times, and I can't recall we had that amount of problems. But I remember that reparenting in blueprints can be a pain in the ass, so I do agree that there is also room for improvements.


Yeah, this is rather trivial to demonstrate. Here is a few lines of code from one of my base creature classes:

Ok, I agree that this is something you would not want to make in blueprints. As I though, its math & multiple return values which tend to be really asinine in blueprints. I'd still take you exaggerated with the one line vs 2 minutes blueprints analogy - because even though it sucks, I certainly wouldn't take 2 minutes to create ie. the last return statement. But again, I agree that it will take longer to create this function in blueprint than in code. I still have to ask a few more times here:

5. Create a temporary "float" value in the functions local values.
6. Create a "get actor location" node to get a vector
7. get the explosion origin vector wire from the input and feed that into a "vector distance" node, and set the local variable node to the output (four nodes to do 1 line here)

Is the temporary in 5. the equivalent to "Dist" in the code? In that case, I say: You don't need that. In blueprints its common (at least I always assumed) to only create local variables when you really need to. So instead of having all those nodes and putting it into a local, you just directly connect it to wherever it is needed in the alpha calculation. You don't need a local here - in C++ you create a local variable because otherwise you will have 2-monitor wide lines, but in blueprint you can already group calculations by position (and/or a comment block if needed), so there is no need to store every single step of calculation. Exception is of course if you need that value multiple times in the same function, AND performance really matters (I can't count how many times I've just connected the same block of calculations to multiple other nodes and it didn't matter the least). I'm even thinking about giving a shorthand for this case by making the user able to mark individual nodes as "constexpr", which would mean that in the current function, whenever the node is called, only the first time it is actually executed and the rest of the time its value is just reused, like a local variable would, eliminating the need for local variables for even more cases.

Also, from your description I can't really see if you create the nodes separately or already create them by dragging a wire from the output-slot and then type in the node name. Because if you don't do that already, there is also a lot more time to be gained. If you do that, the time between writing those one line and creating multiple nodes does reduce a bit.

So yeah, I'm not even arguing that such a function is better written in code than in blueprints, I used to put just that type of calculation in C++ whenever it got outhand. Again, thats why I'm trying to design some textual math-extention for visual scripting to solve this problem, as I doubt math will ever be douable with nodes just as good as text (unless someone comes up with a brilliant solution). Though what I'm saying is that there is a lot of shortcuts to reduce the time and amount of work you have to do in the blueprints. If you also count/add the compilation time to the mix (our project with ~10k LoC used to compile like 2-3 minutes for adding a function like this sometimes), then I'd say the difference shrinks.

EDIT: Also I noticed that if you screw the local variables you can simplify the blueprint version of your code even more, by converting the conditional return to a set of OR/AND statements, and using a Select-statement for what you return as Impulse. I'll check if I have a workable Unreal at this PC so I can give you an example of what I mean. Thats kind of what I'm trying to get across though - if you try to 1:1 map code to blueprint, its not going to work. But those visual scripts sometimes let you create code in a different fashion than you would in a textual language, which (at least) reduced the overhead by a significant margin.

Advertisement

I am looking at "node-based-programming" as a programming paradigm.

A quote from my previous thread:

"I also think that node based programming could work for any language. I could see myself programming in c++ with a node based system."

I am now wondering how I would simulate a node-based system in Python.

Again, at it's core, it all centers around input and output. All programming languages, at their core, deal with input and output.

I don't see why one node-based system could not work with more than one programming language at the same time (that is an interesting thought). These languages are interfaced together via this node based system. All the rage then would be a node for each language. The system is extensible.

Again, I haven't played around with UE4 as much as I will be (I am working on feeling out the Godot Engine right now), but I am excited about it, and I already see its promise.

Great replies here too. A lot to consider.

Node Oriented Programming?

They call me the Tutorial Doctor.

I am now wondering how I would simulate a node-based system in Python.

Python is littered with frameworks for this sort of thing, to the point that it warrants a top-level wiki page.

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

Thanks for that link. I was JUST looking into it. I have another keyword. "Component oriented."

I will check it out.

Edit: that is funny. I was just on that same page. haha.

They call me the Tutorial Doctor.

I think I just discovered that uneasy feeling about visual scripting. It is sorta scary. I am trying to learn how to work with UE4, and all of the tutorials show the Blueprints system for scripting. If this get's easy enough, perhaps people will loose site of textual programming, especially beginners.

Although, at the end of the day, it is really the logic that matters. I guess the focus would shift from knowing the syntax and features of a language to understanding the logic of what you want to create. Even then, what we really want to do is MAKE something. Where we might miss out is the understanding of how the underlying system works. Again also, I don't really care about the underlying system of the 3D modeling package I use when making a 3D model.

They call me the Tutorial Doctor.

This topic is closed to new replies.

Advertisement