@vilem otte Thank you for the technical details.

I am not claiming current state of AI is satisfactory, I claim it is very promising.

If you manage to create a good network that is not overfit to only the tracks that it has been shown, it would be able to drive on unknown tracks. It does "think" inside(to say it in a way). Every single track in this universe has borders we don't want our car to cross. It could be walls, or rocks, or a lake. But this rule - “try to not cross the limits” is shared among all the tracks and a NN can learn it. Then it will learn another thing that is shared among all of the possible tracks - “the faster you go, the harder to turn” etc. A NN that is not overfit(or underfit) actually “thinks”.

It is still a lot of manual work to create a good NN. But i think the field of ML/AI is very promising.

NNs indeed do show signs of intelligence. It is hidden in the hidden layers. NNs can figure out features that the developer did not see. This behavior is a candidate for some kind of (non general) intelligence. Then what is intelligence… the job of a cat is to meow…

A good setup of NNs would be able to play various games. I mean, that goal of generalization - to be trained on tetris, and to play mario. It could be perfectly done in theory. It is just that NNs are a field in development yet. I talk about complex networks that are already available. And i can not setup them now. I am still learning. Having the perceptron as the base, they already offer very advanced setups of various compicated NNs working together.

What i don't like the most from NNs is - they still require lot of human attention. Lot of human guidance.

Theoretically, you can just throw the input to a network and it should learn anything.

Then, in practice, we are limited by computational power. That's why even the big corporations have to use CNNs, because even their supercomputers can not handle the computation. In practice, somebody trains a network for days and it results it is less effective than the network he trained the past week, and needs to restarts the training process again.

In practice it is meh, but theoretically, not sure how soon, we can just throw any input and the NNs will learn it. If we had enough computational power ofc. If there is some rule or hidden feature to be discovered, a NN surely should figure it out, again - in theory. In practice, the process is not so exciting.

My understanding is not reaching RNNs yet, but many researchers claim RNNs could become turing complete… if you manage to train it well.

About the need for vast computation, there is research being done in using analog hardware for Neural Networks. This could skyrocket the performance of NNs if they manage to introduce such hardware.

Right now, NNs/AI are not a miracle. Mainly because an expert needs to guide them constantly. But i think in the future, maybe a non expert would be able to train a NN to simulate any program he wants.

Right now not yet. Still i have to code manually ten years more for my traditional project, and in 10 years who knows how advanced NNs would be.

(imagine i give you eternal life elixir to drink. Then i tell you to manually create a NN that drives on any planar track. You have to manually slide every single of the weights of the network and manually decide how many layers, what connections and how many neurons to have inside. It would take you 1000 years to manually do it. But in the end it will drive on any track and it will be a NN “without an engine”. An absolutely perfectly trained(programed) NN.
This is why I use the word “theoretically” so often. It is possible for a sophisticated NN to do it, but it is unfeasible to train it to do this. Researchers are working on ways to improve the training and all about NNs in general. I think in 10 years we should see a very big breakthrough)

While my understanding of the topic is much lesser, I agree with JoeJ in that you should just tell us roughly what are you trying to do.

After all those years, you should know exactly how much one can ‘steal’ from random concepts read on the internet. Nothing.

Do go install TensorFlow now. Just do it and play with it. How could that possibly interfere with your long-term work on your shader of doom, at which you appear to be stuck?

pcmaster said:
How could that possibly interfere with your long-term work on your shader of doom, at which you appear to be stuck?

My shaders are unreadable. When i leave a shader finished to start programming another one, and after one month need to peek something in the old shader, it is a huge pain to read that. If i stop now, i would forget what i did and it would hurt to read all that again.

But, more taxing is the change in mindset. It is like when somebody asks in the forum about something about HLSL for graphics engine. I want to help, but it is hard to switch my mind to graphics for a moment then back to compute again.

To say it in a way, in my case, personally in my case, if i code one year OOP and then put that project aside for 2 months to learn some functional programming language and make some project in it. Then, after two months of doing functional programming, if i have to get back to my old OOP project, it would hurt to switch again to a different paradigm. Just saying that as an example.

ML is a big mental switch. It would take time to get back and forward. At least for me. The way to think changes too drastically.

I think i should dedicate full time to ML in order to learn. The basics are fine, but more sophisticated NNs become very complicated pretty fast. I would need 24h of dedication to learn the more complicated stuff.

I think, when i finish the current shader i could try TensorFlow. The moment between shaders is the only moment i could try something different. I hope i will finish this current shader sooner than later. Added to the parallelism, is the difficulty of the API. I feel like i am coding triple more code because of shaders. I am blind to what happens inside the shader at execution time. Most of the operations i perform inside the shader i need to flush their result to some resource. Then inside C++ i read that and repeat the same operations, then compare the results to see if C++ thinks the same way as HLSL. I find as many bugs inside the shaders as inside the testing code. It is more complicated than it sounds. It is very time consuming. I desire to switch already from “add tests for anything i do” to “ignore what happens inside the black box” paradigm. If it is even possible for me to ignore the content of the black box. It depends from one person to another.

I try to motivate myself with some interesting project. But there are no such projects for me for now. I watch somebody talking about recognizing cats and dogs. I have free image databases to train, and even free GPUs to use in the cloud. These free GPUs are limited, but still much better than my laptop. When i listen to the lector explaining NNs, i don't feel motivated to try it myself. I don't know why. In the beginning of my life as a programmer, it was interesting for me to repeat some tutorial and say - “see! i repeated it ans shows the same result”. I am not motivated to repeat cats recognition. The lector said how it works and i believe him. I don't see a need to put it on a test. The big data i can access for free is accessible to anybody. At least for now, i fail to find some project that can use the free big data and to be something different than the things that anybody else does. I need to feel motivated and do something that is interesting for me. Playing with a 2D agent moved by AI in a simple map looks both doable with the resources i have, and looks interesting to me. But this only when i finish the shader from hell.

I am only talking. I felt depressed about it and wanted to speak it out. Not asking nobody for help. Somebody could suggest to me now some interesting thing to do with NNs, but i can do it only when i finish the shader. And that somebody could feel offended. I am just talking now.

I'll be frank, in Central Europe we never walk around the topic.

You seem to have some serious work organisation problems. It isn't a shame, if it can be worked with. Have you ever talked to anyone about how to alter your approaches so your code is more maintainable, easy to return to after a longer time?

I find it hard to believe you (or anyone!) would have any problems returning to the knowledge you've been accumulating over the many years, helping others, and yourself. Your skill won't go away that easily, you'll dust it off. Can you talk to someone about this fear?

While replicating what those introductory TensorFlow tutorials show, you will not only replicate their results ("Ha, it recognises shoes for me, too!") but you'll make adjustments which will reveal to you, and it will gradually force you to learn and experiment, and gain insights on how it works under the hood. Baby steps.

We should only treat certain things as a black box and never insist on remaining ignorant to how they work. Surely in the beginning you totally want to do that, and it's fine. But remain open.

Duuude, after reading what you wrote, it almost sounds like a wannabe script kiddo coming here and insisting he doesn't need to try anything smaller than jumping at making his dream MMO-XYZ-RPG as his first project – why bother with small things, aren't small things for the plebs? :P Forgive me :P

NikiTo said:
Theoretically, you can just throw the input to a network and it should learn anything.

Given unbounded space and time, perhaps. In practice the complexity of a NN is tightly bounded by those two resources. A human takes more than a decade to learn about the world - an unstructured NN given data soup is unlikely to learn particularly much more efficiently than we do.

NikiTo said:
I am not motivated to repeat cats recognition. The lector said how it works and i believe him. I don't see a need to put it on a test.

When learning something new, reproducing a basic example is the way to go. And if you are interested on the subject, reproducing the correct results feels motivating and rewarding. It's a confirmation of your progress of learning, if (and only if) you understand how it works.

If you don't feel that or are not interested because the example is basic, trivial and nothing new, then you have no real interest on your own contribution on the subject. It may be you are interested in AI for it's goals, but you are not interested in knowing how it works.

It may also be you consider NNs as an alternative route, because you realize your own but different approach is going to fail. This makes you tired and explains why you are not really willing to learn state of the art NN aproaches.

That's my personal guess. Because you do not answer our questions about what your work can actually do, i can only conclude it simply does not work so you have nothing to tell or show.
You could at least say what you try to do. If you think that's already something worth to keep secret, you simply deceive yourself.

NikiTo said:
Not asking nobody for help.

I think you do, indirectly.

I hope i'm wrong with my assumptions. It would be so easy for you to disprove if so.

Regarding your programming problems, i know them all. Guess we all know exactly what you mean.
My proposal is to do research on CPU where we can debug, maintain and manage much easier.
The speed up of GPU to CPU can't be high enough to say that's no option.
After it works, port to GPU. But i proposed that long before.

EDIT: Multiple people here say the same thing, independent of each other, without reading the cross posts.

What does that tell you?

pcmaster said:
You seem to have some serious work organisation problems. It isn't a shame, if it can be worked with.

This is something that happens very often. It is a very know problem of “i can not read my own old code” that many people have. I try to organize well, but it is not easy.

pcmaster said:
While replicating what those introductory TensorFlow tutorials show, you will not only replicate their results (

You are right.

pcmaster said:
We should only treat certain things as a black box and never insist on remaining ignorant to how they work. Surely in the beginning you totally want to do that, and it's fine. But remain open.

People usually do not try to peek inside the black box of NNs. Ofc there are people who work on this too - a way to peek into the unreadable guts of NNs. This is how DeepDream of Google originated. I try too, and i always try to understand the very basics of NNs, but a trained network is very hard to peek inside. People work to find ways to peek better inside the black box of NNs. I mean, you could know exactly how a NN works in the very bottom base. Then you train a NN and can not understand what is happening inside. Not quite. These are two different things - understanding how NNs work and understand what the weights of a trained network do.

pcmaster said:
:P Forgive me :P

No problem. It could be that i sound as a newbie here, because i am hurrying up. At my age, when possible, i try to skip over many steps. But you are right too, that even a basic tutorial could help. Working with agents on a map is like a game. Could be fun and more bearable. I would fall in deep depression if i had to work with numbers for finances. And i can not develop something to sell it to Waymo. People who are developing autonomy software for a German car maker said on a video that they would perfectly need millions of kilometers of driving video data recorded. I can not develop at home such an exigent project. Training a NN to sum two numbers is a too small challenge. A simple silly looking 2D agent could practically introduce me to RNNs. And i speculate that the GPU of my laptop is enough for such a small silly looking task.

@joej I agree i have psychological problem about premature optimizations. And i am not ashamed to say i failed at my current/last project. It is a failure of not being able to finish it. I believe when i finish, it will work.

I could try to switch to the CPU at least part of the workload. And later pass it to the GPU. I wait 15 minutes in total to test the result every time i make a minor change in the shader. So i could accept the CPU taking 10 minutes to compute. I need tests to know that. When i finish the current shader.

Does not sound your premature optimizations were enough. We have cars which do CV in real time.

Anyways, i see you do not consider to give up, and i got the thread wrong.

JoeJ said:
Anyways, i see you do not consider to give up, and i got the thread wrong.

The problem is not well stated and leads to a lot of speculation.

If I am right, the OP wants to know if he should continue programing a GPU compute shader that will cure cancer or if he should abandon that project to learn how to use machine learning to write the shader that will cure cancer. He doesn't want anyone to share in the glory of curing cancer so he keeps relevant details hush-hush.

Is this accurate?