Most game engines have the option to create or open a project and if I remember correctly these engines have a build system(?) which I believe generates the project files I'm not entirely sure how this works, but I thought of a potentially simpler alternative which I wanted to know if it would work.

Basically, could I not just create an empty directory that will store assets (scripts, meshes, textures, etc) so within the engine when I load a mesh or script it gets saved to that directory and essentially this directory is just a database that the engine reads from so if I wanted to open a new project I'd just plug in a different root directory and then I'd probably also have some sort of serialization so that I can load save the state of the scene and load it back.

Yes, that's a thing that is done.

Normally the engine detects the naked file, notifies the developer, and creates metadata about it for how the file is to be processed in the future.

That is exactly how game engines work. Once you select a “root” folder, the engine indexes all files to show within the editor's file viewer. If you proceed to create a scene and drop two objects, somehow the engine needs to remember your game configuration in a … file. (Everything is a file, even a database. It doesn't really matter from an abstract point of view) That file, simply explained, could look something like this:

<Game name="my game">
   <Element Id="XYZ1" Path="/Assets/Tree.xnb" X="0.0" Y="0.0" Type="Model" />
   <Element Id="XYZ2" Path="/Assets/Grass.xnb" X="0.0" Y="1.0" Type="Model" />
</Game>

You could also replace Path inline with the binary itself, such that you don't even need the physical files anymore. (If that is what you mean with “also have some sort of serialization so that I can load save the state of the scene and load it back”.)