I'm being plagued by a desire to make a game where the player has a level editor that allows the player to draw a 2D level map and then it will pop up into a 3D level. Of course that sounds much like a height map, but sadly I have ambitions beyond what a height map alone can offer. I want the player to be able to draw a curve on the map and have that curve become a vertical cliff. I want the player to be able to draw a thick line and have that line become a road, its vertices lined up with the vertices of the surrounding landscape, but horizontal from side-to-side and with UV coordinates set to allow its texture to follow the direction of the road. If the player draws a road across a chasm, I want that to become a bridge.

That may seem like it's asking too much, and it's true that for as long as I've been thinking about this problem I have yet to find an approach that works to my satisfaction, but there are limits to the goals of this project. Just like a height map, this project doesn't attempt any sort of cave or overhang. The final level needs nothing that cannot be represented in a 2D map. Aside from bridges, no part of the level ever needs to cross over itself. Aside from vertical cliffs, the landscape is restricted to being smooth slopes or flat land; there is no desire for the kind of jagged detail that's possible in a height map for this project. Aside from the cliffs that are specifically drawn in the level editor, there should be nothing blocking the player from moving around the level, so everything except the cliffs ought to be relatively smooth.

I've tried starting from a regular mesh of equilateral triangles and adjusting the positions of the vertices to match the player's map. I appreciate the regular mesh because it makes it easy to give every vertex, triangle, and edge a number and store the level in an array. It also forms a graph structure that makes it easy to create smooth slopes and know when those slopes ought to be interrupted by cliffs. Unfortunately I have never been able to overcome the technical challenges of making the mesh and the player's drawings line up.

I've tried starting from the player's drawn map and building a mesh around it. Unfortunately, computational geometry has never been one of my strengths, so figuring out where to put the vertices and edges to smoothly fill out the rest of the map is daunting. I've considered simulating the vertices as if they were electrons so they can form a minimum energy distribution around the fixed vertices specified by the player, but I'm not sure how to maintain the smoothness of the slopes if the vertices keep moving as the player draws.

The bottom line is that I'm really not sure how to even begin solving this problem. I'm willing to put effort into implementing a complicated system, but first I need an idea for how that system ought to work. I really need the wisdom of someone more experienced than myself.

Have you considered that perhaps a 2d image isn't the right representation of a 3d map?

I wouldn't do it that way, and if you look at most 3d level editors they're 3d in nature with a three coordinate system x, y and z, so that it's easier to visualise height and depth and adjust in real-time.

What you're asking for means the level designer can't visualise in real-time what they're creating which makes the creation process awkward.

Hope this helps!

On the editors I've worked on like that, there are layers.

Generally they start with the terrain's heightmap coupled with layers for viewing items on the terrain. That can viewing terrain's graphics texture, or viewing tree, grass, or water layers. It can mean viewing markers for navigation information, and so on.

One layer view can be items placed on the ground.  A level designer might place a building, or spawn points, or bridges as you mentioned.  Some objects could modify the heightmap if necessary, as defined by code and data. The system might say a building must have flat heightmap and level it out around the building's base.  The system might say a bridge needs to have a flat heightmap at both endpoints and must be equal height on both ends, and automatically adjust the heightmap accordingly.

Most major engines have moved away from that style of map editor. There is often more to game maps these days so level designers want more comprehensive views of the world.  

Have you looked at the starcraft or warcraft3 map editors? I think you might get some ideas, mainly doing with frob's suggestion of layers. I think age of empires 2 has a cliff system similar to what you describe as well.

The motivation behind this level editor is not for level designers, but rather for players. By that I mean I want the level design process to be fun and easy, not something someone would do as a job. That's why the fundamental goal for the level to be designed as a 2D image that the player can paint with the mouse. I'm trying to deliberately simplify the level design process and the resulting levels. So there are only three times of land: flat land, smooth slopes, and impassable cliffs. I'm trying to keep the third dimension simple, but still leaving room for depth and creativity.

1 hour ago, Brain said:

What you're asking for means the level designer can't visualise in real-time what they're creating which makes the creation process awkward.

I like to think of it as saving the level designer from needing to visualize the third dimension. In an outdoor setting, most of the interesting design choices are going to be horizontal. For example, the main gameplay impact of a hill is to block horizontal visibility when you're standing beside the hill, and expand horizontal visibility when you're standing on top of the hill.

49 minutes ago, frob said:

There is often more to game maps these days so level designers want more comprehensive views of the world.

I'm trying to keep my game map deliberately simple. Solving this technical challenge is a learning experience, but creating elaborate game maps would require a team of 3D modellers. I feel like compelling game maps can be created without detailed models just by putting thought into what goes into the map,

8 minutes ago, h8CplusplusGuru said:

I think age of empires 2 has a cliff system similar to what you describe as well.

That sounds very promising. Thanks! I will need to take a look at those editors.

Right now I'm working on a computational geometry approach by using Ruppert's algorithm to fill in the mesh details between the details that are drawn by the player. That should cover the landscape with an irregular grid, and then I plan to use cellular automata in that grid to create a smooth landscape in the blank areas where the player hasn't drawn anything.

I think the Starcraft two map editor is what you might be looking to get inspired by, are you familiar at all?

1 hour ago, h8CplusplusGuru said:

I think the Starcraft two map editor is what you might be looking to get inspired by, are you familiar at all?

I've never played Starcraft 2, but I found a video of the map editor on youtube and I see what you mean!

Even so, it has a very noticeable grid underlying its levels and that makes it hard to do natural formations. I could save myself much trouble by using a grid, but I think I'd get much better results with free-form levels. I am seeing great promise in using a quadtree to structure the levels, because that allows it to get more detailed as necessary in response to whatever the player draws.