I am making an open world in Unity 5. But the engine is not important in this thread. What I want to know is how to make a good open world that looks good from every angle.

So far I have a few mountains, a lake, some trees and alot of grass. It's your typical open world rpg with spells, weapons, monsters, and quests. The map isn't on the scale of Just Cause 2 or Oblivion, but it is pretty big.

When I make a mountainous area, it feels too cluttered. When I make an open grassy field with sloping hills, it feels too empty. When I try a bit of both it feels too inconsistent.

What is a good way to build up a fantastical, yet believable world? Is there an order or style?

Grassy fields that feel empty seems like the easiest of these problems to tackle. First you want to vary your grass, such that you have wildflowers or taller grasses with grain heads in some places, shorter green grass, clover, or muddy patches in others. Then you can add a tree stump or a location where animals appear sometimes, or a fallen-apart old fence or stone wall, or a little pond...

Also you may want to check if you are making your mountains too small proportional to the world, that might make it feel cluttered.

The style of a world is something you can research and then establish with a page of doodles or several pages of rough sketches. An iterative design process in general is a good method for getting a consistent feel.

This might help, though admittedly I didn't browse much yet.

http://www.worldofleveldesign.com/

I imagine a lot of your inspiration should also come from games you've played. I'm thinking along the lines of Skyrim. There's a lot to learn there by just looking around.

Adding signs of civilisation helps fight the emptiness.

Do you have towns and villages yet?

Add towns and paths, fences, docks and jetties on the side of river banks, add npcs and before you know it the world will feel more alive :)

You can also do tricks to make it feel like there are people where there aren't. For example in my rpg when it becomes night all the windows of the towns light up with candle light...

Could you share images so we have a better understanding on what you call "cluttered" or "empty"?

Would certainly help spotting what could be the problem in your case.

You'll just have to observe landscapes and try to duplicate what interests you.

Picking the brains of landscape architects seems to very helpful, this was done for The Witness. Luis Antonio, one of the artists for The Witness has done a GDC talk and has an art dump where he talks about their approaches to creating the world which you might be interested in.

So far I have a few mountains, a lake, some trees and alot of grass. It's your typical open world rpg

Yep. Certainly sounds like a typical RPG world.

What's influencing the world's design? I mean, apart from every generic fantasy game you've played.

How's the lore, cultures/civilizations, and the plot shaping the world while at the same time being shaped around the world?

Be careful that you're not just taking all the fantasy games and books you've consumed, and distilling it to their common elements, and just using those common elements, resulting in a cliched and bland world, forcing you to try and spraypaint on top of it some "uniqueness" back in, after having already removed what was unique from those other worlds?

Or let me phrase this another way: How is your world different and unique from, say, Lord of the Rings or [insert your favs here]? I'm not talking geometry. I'm not talking characters. I'm not talking cliched plot or blatantly-ripped-from-mythology-or-real-life cultures.

What (hopefully dozen or more things) makes your world different? Make a list for yourself.

Then, use that list to shape your world's geometry, lore, cultures, people, social laws, natural laws, and mechanics, and so on, while also using all that to recursively feed back into the world's uniqueness again. Your design should influence your design should influence your design.

The map isn't on the scale of Just Cause 2 or Oblivion, but it is pretty big.

Why is it big? Is it big merely to be big, or do you actually have content to fill it?
Are you merely creating a large empty world and then trying to fill it with content, sprinkling it around, making the content spread out and watered down?

Personally, I suggest you shape the world around your content at the same time as shaping your content around your world, letting them recursively influence each other, along with being recursively influenced by, and recursively influencing, the story, lore, music, art, enemies, and so on.

Otherwise, what you get is basically empty geometry with annoying enemies bothering you while you travel, with a poisson-distribution of points of interest with too much "nothing" between, regardless of how beautiful that nothing may look.

I suggest you play Minerva: Metastasis (an single-player FPS mod of Half-life 2 for the PC), and King's Field: The Ancient City (a PS2 first-person action-RPG). I guess a more modern version would be Dark Souls 2 (from the same people who made King's Field), though I haven't personally played that.

These games do an excellent job of hyper-compressing their world, to fit as much content into as small a world as possible. That goes counter to some people's goal of being able to brag about "My world is 50 square miles in area! It takes you two hours to walk from one end to the other!! It's so empty and bland walking though it though, that I had to add quick travel to compensate!!! Isn't that amazing!!!!".

Even if you don't hyper-compress your world as compactly as they did, you can still learn better area design from them, and if you practice hyper-area-compression like this, you could then apply the lessons learned to the level of area-compression that you think best fits what you're going for.

Otherwise, what you get is basically empty geometry with annoying enemies bothering you while you travel, with a poisson-distribution of points of interest with too much "nothing" between, regardless of how beautiful that nothing may look.

I loved just cause 2 and played it to death... But a lot of its map did sometimes feel like this. It was often repetitive due to it's sheer size...

The map in my game is pretty small. To scale with the player it's maybe a couple of miles square and can be traversed in 5 minutes as the crow flies. I plan to make the land difficult to travel in straight lines, and I have tons of content that needs to go on it. The map is basically a small country (think Luxembourg) so the idea of map compression is being used there.

Food for thought!