Build a golden palace ?
The nice thing about gold is is that it is so shiny, so people value it highly just by looking at it.
Same thing for silver/diamonds etc.
These all being rare helped alot as well, but that probably became important in a late age.
ah yes, the classic "treasure room" answer. i had an archer/smith/enchanter in skyrim that was the head of every guild, and would craft dual enchanted ebony bows with his money. he had a pile of almost 1000 of these by the side of heirlarchen hall.
gold, silver. diamonds etc are represented in caveman as "cool rocks". this could be anything from a gold nugget to an amethyst geode to a piece of pyrite to rolled hematite -anything that looks cool and unusual. they are hard to find, require a lot of stonelore skill, and have a high trade value, but are only used for sacrifices to the earth god Gah.
"treasure room" is a viable money sink in an RPG, and a fondly remembered feature of dungeon keeper. IRL i was contemplating one before the dot com blowout ended my early retirement. but i find that it also tends to be a rather hollow pursuit, especially when you've accumulated a large number of treasures. the thrill from that first gold bar is great. the thrill from the 1000th gold bar is almost non-existent unless you're one of those obsessive-compulsive 100% completion types.
but things like treasure room, achievements, and other such "bragging rights" type things might be all that's possible.