Waterfall or agile? That's the first thing to decide :)
Tip: Agile is the correct one.
Next, how many times you have failed already? It changes things, unless you are overgod (there are some people like that), you will fail several first projects. So for first time failure the critical part is a strict deadline (if you are pressed on time), since you want to fail fast, early and a lot (on the other hand going on with a long project doomed to failure gives you some additional experience). After you finished your first game you can give yourself some more slack (but not too much :D).
Basicly, determine your time budget, this is the most important aspect that will affect the whole design process.
Next, get a decent solid idea before you start (it's highly recommend to talk about it with others - not your team). You don't want to invent core features after you started the project (I make this mistake over and over again and always pay for it :D So, don't be like me :)).
Then determine your financial budget (how much money you are willing to put into it before you get any sales - if that's your first project set it at "0" :)).
Once these are covered I start the project:
1) I write down all core design goals and explanation of some mechanics (not all). Also I research the "marketing" part (who will play my game, track appropriate forums, websites of that genre, fan groups, etc).
2) Then I decide on release date (extremelly important!) and the price.
3) Then I make a prototype.
4) If prototype is good I proceed, if not the project is scrapped.
5) I decide on some sort of overall shedule, at least the milestones like alpha, beta, crowdfunding/greenlight (if needed/planned), final release, etc. Also look for contractors (artists in my case).
6) Then I make the game (if lucky :D) adjusting it/redoing as needed.
7) If the game was not cancelled by now I do the release (and all alpha/beta/whatever). And... go on vacations :D
8) Then, after release I end up in a hell, bug fixing, missing urgent features, etc, etc. All the usual pile of stuff that pop up after you thought you made the game.
9) After like 6 months it all calms down and I only maintain the game and can proceed with another one.