IMO, unless you don't have time to put into doing input handling right (and then most probably you run into problems later anyway when people start complaining about not being able to re-assign the keyboard keys and mousebuttons), always cater to KB/M AND gamepads at least (driving wheels and joysticks make less sense for a 3D TP View game, but then again these are just other types of input devices).
Ideally your game is agnostic to WHAT people are using to input their commands with... it just recognizes the input devices, and gives players options to map actions to the buttons these devices support.
The only real complication comes with handling analog axes or the lack of these in certain input devices (like the Keyboard), so you might need to invest some thought into how to handle things that could be assigned to an analog axis on a gamepad (Which gives the player a range of analog states to control the axis) when the same axis is assigned to keyboard keys (which either means fixing the keys to always mean "max analog state", or work with different keys for different states, or multiclicks for moving between different discreete approximations (like what World of Warships has done with full speed, 3/4, 1/2 and 1/4 speed, W and S are used to switch between the states, going from 0 to full speed takes 4 taps on the W key for example).
Other than that, you need to think a little bit about the amount of actions that need to be mapped, but when leaving input devices like Joysticks and driving wheels out of the equation which sometimes still have not that many digital keys, modern gamepads have an overabundance of digital keys to use, so unless you want to give the player a quadrillion of abilities to map to different buttons, you can ignore that point.
If you do, you need to find ways to have that same set of actions work on a gamepad with "only" about 8-10 digital keys, plus 2 if you use the Trigger axis as 2 additional digital buttons, ignoring the analog aspect of it.
Generally, I would concentrate my efforts on a re-mappable Input schema. It will be of use to even hardcore M/KB fans, so everyone wins.
Then create default mappings for M/KB, and Gamepads, and see if your game works fine on both.
To all the guys saying "PCs don't need gamepads"... well, unless you are a weirdo like me that has a console-replacement PC hooked to the TV. Many games on Steam didn't get my business because I wanted to play them on the big TV, but the game does not support Gamepads, and I couldn't be arsed to use a keymapper to create a crappy version of a gamepad mapping for the game because the game doesn't support analog axes, at all... even though the console version does.
Going to buy a PS4 Pro soon, partially to play the games that just got shoddy PC ports without Gamepad support. Rather play the PS4 Pro version that forces devs to fully support gamepads natively.
So there is your reason why you should care about gamepad support, even when developing games for PC exclusively. And its not like its rocket sience as soon as you have remappable input mappings in your game.