Does Bullet provide it's own collision response or do I have to do this?
Bullet does the collision response as well. You just add things to the world, step the simulation then grab back the new positions.
Also, I would like to modify the Center of Mass of my cylinders, can I do this with Bullet?
Can't help on that one, not sure. The documentation would be the best place to start I guess.
Another Question: does bullet use column or row major matrices?
IIRC Bullet's native matrices are in whatever OpenGL uses. I was using Direct3D but literally just Googled for "Bullet matrix D3D9" and there was loads of sample code for how to convert.