I'm making this 2D game with SDL & OpenGL, and it goes super extremely fast on my PC. However, on a Pentium II without 3D card, it goes at 2 frames per second. Does this have to do with the fact that OpenGL in software is slow?

yes.

i don't think theres really any way around this. you could write an API independent rendering engine, say using OpenGL and SDL. when you detect theres no hardware, you fall back and use SDL to render everything, else, you render using hardware and OpenGL.

theres one major problem with this. if you do this, this means you cant take advantage of certain features which hardware rendering provides - rotations and scaling being 2 major ones.

i guess it comes to choice. either run decently to super fast on 80% of machines (and get the bonuses of HW), or run at a mediocre to poor speed on 100% of machines. or, you could just not use rotation and scaling and such in your game. i know theres software versions of these (SDL_gfx), but from what i hear they are slow, but maybe it is possible. however, some things you might want just aren't possible with software i think (modulation comes to mind).