Hi!
To use OpenGL, you need a window to draw to. This means you'll always need some kind of window manager. Either use the OS's native one, or a cross-platform one, like SDL. The benefit of the latter is that it abstracts away a lot of platform-specific details you won't have to bother with, and a lot of work is already done for you.
If simply not using SDL or any other "library", you could do fine with using a native window handler ( WinAPI for example ). Technically that's another library, but I think that's okay, you can't really go lower-level from there.
Now, I'm not entirely sure about this one, but technically you could do "without a window" on some Linux distros when no GUI is present. Then again, you still need to request a surface, which is pretty close to using a window manager. I could be wrong on this one, and I don't advise going this route.
On the other hand, if SDL seems like an overkill, try GLFW. It handles windows and input to some limited extent. You'll have to do anything else by yourself.