As Sean says, as a programmer, its good to learn new languages for the reasons mentioned in previous posts.
for day to day game development you only really need to know the languages and libraries you're using at the moment.
you seem to be off to a good start, and seem to be getting a handle on python and pygames. i'd continue with that until you felt confident enough to start checking into the next level of development, whatever that is for you (C++, unity, etc).
in school i learned all kinds of languages (perhaps as many as 10-20 by now). for games, all i really needed was C++ and directx. and some PHP for a shopping cart software CGI script.
as others have said, if you find a library is not working for you, figure out why. either you'r not using it in the intended manner, or it can't do what you want, or the docs suck so bad its practically useless. only in the latter two cases do you need to switch to a different library.
when you feel confident, you might want to take another stab a SFML. i don't use either, but SFML seems to be more popular than Allegro.
and if you get stuck, that's what gamdev.net is for!
there's a fair number of SFML users here i believe. at least i see it mentioned pretty often in the threads.