Well, you have to look into related skills. People have to walk everyday so they will maintain strength of their legs to a certain level. Therefore, they can still ride a bike after many years of not riding because of the same muscle group usage. Skills are the same way. The finer levels come from the minor muscles that people do not usually use, so these minor muscles weaken, but the major muscle groups can still function. The person can, thus, still use the ability but at a significantly lower efficiency. Another way is they can provide the same performance at a higher energy/time cost than when they are at the peak.

The degeneration should go until a set minimum
Set the minimum to between 1/10 to 3/10 of the max level achieve.

Okay, in the States, the average person performs 8th grade level. The average person has experienced 13th grade; therefore, the average person loses only 5 years worth of education but no more even though they don't use that much academic skills. The ads bombard at the citizens, and the local newspaper are typically at 6th grade level, so it is impossible for the citizen to degrade to below 6th grade level. So what keeps the people at 8th grade level? Applications do because applications are required to maintain a 10th grade level readability. With normal performance at 6th grade level and special performance at 10th grade level (during paperwork), the average performance is somewhere in between and that is 8th grade performance.

The truth is that there are many ways to use the same skill set without even spending time one those skill sets because they are implied or simultaneous with other performances. Just take a walk down any US cities, and in one day you will get more than 3000 ads, each will give you experience towards 6th grade level reading skills. But wait, you're only walking down the streets. So welcome to the real world where you can gain experience without knowing.

How to properly use degeneration:
You must make sure the degeneration is slower than real life degeneration or your fanbase will leave.
You must make sure other related actions account towards maintaining the skills.
There should be implied relation between higher skills that requires lower skills; therefore, within skill hierarchy, the skills at the higher levels must degenerate before the skills at the lower levels degenerate.
For example:
people will lose their multiplication skills before their addition skills.

edit:
off-topic
the real world functions differently. Like I have shown, the ads give you 6th grade reading skill experiences so it can only pull you from 6th grade reading skills towards but it does not reach 7th grade reading skill level. For the same way, your experience should also be a one way (single used) ticket that cannot be given once the player should reach another level. Like a theater ticket, you can use it to watch a movie, but not the next movies. Oh, this is just an idea to keep the experience from inflating. A game too easy is boring, so this idea will come to help you keep the difficulty level from getting to low.