good question. The answer is in between your two suggestions there. Your fitness would decline if you did not stay active in any way but you wouldn’t have to worry about anything major.
One important thing to consider though it diet. To play a lot of sport requires a lot of energy and therefore, a lot of calories. If you eat loads because you’re training all the time, you burn off all these calories. If you then stop training but keep eating loads, you would put on a lot of weight. A diet should always be dependant on how much energy you use.