That is the right way to do it and will promote the longest battery life (of them both together) as well.
And only charge when at the controllers at the point of cutting off power to the motor, or just before - assuming some SOC indication is available. Remember, batteries have a set number of "good" charges, before decline sets in, but the number is only (possibly!) really known (if you kept count), the day the decline sets in!!
I take the 2nd battery with me when I can guess that the first one may run out of power before I am back home, or for long journeys, then I simply charge both beforehand and take them both, even if one is not fully discharged.
But this is really only practical, if the OP has a bike that allows Batteries to be easily and quickly swapped - some bikes simply don't make that easy......and then I wonder just how often the batteries will get swapped, and as someone else already correctly pointed out, that is simply not good for a battery to be not in regular usage. They are way too expensive to leave lying around.....
I hope this helps the OP.
regards
Andy