My Boardman hybrid mx/cx 700c day out bike is rear hub Aikema 128c has a downtube battery and a 91 cell rear rack/topbag battery , handling is only compromised if tackling gnarly slow tight twisty wooded sections. Other wise handling is good in the woods, open trials and on the Downs off piste.
My 700c commute/town bike has a Bafang cst with down tube battery, handling with panniers full of shopping is a different game and can be awful if it is weighty.
For a rack battery to cause handling to be really bad is has to be badly mounted or mahoosive. For rear rack the rack needs to be as far forward and as low over the wheel as possible.