Personally, I would fuse it at the input side only, bit it won't hurt to do both sides.
As far as fuses go, assuming it's 36v down to 6v, then you will have 36/6 (=6) times more current on the 6v side. So if your lights total 1a vfor example) at 6v then a 2a fuse on the 6v side.
On the 36v side a 1a will be good enough.
As an aside, when thinking about fusing it important to consider the cable capacity. No point having a 10 amp fuse if the cable melts and catches fire at 5 amps.
As far as fuses go, assuming it's 36v down to 6v, then you will have 36/6 (=6) times more current on the 6v side. So if your lights total 1a vfor example) at 6v then a 2a fuse on the 6v side.
On the 36v side a 1a will be good enough.
As an aside, when thinking about fusing it important to consider the cable capacity. No point having a 10 amp fuse if the cable melts and catches fire at 5 amps.