I used to do a bit of sailing and a lot of fishing in small boats. I had one of those Kayaking type life jackets because I had a Kayak as well, but I never wore it. One cold November day, I went fishing in my small sailboat with a friend. It was really cold, so I put on the lifejacket as a body-warmer under my PVC jacket. I was wearing wellies and PVC over-trousers as well.
You guessed it. On the way back, I was taking down the sails when my mate mis-steered, which caused me to get knocked into the sea by the boom. I would never have been able to swim with all that gear on, and even with the lifejacket, it wasn't easy to keep my head above the water. That was the only time in 30 years of regular boating that I went overboard, and the first time I ever wore a life-jacket. I guess that used up all my luck, which is why I never win the lottery despite helping loads of grannies across the road even when they don't want to.
It's the same with biking, most of the time, nothing dangerous happens, but anything can happen when you don't expect it. You have to weigh up the risks and consequences against the inconvenience of wearing one.
When I was a kid, biking was more common, but there were no helmets then, and nobody ever spoke of the risks, other than basics of competency in traffic. There were no cycle paths then either, and you definitely were not allowed to ride on the footpath. Despite all that conditioning, I wear my helmet most of the time now, partly to look like a cyclist to avoid being stopped, partly for warmth, and not so much for safety.