I had almost exactly the same experience. I was riding through the town park on the main cycle route on my way to the local electric bike shop. The whole journey is only 5 miles and slight downhill all the way. I ride my road bike most of the time, except when I want to go shopping on the way home, like this time, but to keep my fitness up, I pedal without power whenever I can.
So here i was going down the cycle route at about 15 mph, when I came across about 4 people on bikes doing about 10 mph spread across the whole path. I rang my bell a couple of times and shouted "coming past", which had them moving to the side, so, to cause them the least inconvenience, I started pedalling hard to get past them as quickly as possible. Once up to speed, I eased off a bit, but I was still doing about 18 mph. Next thing, one of the the guys is alongside me saying, "How fast does your illegal electric bike go then?". I told him" not so fast when it's switched off, like now" and pointed to the blank LCD. I'm not sure whether he understood that, but the rest of his gang were left behind, so he slowed down again to let them catch up.