I know you've said its not about the speed, but I think I can give you a few examples that I think might help.
Its not about the speed you're going, its about the speed the drivers expect you to be going, and how much room / time to give you.
Your riding a "normal" looking bike, in normal...