On foot’ is more commonly used. It’s what I use, too.
But don’t worry. You won’t be misunderstood. And that’s what’s most important.
We use ‘by‘ when we talk about other means of transport — by bicycle, by tram, by train, by plane, by car, by rickshaw …