Walkers are also septs of MacNaughton and MacMillan. In the case of the M'Millans the name refers to working (or wauking) wool, prior to its being spun into yarn for weaving. Two centuries ago "Thomas MacMillan" could have been called "Tom the Wauker" by his neighbours and friends which, in the course of a generation -- or less, would have become "Tom Walker" in English.

I personally wouldn't wear Robert Walker's tartan-- I'm not related to him nor do I owe him any personal allegiance, two things which are certainly implied by the wearing of a clan tartan. If you know where in Scotland your family came from I'd go with that district tartan, or join one of the clan societies which recognize "Walker" as one of their own, and wear that tartan.