The authors are trying to construct a real-time vital sensing system during exercise where humans wearing sensor nodes move quickly and their density becomes sometimes higher. In this case, existing multi-hop networking using RSSI or GPS to gather vital signs exercisers may not work appropriately. To solve this problem, the authors are proposing image-assisted routing (shortly IAR) that estimates the locations of sensor nodes by image processing. This paper proposes a tracking scheme with error correction based on color information, which is indispensable for IAR. Experimental results using actual images taken from a UAV showed that the proposed scheme achieved accurate tracking using only simple operations without sophisticated state estimation and computationally exhaustive deep learning: MT reached 100% by the proposed scheme.