A method of graphical color scheme is proposed for people with visual handicap, such as color blindness or low vision by aging, considering design property. This method first sets the contrast of luminance between adjacent regions to be large in order to make the image understandable only by the luminance, and then assign fine color combination in order to make the image satisfactory to people with normal sight. In designing the color, samples of well-known fine color combinations are adopted. Here, the user can select his/her favorite color and reflect it to the design by adopting color combinations containing the color. In applying the color combination, the color information of each color in the combination is assigned to the region in the image considering the luminance and the area of the region, so that the user's taste to colors can be reflected adequately. Both the luminance and the color information are set at preferable values by interactive evolutionary computing (IEC) which can design the system considering the user's subjective criteria. Computer simulations verify the high performance of this system.