Experimental modal analysis is one of the key technologies in structural dynamics analysis. However, in cases involving extremely high or low modal damping, it is difficult to accurately identify all the modal parameters. In particular, for systems with extremely low damping, there may not be sufficient data to allow curve fitting in the vicinity of the resonant peaks. To overcome this difficulty, we propose a linear fit method of modal parameters on a new mapping plane. This method uses a basic equation linearized from the nonlinear equation of the frequency response function (FRF) by erasing the residue, which is a modal parameter. Then, the basic equation becomes linear on a mapping plane related to the ratios of the real and imaginary parts of the FRF. The linearized basic equation can identify the modal parameters of a vibration system with extremely low damping. It was observed that the influence of the measurement noise degrades the identification accuracy of the linear fit method. Consequently, it was confirmed that the identification accuracy deteriorates when data with low coherence and far from the natural frequency are used. Thus, a weighted least squares method using the coherence and Gaussian kernel function was proposed for the linear fit method. Finally, the modal parameters obtained using the proposed method and the conventional least-squares complex frequency (LSCF) method, from the FRF including noise, were compared, which indicated that the proposed method can produce estimation results with an accuracy comparable to that pertaining to the LSCF method.