Traditional PV power interval forecasting relies on specific probabilistic distribution assumptions, which often result in inconsistencies between the assumed probability distributions and the actual heteroscedastic nature of PV power distributions, thus affecting the accuracy and confidence level of interval predictions. To address this issue, an ultra-short-term PV power interval forecasting method based on time-series decomposition and conformal quantile regression (CQR) is proposed. Firstly, the PV power series is modeled as the sum of three additive subseries: trend components, periodic components, and autoregressive components, based on the NeuralProphet time-series decomposition framework. Then, piecewise linear models, Fourier series decomposition models, and AR-Net models are respectively employed to fit the three subseries, with the Fourier series decomposition model enhancing the fitting capability for daily and seasonal periodicities of PV power. Finally, by calculating the prediction uncertainty of the CQR model, the quantile interval of the prediction results are determined based on conformal scores, enabling dynamic adjustment of the prediction interval width without the need for preset probability distributions. Case studies demonstrate that the proposed method outperforms the advanced Transformer-based algorithms like TimesNet and Informer in deterministic PV power forecasting tasks, and with the introduction of the daily and seasonal periodic components, the prediction error is further reduced by 11.65%. In interval forecasting tasks, the proposed method surpasses the traditional quantile regression algorithms in terms of prediction interval coverage rate, normalized interval width, and coverage width-based criterion.