Ghulam Ali started singing for Radio Lahore in 1960. Along with singing ghazals, Ghulam Ali composes music for his ghazals. His compositions are raga-based and sometimes include a scientific mixture of ragas. He is known for blending gharana-gaayaki into ghazal and this gives his singing the capability to touch hearts. He beautifully sings Punjabi songs too. Most of his Punjabi songs have been extremely popular. Though from Pakistan, Ghulam Ali remains as popular in India as in Pakistan.

He entered Bollywood with a Hindi film song Chupke Chupke Raat Din in B. R. Chopra's film, Nikaah. Other popular ghazals include Hungama hai kyon barpaa, Awaargi, and Yeh dil yeh paagal Dil mera.

On being questioned about Pakistani pop groups, Ghulam Ali replied, "Frankly, I am really bewildered at their style of singing. How can you sing a song by running and jumping around the stage? The stage is meant for performing not for acrobatics."

Ali has also sung some beautiful ghazals like Kina kina timro tasveer, Gajalu tee thula thula aankha, Lolaaeka aee thula and Ke chha ra diun in Nepali language with Narayan Gopal, a well known Nepali singer, and composer Deepak Jangam. These songs were compiled in an album entitled Narayan Gopal, Ghulam Ali Ra Ma, and are popular among Nepali music lovers to this day.

One of his memorable concerts was at the Taj Mahal.