The first irrigation works along the Diyala River were undertaken in Parthian times. Indeed, it may be that the lower part of the Nahrawan Canal was originally the lower course of the Diyala. The large-scale canal system of early medieval times however was created in the reign of the Sassanid ruler Khosrau I (531–579), who also established it as a separate administrative district (Bazidjan Khusraw). A treasury and mint were possibly established there.
In early Islamic times, the town of Jisr al-Nahrawan in the middle of the canal was the site of the Battle of Nahrawan on 17 July 658 between Ali and the Kharijites under Abdallah ibn Wahb. Under the early Caliphates, and especially under the Abbasids who made nearby Baghdad their capital, the canal network was repaired and expanded, reaching its peak in the 9th and early 10th centuries. In Abbasid times, the region was divided into three tax districts, Upper, Middle and Lower Nahrawan.
The canal was breached in 937/8, during the revolt of Bajkam against Ibn Ra'iq; the latter tried to impede Bajkam's advance from Wasit to Baghdad by flooding the region in between. The move barely obstructed Bajkam, but succeeded in destroying the agriculture of the region, hitherto the breadbasket of the Abbasid capital. As Hugh N. Kennedy writes, "the breach of the Nahrawan canal was simply the most dramatic example of a widespread phenomenon of the time; and it was symbolic of the end of ‘Abbasid power just as the breach of the Marib Dam was of the end of the prosperity of pre-Islamic south Arabia". The lower and middle Nahrawan were entirely abandoned for almost 14 years, until the Buyids under Mu'izz al-Dawla restored the canal. Nevertheless, the canal network continued to decline thereafter. As late as 1140, the Seljuq governor Bihruz tried to restore it, but according to the 13th-century scholar Yaqut al-Hamawi, infighting among the Seljuqs once again meant the neglect of the canal, and its use as a road by their troops compounded the destruction of the network. By Yaqut's time, the canal network had largely silted up and the countryside along them was abandoned.
In medieval times, the main canal was divided into three sections, described in detail by the 13th-century geographer Yaqut al-Hamawi, in his Mu'jam al-Buldan. The initial feeder canal drawing water from the Tigris at Dur al-Arabaya near Samarra and carrying it to the Diyala at Ba'quba was called al-katul al-Kisrawi ("the Cut of Khosrau"). During its course, it was joined by three smaller canals taken from the Tigris, the Yahudi ("of the Jews"), the al-Ma'muni, named after Caliph al-Ma'mun (r. 813–833), and the largest of the three, the Abu 'l-Jund ("arch warrior"), built under Harun al-Rashid (r. 786–809).
Some 20 km south of the inflow of the Abu 'l-Jund stood the town of Salwa or Bab Salwa (Basalwa), and a little further on the town of Ba'quba, the capital of the Upper Nahrawan district, some 50 km north-northeast of Baghdad. From there, the main canal, now known as the Tamarra, turned south to the towns of Bajisra (originally Bayt al-Jisr, "bridge-house") and finally Jisr al-Nahrawan, from where it was known as the Nahrawan proper. From Bajisra a canal, the Nahr al-Khalis, connected the main canal with the Tigris at Baradan and supplied the eastern suburbs of Baghdad with water, while another, the Nahr Bin, connected Jisr al-Nahrawan with Kalwadha south of Baghdad. Just south of Jisr al-Nahrawan there was another canal, the Diyala canal—the present course of the namesake river—which joined the Tigris some 5 km south of Baghdad. Jisr al-Nahrawan itself was a wealthy place, as there the Khurasan Road connecting Baghdad with Central Asia crossed the canal. Surviving descriptions record that it was extended on both sides of the canal, each with its own mosques, markets and hostels for travellers and pilgrims. It was abandoned by the 14th century, as the road to Khurasan was diverted north, through Ba'quba.