Before Oklahoma became a state in 1907, felons convicted in Oklahoma Territory and Indian Territory were sent to the Kansas State Penitentiary in Lansing, Kansas. At statehood, Kate Barnard became Oklahoma Commissioner of Charities and Corrections. During the summer of 1908, Barnard arrived unannounced at the Kansas prison to investigate widespread complaints she had received about mistreatment of Oklahoma inmates. She took a regular tour with other visitors first, then identified herself to prison officials and asked that she be allowed to conduct an inspection of the facility. Barnard discovered systematic, widespread torture of inmates.

Upon her return to Oklahoma, Barnard recommended that all Oklahoma inmates be removed from the Lansing facility and returned to the state. Governor of Oklahoma Charles N. Haskell supported Barnard's proposal, and within two months of Barnard's visit to Kansas, on October 14, 1908, two groups of 50 offenders each were sent by train to McAlester. The inmates were temporarily housed in the former federal jail in the town. Under direction from Warden Robert W. Dick, they built a stockade to house themselves on a 120 acres (0.49 km2) plot northwest of McAlester, which was donated to the state by a group of McAlester citizens.

The remaining Oklahoma inmates in Lansing were moved to the United States Penitentiary, Leavenworth until the state could build adequate facilities to house them all. The next spring, in 1909, the Oklahoma Legislature appropriated $850,000 to build the permanent facility.

Construction began in May 1909 on a prison designed after the Leavenworth facility. The state purchased about 1,556 acres (6.30 km2) surrounding the original plot of land. Using prison labor, the West Cellhouse and Administration Building were completed first; the Rotunda and East Cellhouse came later. The steep hills and grades required more than 6,250 cubic yards (4,780 m3) of concrete and more than 2,000,000 cubic yards (1,500,000 m3) of rocks and soil were moved for the prison's walls alone. The F Cellhouse was added in 1935, and later the New Cellhouse was constructed. A shoe manufacturing plant and a tailor shop were part of the prison's inmate industry program, designed to provide work for offenders; at Lansing, prisoners were forced to work in the local mines, a practice Barnard banned.

Female prisoners were sent to Kansas in territorial days also. The first females brought back from Kansas stayed in a ward near the East Gate built in 1911. housed on the fourth floor of the West Cellhouse. The female population had grown to 26 by the time a separate building about 1 mile (1.6 km) west of the main institution was completed in 1926.

The first prison escape (from behind the walls) occurred on January 19, 1914. Three inmates stole a gun in the escape attempt, killed three prison employees and a federal judge. The convicts were later killed behind a rock ledge located on a ridge overlooking a wagon road.

By the early 1970s, advocacy groups warned the state government that the situation was becoming dire.

On July 27, 1973, trouble began in the prison's mess hall, reportedly by five inmates who, according to a prison spokesman, "were doped up on something." It quickly spread through the rest of the facility. At the end of the riot, three days later, three inmates were dead, 12 buildings were burned, and 21 inmates and guards had been injured. Damage was estimated at $30 million.

A federal court in 1978 found conditions at OSP unconstitutional. The lawsuit, filed by one inmate before the riot, was changed to a class action suit after the riot. U.S. District Judge Luther Bohannon put the Department of Correction under federal control. The last issue of the lawsuit, medical care for offenders, was settled 27 years later, in 2001.

Consequent to the court's orders, four new housing units were built at OSP, and in 1984 the aging East and West Cellhouses were closed. In 1983, all female inmates were moved to Mabel Bassett Correctional Center in Oklahoma City.