Unionism in the Confederacy and Vicksburg’s Shirley Family during the American Civil War

When people think about the American Civil War, the picture that often comes to mind is that of a divisive sectional conflict. The north rallied around the causes of preserving the republic of the Union and (eventually) liberating enslaved people. On the other side was the south who rallied for the creation of an independent Confederacy and the preservation of the economics of slavery. What this narrative typically overlooks is that there were significant parts of the Union and the Confederacy who dissented against these polarities. One group that is typically omitted from the conversation are Unionists within the Confederacy who remained loyal to the United States government. What follows are brief sketches.[1]

An exemplary “pocket” of Unionists persisted throughout the Civil War in Natchez, Mississippi. Many of these were born in northern states and relocated to the south, where they became gentlemen planters (some were also doctors, lawyers and judges). These families maintained social and economic ties to the northeastern United States. They often employed northern governesses and tutors for their children, and their children were often sent to northern schools for their education. During the hot, humid and disease-prone (especially yellow fever) Mississippi summers, these “Nabobs” would travel north to visit family and friends. They read northern newspapers, bought northern goods, and sold their plantation products through northern brokers. The Natchez Nabobs were wealthy and had hundreds or thousands of slaves. They did not become supportive of the Confederacy when the war broke out. They were socially, culturally and economically tied to the Union, but did not feel guilty about possessing enslaved people, nor did they want any infringements on slavery. They sold their cotton to the north through the Union blockade during the war.[2]

Another excellent case of Unionists within Mississippi was the Shirley family from Vicksburg. The Shirley’s shared many of the same background characteristics of the Natchez Nabobs. James Shirley and his second wife Adeline were both born in Massachusetts. James went to law school at elite Dartmouth College. He met Adeline in Georgia, and they eventually moved to Vicksburg. Adeline came from a wealthy “Boston Brahmin” family (the Quincy’s), and she was also highly educated. By the time of the Civil War, Adeline and James had three children: Frederick, Alice and Quincy. James sold real estate, practiced law and served as a judge in Vicksburg. The family ran a moderate-sized plantation which cultivated a fruit orchard (peaches, pears, & watermelons) and vegetables (peas, lettuce, radishes, and peanuts) on about 100 acres in the countryside east of the city. The Shirley’s owned 24 or 25 slaves in the period before the siege of Vicksburg. These enslaved people ranged in age from 2 to 88, with an average age of 29 years old. 28% of these were female, while most of the enslaved were male (72%). In the 1860 Census, 17% were identified as mulatto (white and black ancestry), while 83% were identified as having black ancestry.[3]

Alice Shirley grew up to be a talented and intelligent woman. She was home-tutored, became well-read (even in French), and became an accomplished piano player. At the time of the war, she was an 18 year old college student at the Central Female Institute (present-day Mississippi College) in Clinton (40 miles to the east, just outside of Jackson). She was born and raised on a slaveholding Mississippi plantation. Alice wrote the following in her 1900 newspaper account of the siege of Vicksburg: “As children, we of course had no thought of slavery as wrong; we saw only the bright side of the system. The negroes seemed a part of the family, and I remember I was as polite to Aunt Sarah, the cook, and to Aunt Cynthia, the housemaid, Uncle Will’s wife, as I was to my mother, and they reprimanded me about as often as she did. At that time my father owned twenty-five slaves. The older slaves we children always addressed as aunt and uncle, which would seem very strange these days.” Quite interestingly, Alice married chaplain John Eaton, a Union officer tasked with creating the prototype Freedman’s Bureau in wartime occupied Vicksburg. He was also Colonel of the 63rd U.S. Colored Infantry. Eaton staunchly advocated for assisting the enslaved population in the wake of the war.[4]

When Mississippi seceded and the Confederacy was formed, the Shirley family remained ardent supporters of the Union. The family was ostracized and considered traitors by many in Vicksburg and had to keep to themselves much of the time. They would meet in secret with a small community of Unionists and exchange news about the war gleaned from newspapers. Older brother Frederick was a member of a local military company, and openly stated that he “would rather serve Lincoln twenty years than Jeff Davis two hours.” This agitated Confederate supporters who threatened to hang him. The family hurriedly shipped him off to Indiana, where he remained until Vicksburg fell. In May of 1863, following a series of battles, the Union Army rapidly closed on the city. Unfortunately for the Shirley’s, their residence was along the main road to Jackson, and thus was one of the direct routes the Union would use to attack and lay siege. James took the railroad to Clinton to pick up Alice from college and get her back to the family. They had decided to wait out the impending attack at the family home. But the Civil War rolled right over father and daughter. They became trapped in the village as the Confederate Army, and then the pursuing Union Army, both stormed through town. The Clinton railroad depot was burned, and the tracks were torn up. They soon greeted Union officers, served them milk and Alice played both Confederate and Union patriotic airs on the piano for them.[5]

Father and daughter were now separated from their mother and brother in Vicksburg. Additionally, the Shirley’s faced a serious challenge. Their residence would surely be destroyed, so they had to convince Union commanders that they were true-blooded Unionists. It was agreed that Alice would stay put and resume her college studies, while 69 year old James began walking the 40-mile road towards home. With the assistance of the Union army, he eventually made it back into Vicksburg. Alice finished the semester but remained stuck. On July 4th, a thundering cannonade proceeded the retreat of General Johnston’s Confederate army back through Clinton. Alice received a telegram from her father to return home, so she boarded an Ohio regiment’s ambulance and joined a Union wagon train headed west to Vicksburg. After stops at night, switching wagons, and a nine mile train ride, she arrived in a few days and reunited with her family. When the war arrived in Vicksburg, Adeline had successfully pleaded with the Confederates to refrain from burning down their house. The Union army soon arrived, also intending to burn it down, but the soldier with the torch was shot dead by Confederates. The house then came under severe and heavy gunfire, so the family’s enslaved carriage driver hung a white sheet/ flag from the upstairs porch. Union soldiers seized possession of the house and began using it as field hospital for the wounded and dying. It was later repurposed as a smallpox isolation hospital. The soldiers found Alice’s diary in the house, read it, and were pleasantly surprised to find laudatory praise for the Union across its pages. This diary proved to the Union military that the Shirley family was indeed loyal to the Union.

After withstanding three days of intense combat in the house, mother, son, and a few of the remaining enslaved people relocated to an earthen dugout cave in the surrounding grounds.[6] James arrived a few days later and joined the family there. They relocated several times to places further and further outside the city. When Alice finally arrived back in Vicksburg, her family was residing in the abandoned house of a Methodist minister within the city. Sadly, James died in August due to his advanced age compounded by the stresses of the war. The remaining family members abandoned their plantation, and left Vicksburg for good. The war took a severe toll on the Shirley Unionists. When the house became part of Vicksburg National Military Park around 1900, Alice saw to it that the house was restored and arranged for her parents to be interred in the backyard veranda.[7] What happened to the Shirley’s enslaved people has yet to be determined. Most likely, they were emancipated by the Union military, and either went to work on Union-run plantations in the surrounding delta region (for example, Jefferson Davis’s plantation to the south of Vicksburg), and some of the age-eligible males may have joined the ranks of the Union’s African American forces.[8] It is also possible that some of the enslaved remained settled in Vicksburg, and their descendants might be residents of the city and elsewhere today.   

The Shirley’s story is certainly a wrenching and dramatic event of the war. Significantly, it highlights the background of and subsequent issues that some Unionists had to face as a result. Most people would be surprised to learn that loyalty to the Union was more common within the Confederacy than they might imagine. Small “pockets” of Unionists were spread out everywhere. The adjacent state of Alabama had its share of Unionists.[9] As highlighted here, some Unionists were wealthy slaveholding planters with social and economic ties to the North, probably interested in keeping their plantations economically viable. Other, less well-to-do southerners (some of whom initially served as Confederate soldiers) became increasingly disaffected by the war. A famous example of this is the later-war Unionists who held out against the Confederacy in the backwoods of Jones County, Mississippi.[10] There are many others like them, for example a group of 300-400 Unionists, from coastal Pearlington, Mississippi, who were army deserters hiding out in the swamps and aiding enslaved people escaping into Union lines.

[1] For a look at dissent within the Union in the form of riots against conscription and the draft, visit my website.

[2] William Scarborough, “Not Quite Southern: The Precarious Allegiance of the Natchez Nabobs in the Sectional Crisis,” Prologue: Quarterly Journal of the National Archives and Records Administration 36(4), Dec. 2004, 20-29.

[3] 1860 Census of Vicksburg, Warren County, Mississippi, page 116. Slave schedules for James Shirley, pages 8 and 34. Alice Shirley wrote a newspaper article published in 1900 about her wartime experiences. It is fully reprinted in the following two sources: Terrence Winschel, Alice Shirley and Wexford Lodge (Washington D.C.: Eastern National, 1993, revised 2003). John Eaton, Grant, Lincoln and the Freemen: Reminiscences of the Civil War (New York: Longmans, Green and Co., 1907), 71-86.

[4] Winschel, Alice Shirley and Wexford Lodge, 8-13. Eaton, Grant, Lincoln and Freedman, 62-86.

[5] Winschel, Alice Shirley and Wexford Lodge, 13-15. Eaton, Grant, Lincoln and Freedman, 71-86.

[6] For a really exciting narrative of civilian survival in besieged Vicksburg, read: Anonymous (A Lady), My Cave Life in Vicksburg: With Letters of Trial and Travel (New York: D. Appleton and Co., 1864).

[7] Winschel, Alice Shirley and Wexford Lodge, 13-27. Eaton, Grant, Lincoln, and the Freedmen, 71-86.

[8]Linda Barnickel, Milliken’s Bend: A Civil War Battle in History and Memory (Baton Rouge, LA: Louisiana State University Press, 2013).

[9] Margaret Storey, Loyalty and Loss: Alabama’s Unionists in the Civil War and Reconstruction (Baton Rouge, LA: University of Louisiana Press, 2004).

[10] Victoria Bynum, The Free State of Jones: Mississippi’s Longest Civil War (Chapel Hill, NC: The University of North

Carolina Press, 2016).