scorecardresearch Skip to main content

Beside the nation’s first Black woman Supreme Court justice is her husband, a ‘quintessential Boston Brahmin’

For centuries the ancestral lines of Ketanji Onyika Brown and Patrick Graves Jackson were impossibly far apart. Then, 30 years ago at Harvard, they crossed.

Ketanji Brown Jackson testified during her Senate Judiciary Committee confirmation hearing as her husband, Patrick Jackson, looked on.Andrew Harnik/Associated Press

He descends from British royalty and New England merchants, some who profited off slave ships or themselves owned people. She can trace her ancestors to plantations in the antebellum South, where they were enslaved. His forefathers led states and industries, accumulating vast wealth; hers were sharecroppers denied the profits of their own harvests. His family signed the US Constitution, a document that defined hers as less than fully human.

For centuries the ancestral lines of Ketanji Onyika Brown and Patrick Graves Jackson were impossibly far apart. Then, 30 years ago at Harvard, they crossed.

Their marriage in 1996 linked two poles of the nation’s past, in a bond their ancestors would have struggled to imagine. And their family histories, now intertwined, tell a uniquely American story, one that marries the nation’s proudest patrician traditions with its most shameful original sins.


The significance isn’t lost on the couple. Years before she became the first Black woman on the Supreme Court, Jackson pointed to the stark differences in her and her husband’s backgrounds during a 2017 speech at the University of Georgia. He was pre-med, she studied government; he was a Harvard legacy several times over, she was only the second generation in her family to go to college. He was a “quintessential ‘Boston Brahmin,’ ” she said, while her lineage led back to slavery.

“We were an unlikely pair in many respects,” Jackson told the audience. “But somehow we found each other.”

Their family demonstrates “the richness of ethnic identity in America,” said Henry Louis Gates Jr., who leads Harvard’s Hutchins Center for African & African American Research and hosts the PBS genealogy show “Finding Your Roots.”

It is also a reminder of which stories this country has celebrated, and which it has chosen to forget.

Their achievements have not as often been described, “but I’m sure that her African American ancestors were quite distinguished, too,” Gates said. “A person like that didn’t come from nowhere.”


In February, when President Biden nominated Jackson to the Supreme Court, a group of local genealogists set out to find where, and who, she came from. It started as an informal project for researchers at the Boston-based nonprofit American Ancestors, also known as the New England Historic Genealogical Society, who tracked both sides of the Jackson family. They wanted to see how far they could get.

From the first, their efforts presented a study in contrasts. Patrick Jackson’s ancestry was relatively easy; one of the genealogists, Chris Child, had already cowritten a book on much of it. His was the kind of family captured in leather-bound volumes and oil paintings. His ancestors appear on the Mayflower manifest and in prep school yearbooks. More than a decade ago, the genealogists had surveyed his family while researching a book, and Jackson’s mother had furnished details about names and birth dates. (Child and Patrick Jackson, it turns out, are half ninth cousins).

But as genealogists reconstructed Ketanji Brown Jackson’s family tree, confirming each new name presented a challenge. Her story would prove a long and rich one, leading from cities in Florida to rural stretches of Georgia and, ultimately, to ancestors who were enslaved and the places where they were held in bondage. It could not be found in library books. Genealogists went searching for wills that transferred ownership of people treated as property, scouting for birth certificates and marriage licenses from a time and place where such documents were often inconsistent or nonexistent. They relied on the survival of antiquated town records, the accuracy of centuries-old maps, and the good will of Southern historical societies.


It was a slog.

“I would say, ‘Oh, I’m so happy I found her grandparents.’ My colleague would say, ‘I found his 12-times grandparents,’ ” said Sarah Dery, a researcher tracing Ketanji Brown Jackson’s family line. “Individuals that are well known like Patrick’s side of the family — people have always been recording everything that family has been doing, generation after generation.”

Genealogy makes the argument that all families merit that same attention. But, reflecting our history, it struggles against the undertow of past inequities. It’s far more difficult to trace the ancestry of Black Americans than that of white Americans, in part because enslaved people were not named on the census until after the Civil War.

“Black history is American history, but it’s not always viewed that way,” said Shelley Murphy, a member of the Afro-American Historical and Genealogical Society. That attitude, which persists in some quarters, makes such research all the more difficult.

Before Emancipation, enslaved people were recorded in the decennial census count only as “hash marks,” Gates said — described just by age and gender. Even after 1870, the first count that aimed to include every American, other variations on oppression kept Black families from generating the written records genealogists’ work requires. Historians rely heavily on documents tied to estates, property ownership, and money — meaning the wealthy are more easily remembered and their stories more easily reconstructed. Black Americans had been both treated as property and forbidden to own it.


“The records, historically, were not created for this community,” said Lindsay Fulton, a genealogist with American Ancestors. That makes it harder to tell Black families’ stories, but no less important. “These people are worth remembering, and their names are worth learning,” she said.

Given those challenges, the genealogists started with what they knew, tracing backward from Jackson’s parents. Her father, Johnny Brown, was the attorney for the Miami-Dade County school board; her mother, Ella Ree Brown, née Ross, worked as the principal at New World School of the Arts in the same city. They were married in Miami in 1968. Both had family roots in Georgia; both had ancestors who had been enslaved.

The ancestral line that gives Ketanji Brown Jackson her maiden name proved the hardest to trace, in part because “Brown” is such a common surname. The genealogists added, then removed her paternal grandparents what felt like dozens of times because they at first couldn’t be sure who were the correct ancestors. They were struggling to distinguish between two couples named Brown: Both went by Thomas and Queen, both lived in Ben Hill County, Ga., in the mid-20th century, both had a son named Johnny. (They have since confirmed it was the couple whose Thomas had the middle name Logan.)


Thomas Logan Brown was a chipper in the naval stores industry, working with wood to produce material for ship repairs, and his wife, Queen Anderson, worked as a housekeeper and nurse’s aide. Ketanji Brown Jackson’s maternal grandparents were Euzera Green, who was born in 1917 in Georgia and died in Florida in 1989, and Horace Ross Sr., a landscaper who worked at a Miami airport.

As the genealogists reached back further into time, inconsistent spelling, ubiquitous in this kind of historical research, worked against them, too. They tracked Rutherfords also known as Relefords, both Merriweathers and Mayweathers, an Olmstead sometimes spelled Armstead.

They finally verified a man named Armstead Rutherford as Jackson’s great-great-great-grandfather, born around 1820, and married to a woman history has recorded only as Lucy, born around 1825. They lived on a 700-acre cotton plantation near modern-day Hayneville, Ga., about 40 miles south of Macon, where as of 1860, John H. Rutherford enslaved 65 people. Genealogists believe he may be the source of their name, but they cannot yet be sure that he was the man who enslaved them.

In their struggle to locate records, the researchers discovered one particularly striking document: an 1867 sharecropping agreement between Armstead Rutherford, now emancipated, and John H. Rutherford, the man who owned the plantation. It amounted, many would argue, to merely another form of slavery: Armstead was paid no money, only permitted to keep a portion of the harvest. It was a common, and oppressive, arrangement after Emancipation.

Census records indicate Armstead Rutherford was never taught to read or write; the page’s consistent, cursive handwriting is someone else’s. But beside his name, he signed with an X, an assertion in ink of his place in this country’s history. So far, researchers know little about her ancestors before the time of Emancipation.

1867 sharecropping agreement of Armstead RutherfordFreedmen's Bureau

Hundreds of miles north, by the time that sharecropping agreement was signed in 1867, Patrick Jackson’s ancestors had secured a far more powerful grip on national commerce and government.

Many were wealthy merchants in the Boston area, in trades including law, insurance, and textiles. Others were behind the country’s founding institutions of government, higher education, and industry. There had been John Howland and Elizabeth Tilley, who survived the Mayflower’s perilous journey in 1620; Thomas Dudley, the third governor of Massachusetts; Nathaniel Gorham, a Massachusetts judge who signed the Constitution and helmed the Continental Congress; and John Lowell, a lawyer, politician, and judge who contributed to the abolition of slavery in Massachusetts.

Like many people who boast centuries-old New England ancestry, Patrick Jackson’s family had a complicated relationship with slavery. There had been Lowell, who wrote into the Massachusetts Constitution that “all men are born free and equal.” But some of Jackson’s early ancestors enslaved people in Massachusetts, historians say, while others profited from the slave trade.

Around the start of the 19th century, there was Peter Chardon Brooks, a businessman and politician at one time reputed to be the wealthiest man in New England. Historians say Brooks — Patrick Jackson’s great-great-great-great-grandfather, on the paternal side — insured cargo ships, including vessels owned by slave traders.

Patrick Jackson descends from the Jacksons, Cabots, Lowells, Gardners, and Saltonstalls, five dominant families of Boston Brahmins, an elite class of wealthy, influential New Englanders that takes its name from the top rung of India’s caste system. His distant relatives include figures as prominent as President John Quincy Adams (his fourth cousin, five times removed), Supreme Court Justice Oliver Wendell Holmes Jr. (his second cousin, four times removed), art collector and philanthropist Isabella Stewart Gardner (his first cousin in law, four times removed), and Governor Charlie Baker (his 10th cousin, once removed).

Patrick Jackson was born into that grand tradition in 1969, near Boston. He followed many in his family to the Groton School, where he excelled in math and impressed in the wood shop, painstakingly shaping and sanding Bombe furniture, a popular style in Revolutionary War-era Boston that his ancestors might have appreciated, too.

Then, in the late 1980s, Ketanji Onyika Brown and Patrick Jackson started at Harvard, he a year earlier than she. In landing there, he had done what dozens of family members had modeled. She had achieved what no one in her family had before.

At first, she introduced him to her roommates as just a friend, said Lisa Fairfax, a University of Pennsylvania Law School professor and one of Ketanji Brown Jackson’s college roommates and close friends. It soon became clear he was far more than that. The close-knit group of Black sophomore women initially looked with skepticism at the white junior now competing for their friend’s attention. But he soon impressed them with his humor and his absolute devotion to Ketanji, Fairfax said. He made an effort to fit in with their group.

“I did not realize that he was from a family with long roots in the area because he came off as so down to earth, so friendly, so unassuming,” Fairfax said. “If we had an image of a quintessential prep school boy, he was not it. . . . That was not his personality at all.”

They married in 1996. From college, she went on to Harvard Law School, roles on the US Sentencing Commission and as a judge on a federal appellate court, and, as of June, on the Supreme Court; he, to Columbia Medical School, residency at Massachusetts General Hospital, and the chief of surgery job at Georgetown University Hospital.

The branches of his family tree already bore several members of the Supreme Court by the time Biden nominated Ketanji Brown Jackson in February. By contrast, she has said her grandparents, who had no education beyond grade school, could never have imagined her there.

In June, Jackson stood beaming in the Supreme Court building as Chief Justice John Roberts administered her constitutional oath. As she made history, Patrick Jackson stood just behind her, holding two Bibles beneath her hand.

“In my family, it took just one generation to go from segregation to the Supreme Court of the United States,” she said, voice wavering slightly, after she was confirmed by the Senate.

And Jackson takes on that role “while ‘bringing the gifts my ancestors gave,’ ” she said, quoting Maya Angelou. “ ‘I am the dream and the hope of the slave.’ ”

Emma Platoff can be reached at Follow her @emmaplatoff.