Multiple one-shot prompts with no history. I don't have the exact prompt handy but it was something like "Create a short biography of a family, summarizing each person's age and personality".
I just ran that prompt 3 times (no history, new sessions, that prompt for first query) and got:
1. Hard-working father, stay at home mother, artistic daughter, adventurous son, empathic ballet-loving daughter
2. Busy architect father, children's book author mother, environment- and animal-loving daughter, technology-loving son, dance-loving daughter
3. Hard-working engineer father, English-teaching mother, piano- and book-loving daughter, basketball- and technology-loving son, comedic dog (!)
I'm summarizing because the responses were ~500 words each. But you can see the patterns: fathers work hard (and come first!), mothers largely nurture, daughters love art and dance, sons love technology.
It's not the end of the world, and as AI goes this is relatively harmless. But it is a pretty deep bias and a reminder that AI reflects implicit bias in training materials and feedback. You could make as many families as you want with that prompt and it will not approximate any real society.
I agree that this is a good illustration of model bias (adding that to my growing list of demos).
If you want to work around the inherent bias of the model, there are certainly prompt engineering tricks that can help.
"Give me twenty short biographies of families - each one should summarize the family members, their age and their personalities. Be sure to represent different types of family."
That started spitting out some interesting variations for me against GPT-4.
While I haven't dug into it too far, consider the bias inherent in the word "family" compared to "household".
In my "lets try this out" prompt:
> Describe the range of demographics for households in the United States.
> ...
> Based on this information, generate a table with 10 households and the corresponding demographic information that is representative of United States.
(I'm certainly not going to claim that there's no bias / stereotypes in this just that it produced a different distribution of data than originally described)
Agreed -- I ultimately moved to a two-step approach of just generating the couples first with something like "Create a list of 10 plausible American couples and briefly summarize their relationships", and then feeding each of those back in for more details on the whole family.
The funny thing is the gentle nudge got me over-representation of gay couples, and my methodology prevented any single-parent families from being generated. But for that project's purpose it was good enough.
I just tried the prompt "Give me a description of 10 different families that would be a representative sample of the US population." and it gave results that were actually pretty close to normative.
It still was biased for male head of households to be doctors, architects, truck drivers, etc. And pretty much all of the families were middle class (bar one in rural America, and one that was a single father working two jobs in an urban area). It did have a male gay couple. No explicitly inter-generational households.
Yeah, the "default" / unguided description of a family is a modern take on the American nuclear family of the 50s. I think this is generally pretty reflective of who is writing the majority of the content that this model is trained on.
But it's nice that it's able to give you some more dimension when you ask it vaguely for more realistic dimension.
I just ran that prompt 3 times (no history, new sessions, that prompt for first query) and got:
1. Hard-working father, stay at home mother, artistic daughter, adventurous son, empathic ballet-loving daughter
2. Busy architect father, children's book author mother, environment- and animal-loving daughter, technology-loving son, dance-loving daughter
3. Hard-working engineer father, English-teaching mother, piano- and book-loving daughter, basketball- and technology-loving son, comedic dog (!)
I'm summarizing because the responses were ~500 words each. But you can see the patterns: fathers work hard (and come first!), mothers largely nurture, daughters love art and dance, sons love technology.
It's not the end of the world, and as AI goes this is relatively harmless. But it is a pretty deep bias and a reminder that AI reflects implicit bias in training materials and feedback. You could make as many families as you want with that prompt and it will not approximate any real society.