[Music]
and with us we have Jay Rosen bomb who's
a Melbourne based AI artist and
researcher working in 3D modeling
artificial intelligence and extended
real Technologies they have a PhD from
rmit University uh studying AI
perception of genders and the nature of
AI generated art and human gen uh human
hands behind the process that engender
the bias uh and today they are going to
give us a talk on AI perceptions of
gender I'll hand it over to you thank
you so
[Applause]
much yep just wait for the slides to pop
but yeah so hi hi everyone um I'm so I'm
so excited to be here today I love Pon
uh my name is Jay Rosen and I'm an AI
artist and researcher and I te and I'm a
lecturer in critical AI at rmit school
of
design the term AI artist has become a
really loaded term lately um when I
first started working with machine
learning art nearly a decade ago it was
easier to to say AI artist The Machine
learning artist um but nobody knew what
machine learning meant an AI artist was
sort of a Sci-Fi fancy fun marketing
term and now of course it's vokes ideas
of shortcuts and plagiarism and creative
theft and I don't really like being
associated with that anymore but I do
still use the term after a lot of
internal wrangling I do still use the
term AI
artist sometimes with a little bit of
irony and because my practice deeply
engages with AI systems not just as
tools but as objects of critique and
study today I'm going to talk about my
research uh and the art I created during
my PhD which explored AI perceptions of
gender my focus was on how AI interprets
and represents gender challenging the
binary Frameworks that dominate machine
learning and expanding them Beyond
biological
essentialism gender is fast it's deeply
personal it's so much more than just
Anatomy or a checkbox on a form through
my work I've asked questions like why do
AI systems perceive gender and how do
they qualify gender in the images that
they generate if we create a biased data
set and slowly debias it by introducing
new data how will the generated images
be affected and what are the
implications for visual representation
present in generative images of people a
lot of these questions are still ongoing
today like the PHD is finished but I'm
still finding myself coming back to
these questions over and over
again I trained a bias neural network
and I sought to see if I could add data
to change its biases and what it looks
like if I did I examined existing
systems and looked at how they explore
gender and what they see when presented
with gender beyond the binary and I
explored a new way of looking at gender
classific
completely by assigning people a custom
mixed
color if you were at Pon last year you
might have caught my talk on that one um
my first project set in stone began with
exploring whether we can change these
biases in a biased system uh people
often talk about the way to fix a biased
system is to add more data so I went
okay let's start from that point let's
add more data let's change the narrative
so I worked with generative adversarial
networks Gans um using techniques like
transfer learning where you train
retrain a neural network after it's
already been trained and what I liked to
call disruption training where I would
just add in new data throughout the
training process while while it was
still training um and that approach
really allowed me to see whether a gan
could unlearn its original biases and
adapt to new representation
sorry masks make you
[Laughter]
dry the light the title comes from a
metaphor that I kept coming back to the
way AI systems encode bias as if carving
it in stone so I started with DC Gan and
pran models which generate images by
learning patterns from a training data
set my goal was to see if I could
reshape that stone by injecting data
that defied stereotypes I created 3D r
faces for my data set representing
masculine feminine and non-binary traits
with classical marble Aesthetics leaning
into that set in stone
metaphor I tried to make them as
racially diverse as possible I created
an animation with 3D rotation and used
all the Stills as the um training data
set um in this 3D rendered
environment so for this project I first
trained a series of DC gains or deep
convol IAL generative adversarial
networks with my data set of masculine
faces over time when once it was well
and truly biased I introduced feminine
faces and then faces representing more
non-binary Aesthetics images emphasizing
external markers of gender expression
like brightly colored hair and makeup
these elements were chosen to highlight
how gender minorities often communicate
their uh their gender and highlight um
how how we see our identity visually
because non-binary people don't always
you know look non-binary so how do you
tell that to a computer how do you so
it's a little on the nose the colored
hair and makeup it's a bit of a
stereotype but sometimes you need to be
a little literal um and it worked really
well for that breaking of the stone
metaphor as new data entered the system
the generative outputs shifted initially
the gans's output reflected the rigid
binary ideas of gender but as the
training progressed these boundaries
softened vibrant colors emerge and take
and The Works take on a more fluid and
joyful take on
gender however even as the gain
integrated the new data I noticed a
tendency to revert to its original
masculine bias over time increasing the
volume of non-binary INF feminine coded
data helped to counteract that reversion
but the model never truly forgot its
initial
biases switching to a PR a pran or
Progressive growing Gan yielded smoother
transitions the technique used to debias
was the key observation here as I had to
create directories filled with the same
masculine data set uh initially it's a
labeled uh Gan so I needed to create the
label data sets filled all with the same
data and then changed the them up uh
over time and then I so I replaced the
data gradually with more diverse images
this was a fitting metaphor for how some
genders need to make room for the
representation of others by gradually
replacing older data sets with newer
images I observed more seamless
debiasing the pran created fewer
artifacts and seemed better uh
integrating diverse representations
without deting to its original biases
the resulting visuals were presented as
mosaics wallpapers and looping videos
from afar they appear as cohesive
portraits but up close viewers could see
that the hundreds of and thousands
sometimes of smaller images that made up
each face these layers reflected the
idea that gender is shaped by
accumulated experiences and
self-expression over time through set in
stone I learned that while debiasing
again is possible it's not always
straightforward the architecture of the
neural network has to play a part in how
successfully it can adapt to the new
data and much like people AI systems can
carry their history with them even as
they evolve the large scale mosaics show
not only the AI Aesthetics and the
debiasing of the neural network but
created a dialogue about how we are the
sum of our experiences not just what you
see on the
surface in the loop video this one the
face continually zooms into to one
square of many in the Mosaic which then
shifts and changes developing into a new
face which is zoomed into again each
face exists because of all the faces
that surround it helping it
become I show here not only the shift
caused by debiasing and AI techniques
but also the experience of being
transgender the exploration of gender as
observance as expression as experience
this concept of gender as experience is
one I returned to in my gender tapestry
work discussing gender as a highly
personal lived experience that is unique
to every
individual the second project in my PhD
was an analysis of existing systems in a
complex Loop of AI architectures if
you've ever played a game of telephone
you know how a message changes when it's
passed along the chain or telestrations
the board
game I wanted to see what would happen
if I turned that idea into an AI
experiment what happens when I link a
bunch of different AI systems together
and observe their output so
Frankenstein's telephone explored how AI
systems gender images revealing the
biases embedded in their data sets this
project works like a game of telephone
using a chain of AI systems that pass an
image and its interpretation from one to
the
next the process started with attn Gan
the very first text to image generator
they created an image based on a user's
input prompt that image was segmented by
deeplab breaking it into labeled regions
like Sky person tree and the segmented
image was then reimagined by Spade Coco
which reconstructed it as a new image
and finally IM to text generated a
caption for the fin image so that's
probably very unreadable but that's okay
cuz we've got a readable version coming
up
oh this chain highlighted how each
system introduced new layers of
interpretation Distortion and bias so
this this what the prompt a woman with a
dog on a leash it's now segmented the
dog as partly um partly a
person and um it's been a bit
interesting which has then turned into
this version this further version which
um has a a these are very chaotic images
these are are very early um generative
AI systems they're not what we're used
to now but then image to text um
captioned it as a man writing a
skateboard down a
street
so this chain highlighted how each
system introduces new layers of
interpretation Distortion and bias for
example deep lab segmentation labeled
all human simply as person removing
gender entirely yet by the final stage
gendered language had appeared in the
captions again binary words like man or
woman would emerge based on contextual
cluthes like clothing or background
elements I Trac this bias back to the
human generated captions in the training
data sets these captions frequently
gendered people even when their faces
weren't visible or when the person in
question appeared abstracted or
incomplete this pointed to the
assumptions of the data set's human
annotators many of whom worked under
vague instructions through Mass labor
platforms like Amazon Mechanical turque
there's little to no training in
situations like these just a short
series of instructions it strikes me
that a specialist could be hired to
specify clearer instructions to avoid
gendered
outcomes like I observed in the um in
the data set and especially the
occasional sexist or racist outcomes I
observed in the data set the um the data
set often would uh have like a value
judgments a very beautiful woman doing
this and then if it was a man it was a
man you know um whereas gender actually
if one was just a hand and they went oh
that's a woman's hand it's like it's a
black and white hand it's barely in
Focus who knows whose hand that is come
on we don't need to gender these things
but that gendering that bias that that
came through those assumptions all came
through in the
work the visual outputs of
Frankenstein's telephone are surreal and
unsettling pale fleshy feature figures
with exaggerated alien features these
dreamlike blobs reflect the data set's
lack of grounding in real human anatomy
and raise questions about how machines
create images and perceive images around
gender for me the work became a metaphor
for how transgender and non-gender and
gender non-conforming people people are
often
perceived misunderstood distorted by
societal
expectations an interactive version of
this project let users upload their own
images to see how the AI chain
reinterpreted them and that went about
as well as you could
expect many participants were surprised
and um some were quite vifly upset uh by
the results particularly when the system
misgendered them uh the complaints I got
were all from cisgender men white men I
should say what a
surprise uh this sparked conversations
about how machines and people perceive
identity and those those very real harms
caused by
assumptions this project unfortunately
is no longer online because the systems
that were being hosted um have been
taken offline in favor of generative AI
systems so I need to go back into the
code rehost these um on a different
platform set them up and um re
reinitialize the project and I also want
to take it into generative AI systems
and redevelop it to see have have these
systems evolved have they updated the
images are better but are these biases
in the chain do they still come out do
the machines still misgender people do
the systems still observe a new gender
so that's what I'd love to contrast with
the earlier versions see what
developments if any have
happened I've also put spun this
Research into a series of image
Generations across multiple stages of
generative AI systems to see how they
all interpret the same prompt this
project is ongoing and may never really
have an end date as new versions keep
coming out but the viruses are very very
clear I I should explain that my
methodology here was that it was the
middle of winter and I was very cold um
and I think it was during lockdown as
well so I wanted to be at the
beach this is this is a legitimate
methodology so just in looking a
generative images of non-binary people
I've noticed the same Trends every
person is Young of course they're all
gorgeous almost all of them have the
short fairy hair pale skin and most are
fem despite the many non-binary people I
know with beards I've actually never
seen any on a generative on a generated
non-binary person without specifically
asking actually apart from the age they
all tend to look very
similar which is great for all of the
people who fit in that box I guess um
but what about non-binary people of
color what about high fem or high mask
non-binary people it's so very hard to
get a non-white non-binary person that
even if I explicitly State an ethnicity
it still struggles the eurocentric
biases of these systems have always been
really wrong but for non-binary
representation it's an almost impassible
barrier I'm taking similar directions in
some of my new work to explore how image
toex systems view a pixelated image of a
person and reinterpret it as a new image
this shows the biases of the system very
clearly and by making it personal shows
how these systems average out features
and remove individuality through their
biased data actually really like this
image
though I for I didn't save the caption
though um uh I wasn't wild the captions
are quite horrible they're through clip
but I really actually quite liked that
result I felt very
validated
um sometimes these systems don't
suck no they do they all
suck the final project in my PhD gender
tapestry brought together the insights
from my earlier Works into a unified
piece this interactive project aimed to
reframe G gender classification as a
spectrum rather than as a binary using a
custom classifier that assigned each
participant a unique
color the data set for this work was
mostly generated it felt less
problematic to use generated images of
people and originally I had an even
spread across genders however I found
that the color results in that case were
very very limited I actually needed to
bias the results or bias the data set
which really surprised me that was a
crazy thing um I've been fighting bias
my whole machine learning life and now
you're telling me I need to put in bias
but the data didn't lie on that front so
I
actually biased the data set in line
with current representational data um I
still kept it as um diverse as I
possibly could but you know um and I I
sorted them into six different pronoun
categories users upload an image of
their face which is analyzed by a
multi-label classifier trained on these
pronoun based data sets instead of
assigning a gender the classifier
outputs percentages that are then uh
added together to create a an R GB
value these values were used to generate
a custom color which represented the
user's unique experience of gender what
I particularly enjoyed here was that um
by taking six and turning it into three
by adding those those disparate Parts
together I created some Randomness and
then if the number went over
255 the modulo operator kicked in and um
brought it brought the number back
around which again increased that
Randomness so while these
classifications and these colors are
based on fact they're actually all still
um they're based on on a scientific
methodology and on on proper
classification but there're still an
element of you know magic and weirdness
and math that helps create those
beautiful random
effects you can actually uh it's
probably um still up at gender
tapestry.com uh the Gan won't be running
but the rest is the users color and
portrait were then added to a growing
data set which was used to generate a
mosaic of can created faces over time
the Mosaic became increasingly complex
as more colors and images were added
illustrating that diversity of gender
experience the concept of a of assigning
a color to gender stem from the idea
that like color perception gender is
subjective and deeply personal my
research revealed that we all actually
see color differently the blue sky that
I see is very different from the Blue
Sky you
see so we it's actually all a very very
subjective experience and gender is a
subjective experience and it's all
deeply personal so while two people
might have similar experiences no two
are exactly alike this approach
challenge the feasibility of gender
classification by AI emphasizing that
gender isn't something that can or
should be reduced to a
binary across these projects I explored
the ways AI reflects human biases and
the potential for Creative interventions
to challenge those biases some of my
findings include shocker a bias is
deeply embedded in data sets who knew
even when AI systems are trained on
neutral data the assumptions of their
creators can shape their outputs in the
paper biased data sets are not enough
results like uh a classifiers would
classify an image uh as a woman if a
person appeared in a kitchen because
photographs of women were more common in
kitchens than photographs of men in
kitchens just a small example there of
some yeah systemic bias there AI can
evolve but not perfectly while new data
can shift a model's perspective it often
retains traces of its original biases
much like humans do
representation matters expanding data
sets to include more diverse identities
is crucial for creating more Equitable
systems but it's only part of the
solution e through my practice I hope to
highlight not only the limitations of AI
but also its potential as a medium for
reflection critique and growth AI
systems may never fully compute what
gender
means but by questioning their outputs
and our own biases we can move toward a
more inclusive future the best time to
start is yesterday we know that but the
next best time is now today so initially
humans are told how to perceive gender
they learn what gender is from their
parents teachers and peers they grow and
learn they learn to perceive gender how
they want to perceive it they research
or they don't they learn and listen and
their understanding of the concept of
gender
grows similarly machines are taught how
to perceive gender by humans they are
shown images and told that these images
correspond to genders an AI will not
update its definitions on gender unless
instructed to it will not change its
perspective or enhance its understanding
unless it is specifically retrained it
can only do as its instructions permit
it then falls to the people training the
AI to and constructing the data sets to
ensure that it's trained without bias a
computer itself has no inherent biases
it's only taught bias as it learns and
incorporates bias data and
algorithms the people working with the
AI therefore need to understand the
potential for bias and the inherent
issues in facial classification tasks
including gender classification
especially gender
classification people are naturally
consciously or unconsciously biased and
their inherent biases creep into their
work we can help subvert bias by being
self-aware of our own biases and our own
issues gender is an Ever evolving social
construct informed by experience and
upbringing similarly AI is an Ever
evolving machine construct with human
supplied inputs that could be kind of
equated to a form of experience and
upbringing raising an equitable AI
requires the voices of many
and the intention to listen when issues
are exposed as we are currently in the
nent stages of understanding AI now is
the best time to explore how they
interrelate untangling the convergence
of machine learning and social change is
a key requirement right now especially
as AI development is so rapid the time
is Rip to merge these developments and
to realize we cannot train AI in the
outdated social Moray of the past but
rather think ahead to a new stage of
development and understanding of
ourselves and our AI we need to be aware
and critical of our biases examining the
data sets we create and the methods we
use to train AI this vigilance isn't
just about fairness it's about creating
systems that reflect our Humanity in its
fullness not just a narrow biased sliver
of it and that brings me to why this
work matters it's not just about Ai and
data
it's about us it's about how we Define
ourselves and each other it's about how
we interact with technology and how
those interactions shape our future if
we allow AI to have learned from our
worst habits our biases prejudices and
assumptions it will replicate and
reinforce them in ways we might not even
notice until it's too
late but if we teach AI to recognize the
diversity and fluidity of human identity
if we build systems that acknowledge and
celebrate the complexity of gender we're
not just making better AI we're creating
a better reflection of a better world
one that sees and values everyone thank
you