What data and models were used to synthesize these predictions?
On a completely different note, I am not sure what you’re getting at with the racial stuff. Like what are you suggesting? That perhaps some of the emperors weren’t totally Caucasian but rendering them as such from the best available evidence is somehow racist? In my humble opinion, trying to find racial undertones in every tangential thing is a distraction from the real conversations that we should be having. And there’s a real possibility of alienating people that are otherwise sympathetic to the cause.
In this case it isn't tangential but central, since the point is extrapolating physical appearance from incomplete data. It matters if the results predict inaccurate racial features because plenty of people unfortunately care about kind of thing.
And remember that laypeople read "researchers used machine learning" as "super smart people used advanced science to get this result so it's probably true."
And that’s exactly the point.
Sadly this _is_ an area which is indeed full of bias.
The color of Augustus here[0] is definitely blonde for me, bordering on platinum, while what I meant is that it might as well have been darker, i.e. [1] which I would call "light chestnut".
[0] https://voshart.com/ROMAN-EMPEROR-PROJECT [1] https://external-content.duckduckgo.com/iu/?u=https%3A%2F%2F...
This wasn't at the Jurassic age.
These days, sadly, yes. The definition of racism has been so diluted in recent years that, as you implied, I fear it's often a distraction from actual racial injustice. Anyone ever hear of the boy who cried wolf?
Downvote away! :)
If someone is prone to accepting the idea that caucasians are superior to others they may use something as seemingly innocuous as this neat project to justify their positions and the lack of disclaimers towards the inaccuracies of certain aspects of these renderings may do a small harm societally.
On its own it’d be fine but in conjunction with other symbols and narratives it can become part of a larger, more sinister narrative.
What I am saying is that this sort of work can lead to people believing these images as true, i.e. "this is what Roman Emperors looked like". If you ask an ML system to solve a problem for which the inputs cannot possibly determine the answer (i.e. the color of a fossilized animal or the skin color from a person's statue), it will give you a highly biased answer.
I fear that some laypeople think of ML as "magic software that can do anything" without considering whether a problem is non-ambiguous given its inputs (e.g. something like object classification is less ambiguous than this problem).
I'm not saying "burn it to the ground" - I wanted to learn more about what data and model was used to generate these images so that I might understand better where potential sources of racial / ethnographic bias might come from.
> trying to find racial undertones in every tangential thing
If you read my comments history you'll see that I'm no critical race theorist, nor do I discuss everything in a racial context.
Out of curiosity, what "cause" are you talking about?
The same can be said of the original advanced technology used to render a facsimile of the Roman Emperors, and in fact .. many stone sculptors have lost their prestige for the sake of this very same argument.
Since culture is a lie which persists only in the telling, this IS what the Roman Emperors look like, now. You shouldn't resist that, because there is no truth on this issue, anyway.
Saying "we'll never know anyway so stop asking" is abhorrent to me, and I imagine many on this forum. Not to mention it aims to end a discussion it doesn't understand.
Money is a fiction, history is a fiction, most scientific concepts eventually become fiction at a low enough level. And look at the world we've built on top of those fictions. It may not be perfect but it is undeniably meaningful.
Or, we can just assume we are already correct because we'll never know anyway.
A realistic ML model could theoretically compensate for that by reintroducing a bit of variability based on the variability we see in today's population of that region.
It is generally agreed that the invasions that followed for centuries after the fall of the Roman Empire did not significantly alter the local gene pool, because of the relatively small number of Germanics, or other migrants, compared to the large population of what constituted Roman Italy.
In 2008, Dutch geneticists determined that Italy is one of the last two remaining genetic islands in Europe, the other being Finland. This is due in part to the presence of the Alpine mountain chain which, over the centuries, has prevented large migration flows aimed at colonizing the Italian lands[1]
The migrants comprised war bands or tribes of 10,000 to 20,000 people,[5] but in the course of 100 years they numbered not more than 750,000 in total,[citation needed] compared to an average 40 million population of the Roman Empire at that time.[2]
I might call someone blond with blue eyes, but someone else might describe the same person hair with light brown with green eyes. If we had third person choose which colors represented the descriptions, they might pick completely different colors based on what they thought blond or light brown meant.
Caligula looks to be on point based on the restoration of his original colors in that article.
To your second point, that reminded me that Japan's traffic signals use a different color "green" (looks more blueish) than the rest of the world and they have a common, named color that doesn't really map to blue or green in the rest of the world: https://en.wikipedia.org/wiki/Ao_(color)
My understanding is that the racial identity of the ancient Romans are actually a lot more ambiguous then most people realize, and the way the author attempts to reconcile that ambiguity has a huge impact on the image output here. And it's an interesting problem, Yann LeCun has discussed a bit about how to account for racial diversity via ML techniques (it's not just about the training data set!).
From what I recall, the reason for this racial ambiguity is because ancient Rome (and the entire Mediterranean region) was a cosmopolitan hotbed where many different ethnicities were mixing together all the time. On top of that, they didn't have the same kind of racial constructs we had, and so it's actually unclear what the races of various ancient Romans actually were.
Your comment frustrates me greatly because you are correct that people may internalize ML-generated images, and that's a real risk. ML already propagates racial bias within our prison system, as I'm sure you're aware. But Voshart's model isn't making decisions about employment or imprisonment, no, he's literally making art.
Art needs to be allowed uncomfortable racial and ethnographic territory. If not art, what do you think should occupy that space? Art is supposed to be the domain where we get to try out all these ideas and give the public exposure to what things like ML-generated images can be.
If this art made you feel something racial, try to do something positive with that. Maybe get involved solving some of the real life ML deployments that are already causing problems: https://www.technologyreview.com/2019/01/21/137783/algorithm...
But mostly, I hope when the author reads your comment, he doesn't feel discouraged that doing anything in public might risk his career. Or that someone thinking about publishing their own art doesn't look at it and go "hmm, too risky maybe, I better not". The chilling effects that comments like yours have is much more damaging than any racial bias you might be trying to raise awareness about.
I remember when the conscious thing to do was to uplift the art of the unseen people in our world. I wish we could go back to that.
It gets into confusing territory when we talk about who is being "silenced" in a simple conversation where folks disagree, but I think the best outcome is that we sell to understand each other's points first, and then share our own. I hope I've demonstrated that here.
https://medium.com/@voshart/photoreal-roman-emperor-project-...
I have to agree with the grandparent, this seems like not very strong an underpinning for a project that clearly has the potential to be very controversial. There's no mention of accounting for the artist's potential implicit bias, the drift of ethnically-associated physiological features over the ~2000 years between the subjects and the data set that the process is based on, etc.
For a long time now, I've felt that the best practice for this sort of presentation isn't necessarily a static image, but an interactive experience that allows viewers to explore a range of reasonable outputs.
"each step towards realism is likely a step away from ground-truth." via https://www.smithsonianmag.com/smart-news/peer-past-these-ph... & https://twitter.com/dvoshart/status/1292107503543689217
I have not in my Roman Emperor write-up itself been as overt. I thought it might be a bit too obvious. My writeups make it clear when no information about skin tone is available.
I did include such a clause in a forensic facial reconstruction experiment: saying: "My gut instinct is that the results are simply easier to look at, that each step towards realism is a step away from ground-truth skull measurements." because it was an experiment where there are likely living relatives.
https://medium.com/forensic-vr/ai-forensic-facial-reconstruc...
Proof: https://twitter.com/dvoshart/status/1294977757756227585
My discomfort stems more from my belief that a non-technical / layperson audience is much more likely to conflate "each step towards realism" with a "step towards ground-truth", the exact opposite to what you suggested. The writeups make clear when information is not available, but the artistic medium itself (a rendered still image showing only one possibility) does not showcase this.
> In response to Cocci’s findings, Voshart removed all mentions of the site and revised several portraits to better reflect their subjects’ probable complexions
You graciously made corrections for plausibly Nazi historical propaganda. However, not every artist/historian may elect to do this, out of scholarly pride or a personal politics or just laziness. The artist subsequently wields tremendous power over the images that show up on Google Image search, and their decisions may unintentionally contaminate future historical research by way of inducing subtle prejudices.
We live in an age of misinformation - and I think the information that must be protected most carefully is information of the past, specifically around people. DeepFakes of living celebrities are fairly easy to verify and disprove, but as soon as you go back ~5 decades or so, some people start to challenge whether certain events (Holocaust, Tiananmen Square Massacre) even actually happened.
Some ideas on how one might mitigate this:
- Provenance technology that shows the editing history (potentially including a MD5 of the dataset) of how an image was synthesized.
- A simple, unobtrusive watermark on the center of the image indicating that it is but one of N possibilities, where N gives a rough estimate of the entropy over the distribution. Sort of like the recycling triangle numbers.
- GIF form showing multiple possibilities given the data you have.
As for GAN bias: once in Artbreeder the skin tone is entirely my choosing. If there is a bias in the depiction it is my own bias. There might be a slight bias towards a white, brow-haired female but is is more of a gravity that can be avoided with a bit of skill. There is also a challenge in hair that is -for lack of a better explanation- hair made with human intervention (braids, tall wigs, cornrows, dreads etc).
AS for your last suggestion: I've already been thinking of making videos that show a transition of skin ton of lightest plausible to darkest plausible. Tiberius for example I have a very compelling version as a pale-skinned white dude or a dark-skinned person who looks indian/pakistani. Done in paintover/colorization in Photoshop --directly on the bust- where proportions match.
In the second version of the poster I actually challenged myself to do at least do a quick depiction of each emperor with dark skin. Version 1 my approach was to assuming an average based on place of birth. This time I experimented with assume-darkest-features possible simply to test my assumptions and question my biases. Although quick and not robust, I think it was of value.
https://medium.com/@voshart/appearance-of-the-principate-pt-...
I wouldn’t be too worried about whether or not people view them as historical renditions. Most layman’s history “renditions” are mostly somewhat made up.
You have to ask the creator of ArtBreeder (https://www.artbreeder.com/about) which apparently uses BigGAN, StyleGAN, etc., trained on different datasets to offer these options:
https://i.imgur.com/5jTNgXl.jpg
That being said, there is a heavy manual touch (based on "Photoshop and historical references"). See for instance how different Septimus can appear compared to the webpage:
...Or were you saying that this is more diverse than is to be expected?