What happens when AI generated media becomes ubiquitous in our lives? How does this relate to what we\u2019ve experienced before, and how does it change\u00a0us?<\/h4>\n![\"\"](\"https:\/\/cdn-images-1.medium.com\/max\/1024\/0*lFCUFUZBGqHZaTVR\")
Photo by Annie Spratt<\/a> on\u00a0Unsplash<\/a><\/figcaption><\/figure>\nThis is the first part of a two part series I\u2019m writing analyzing how people and communities are affected by the expansion of AI generated content. I\u2019ve already talked at some length about the environmental<\/a>, economic<\/a>, and labor issues<\/a> involved, as well as discrimination and social bias<\/a>. But this time I want to dig in a little and focus on some psychological and social impacts from the AI generated media and content we consume, specifically on our relationship to critical thinking, learning, and conceptualizing knowledge.<\/p>\nHistory<\/h3>\n
Hoaxes have been perpetrated using photography essentially since its invention<\/a>. The moment we started having a form of media that was believed to show us true, unmediated reality of phenomena and events, was the moment that people started coming up with ways to manipulate that form of media, to great artistic and philosophical effect. (As well as humorous or simply fraudulent effect.) We have a form of unwarranted trust in photographs, despite this, and we have developed a relationship with the form that balances between trust and skepticism.<\/p>\nWhen I was a child, the internet was not yet broadly available to the general public, and certainly very few homes had access to it, but by the time I was a teenager that had completely changed, and everyone I knew spent time on AOL instant messenger. Around the time I left graduate school, the iPhone was launched and the smartphone era started. I retell all this to make the point that cultural creation and consumption changed startlingly quickly and beyond recognition in just a couple of\u00a0decades.<\/p>\n
I think the current moment represents a whole new era specifically in the media and cultural content we consume and create, because of the launch of generative AI. It\u2019s a little like when Photoshop became broadly available, and we started to realize that photos were sometimes retouched, and we began to question whether we could trust what images looked like. (Readers may find the ongoing conversation around \u201cwhat is a photograph\u201d<\/a> an interesting extension of this issue.) But even then, Photoshop was expensive and had a skill level requirement to use it effectively, so most photos we encountered were relatively true to life, and I think people generally expected that images in advertising and film were not going to be \u201creal\u201d. Our expectations and intuitions had to adjust to the changes in technology, and we more or less\u00a0did.<\/p>\nCurrent Day<\/h3>\n
Today, AI content generators have democratized the ability to artificially produce or alter any kind of content, including images. Unfortunately, it\u2019s extremely difficult to get an estimate of how much of the content online may be AI-generated\u200a\u2014\u200aif you google this question you\u2019ll get references<\/a> to<\/a> an article from Europol<\/a> claiming it says that the number will be 90% by 2026\u200a\u2014\u200abut read it and you\u2019ll see that the research paper says nothing of the sort. You might also find a paper by some AWS researchers<\/a> being<\/a> cited<\/a>, saying that 57% is the number\u200a\u2014\u200abut that\u2019s also a mistaken reading (they\u2019re talking about text content being machine translated, not text generated from whole cloth, to say nothing of images or video). As far as I can tell, there\u2019s no reliable, scientifically based work indicating actually how much of the content we consume may be AI generated\u200a\u2014\u200aand even if it did, the moment it was published it would be outdated.<\/p>\nBut if you think about it, this is perfectly sensible. A huge part of the reason AI generated content keeps coming is because it\u2019s harder than ever before in human history to tell whether a human being actually created what you are looking at, and whether that representation is a reflection of reality. How do you count something, or even estimate a count, when it\u2019s explicitly unclear how you can identify it in the first\u00a0place?<\/p>\n
I think we all have the lived experience of spotting content with questionable provenance. We see images that seem to be in the uncanny valley, or strongly suspect that a product review on a retail site sounds unnaturally positive and generic, and think, that must have been created using generative AI and a bot. Ladies, have you tried to find inspiration pictures for a haircut online recently? In my own personal experience, 50%+ of the pictures on Pinterest or other such sites are clearly AI generated, with tell-tale signs: textureless skin, rubbery features, straps and necklaces disappearing into nowhere, images explicitly not including hands, never showing both ears straight on, etc. These are easy to dismiss, but a large swath makes you question whether you\u2019re seeing heavily filtered real images or wholly AI generated content. I make it my business to understand these things, and I\u2019m often not sure myself. I hear tell that single men on dating apps are so swamped with scamming bots based on generative AI that there\u2019s a name for the way to check\u200a\u2014\u200athe \u201cPotato Test\u201d.<\/a> If you ask the bot to say \u201cpotato\u201d it will ignore you, but a real human person will likely do it. The small, everyday areas of our lives are being infiltrated by AI content without anything like our consent or approval.<\/p>\nWhy?<\/h3>\n
What\u2019s the point of dumping AI slop in all these online spaces? The best case scenario goal may be to get folks to click through to sites where advertising lives, offering nonsense text and images just convincing enough to get those precious ad impressions and get a few cents from the advertiser. Artificial reviews and images for online products are generated by the truckload, so that drop-shippers and vendors of cheap junk can fool customers into buying something that\u2019s just a little cheaper than all the competition, letting them hope they\u2019re getting a legitimate item. Perhaps the item can be so incredibly cheap that the disappointed buyer will just accept the loss and not go to the trouble of getting their money\u00a0back.<\/p>\n
Worse, bots using LLMs to generate text and images can be used to lure people into scams, and because the only real resource necessary is compute, the scaling of such scams costs pennies\u200a\u2014\u200awell worth the expense if you can steal even one person\u2019s money every so often. AI generated content is used for criminal abuse, including pig butchering scams<\/a>, AI-generated<\/a> CSAM<\/a> and non-consensual intimate images<\/a>, which can turn into blackmail schemes as\u00a0well.<\/p>\nThere are also political motivations for AI-generated images<\/a>, video, and text\u200a\u2014\u200ain this US election year, entities all across the world with different angles and objectives produced AI-generated images and videos to support their viewpoints, and spewed propagandistic messages via generative AI bots to social media, especially on the former Twitter, where content moderation to prevent abuse, harassment, and bigotry has largely ceased. The expectation from those disseminating this material is that uninformed internet users will absorb their message through continual, repetitive exposure to this content, and for every item they realize is artificial, an unknown number will be accepted as legitimate. Additionally, this material creates an information ecosystem where truth is impossible to define or prove, neutralizing good actors and their attempts to cut through the\u00a0noise.<\/p>\nA small minority of the AI-generated content online will be actual attempts to create appealing images just for enjoyment, or relatively harmless boilerplate text generated to fill out corporate websites, but as we are all well aware, the internet is rife with scams and get-rich-quick schemers, and the advances of generative AI have brought us into a whole new era for these sectors. (And, these applications have massive negative implications for real creators, energy and the environment, and other\u00a0issues.)<\/p>\n
Where we\u00a0Are<\/h3>\n
I\u2019m painting a pretty grim picture of our online ecosystems, I realize. Unfortunately, I think it\u2019s accurate and only getting worse. I\u2019m not arguing that there\u2019s no good use of generative AI, but I\u2019m becoming more and more convinced that the downsides for our society are going to have a larger, more direct, and more harmful impact than the positives.<\/p>\n
I think about it this way: We\u2019ve reached a point where it is unclear if we can trust what we see or read, and we routinely can\u2019t know if entities we encounter online are human or AI. What does this do to our reactions to what we encounter? It would be silly to expect our ways of thinking to not change as a result of these experiences, and I worry very much that the change we\u2019re undergoing is not for the\u00a0better.<\/p>\n
The ambiguity is a big part of the challenge, however. It\u2019s not that we know that we\u2019re consuming untrustworthy information, it\u2019s that it\u2019s essentially unknowable. We\u2019re never able to be sure. Critical thinking and critical media consumption habits help, but the expansion of AI generated content may be outstripping our critical capabilities, at least in some cases. This seems to me to have a real implication for our concepts of trust and confidence in information.<\/p>\n
In my next article, I\u2019ll discuss in detail what kind of effects this may have on our thoughts and ideas about the world around us, and consider what, if anything, our communities might do about\u00a0it.<\/p>\n
Read more of my work at www.stephaniekirmer.com.<\/a><\/p>\nAlso, regular readers will know I publish on a two week schedule, but I am moving to a monthly publishing cadence going forward. Thank you for reading, and I look forward to continuing to share my\u00a0ideas!<\/p>\n
Further Reading<\/h3>\n
https:\/\/www.theverge.com\/2024\/2\/2\/24059955\/samsung-no-such-thing-as-real-photo-ai<\/a><\/p>\nhttps:\/\/arxiv.org\/pdf\/2401.05749<\/a>\u200a\u2014\u200aBrian Thompson, Mehak Preet Dhaliwal, Peter Frisch, Tobias Domhan, and Marcello Federico of\u00a0AWS<\/p>\nhttps:\/\/www.europol.europa.eu\/cms\/sites\/default\/files\/documents\/Europol_Innovation_Lab_Facing_Reality_Law_Enforcement_And_The_Challenge_Of_Deepfakes.pdf<\/a><\/p>\nhttps:\/\/www.404media.co\/ai-generated-child-sexual-abuse-material-is-not-a-victimless-crime\/<\/a><\/p>\nhttps:\/\/www.404media.co\/fbi-arrests-man-for-generating-ai-child-sexual-abuse-imagery\/<\/a><\/p>\nInstagram Advertises Nonconsensual AI Nude Apps<\/a><\/p>\nhttps:\/\/www.smithsonianmag.com\/innovation\/history-spirit-photography-future-deepfake-videos-180979010<\/a><\/p>\nhttps:\/\/www.brennancenter.org\/our-work\/research-reports\/generative-ai-political-advertising<\/a><\/p>\n![\"\"](\"https:\/\/medium.com\/_\/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e6a8a51800f\")
<\/p>\n
\nThe Cultural Impact of AI Generated Content: Part 1<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\n
This is the first part of a two part series I\u2019m writing analyzing how people and communities are affected by the expansion of AI generated content. I\u2019ve already talked at some length about the environmental<\/a>, economic<\/a>, and labor issues<\/a> involved, as well as discrimination and social bias<\/a>. But this time I want to dig in a little and focus on some psychological and social impacts from the AI generated media and content we consume, specifically on our relationship to critical thinking, learning, and conceptualizing knowledge.<\/p>\n Hoaxes have been perpetrated using photography essentially since its invention<\/a>. The moment we started having a form of media that was believed to show us true, unmediated reality of phenomena and events, was the moment that people started coming up with ways to manipulate that form of media, to great artistic and philosophical effect. (As well as humorous or simply fraudulent effect.) We have a form of unwarranted trust in photographs, despite this, and we have developed a relationship with the form that balances between trust and skepticism.<\/p>\n When I was a child, the internet was not yet broadly available to the general public, and certainly very few homes had access to it, but by the time I was a teenager that had completely changed, and everyone I knew spent time on AOL instant messenger. Around the time I left graduate school, the iPhone was launched and the smartphone era started. I retell all this to make the point that cultural creation and consumption changed startlingly quickly and beyond recognition in just a couple of\u00a0decades.<\/p>\n I think the current moment represents a whole new era specifically in the media and cultural content we consume and create, because of the launch of generative AI. It\u2019s a little like when Photoshop became broadly available, and we started to realize that photos were sometimes retouched, and we began to question whether we could trust what images looked like. (Readers may find the ongoing conversation around \u201cwhat is a photograph\u201d<\/a> an interesting extension of this issue.) But even then, Photoshop was expensive and had a skill level requirement to use it effectively, so most photos we encountered were relatively true to life, and I think people generally expected that images in advertising and film were not going to be \u201creal\u201d. Our expectations and intuitions had to adjust to the changes in technology, and we more or less\u00a0did.<\/p>\n Today, AI content generators have democratized the ability to artificially produce or alter any kind of content, including images. Unfortunately, it\u2019s extremely difficult to get an estimate of how much of the content online may be AI-generated\u200a\u2014\u200aif you google this question you\u2019ll get references<\/a> to<\/a> an article from Europol<\/a> claiming it says that the number will be 90% by 2026\u200a\u2014\u200abut read it and you\u2019ll see that the research paper says nothing of the sort. You might also find a paper by some AWS researchers<\/a> being<\/a> cited<\/a>, saying that 57% is the number\u200a\u2014\u200abut that\u2019s also a mistaken reading (they\u2019re talking about text content being machine translated, not text generated from whole cloth, to say nothing of images or video). As far as I can tell, there\u2019s no reliable, scientifically based work indicating actually how much of the content we consume may be AI generated\u200a\u2014\u200aand even if it did, the moment it was published it would be outdated.<\/p>\n But if you think about it, this is perfectly sensible. A huge part of the reason AI generated content keeps coming is because it\u2019s harder than ever before in human history to tell whether a human being actually created what you are looking at, and whether that representation is a reflection of reality. How do you count something, or even estimate a count, when it\u2019s explicitly unclear how you can identify it in the first\u00a0place?<\/p>\n I think we all have the lived experience of spotting content with questionable provenance. We see images that seem to be in the uncanny valley, or strongly suspect that a product review on a retail site sounds unnaturally positive and generic, and think, that must have been created using generative AI and a bot. Ladies, have you tried to find inspiration pictures for a haircut online recently? In my own personal experience, 50%+ of the pictures on Pinterest or other such sites are clearly AI generated, with tell-tale signs: textureless skin, rubbery features, straps and necklaces disappearing into nowhere, images explicitly not including hands, never showing both ears straight on, etc. These are easy to dismiss, but a large swath makes you question whether you\u2019re seeing heavily filtered real images or wholly AI generated content. I make it my business to understand these things, and I\u2019m often not sure myself. I hear tell that single men on dating apps are so swamped with scamming bots based on generative AI that there\u2019s a name for the way to check\u200a\u2014\u200athe \u201cPotato Test\u201d.<\/a> If you ask the bot to say \u201cpotato\u201d it will ignore you, but a real human person will likely do it. The small, everyday areas of our lives are being infiltrated by AI content without anything like our consent or approval.<\/p>\n What\u2019s the point of dumping AI slop in all these online spaces? The best case scenario goal may be to get folks to click through to sites where advertising lives, offering nonsense text and images just convincing enough to get those precious ad impressions and get a few cents from the advertiser. Artificial reviews and images for online products are generated by the truckload, so that drop-shippers and vendors of cheap junk can fool customers into buying something that\u2019s just a little cheaper than all the competition, letting them hope they\u2019re getting a legitimate item. Perhaps the item can be so incredibly cheap that the disappointed buyer will just accept the loss and not go to the trouble of getting their money\u00a0back.<\/p>\n Worse, bots using LLMs to generate text and images can be used to lure people into scams, and because the only real resource necessary is compute, the scaling of such scams costs pennies\u200a\u2014\u200awell worth the expense if you can steal even one person\u2019s money every so often. AI generated content is used for criminal abuse, including pig butchering scams<\/a>, AI-generated<\/a> CSAM<\/a> and non-consensual intimate images<\/a>, which can turn into blackmail schemes as\u00a0well.<\/p>\n There are also political motivations for AI-generated images<\/a>, video, and text\u200a\u2014\u200ain this US election year, entities all across the world with different angles and objectives produced AI-generated images and videos to support their viewpoints, and spewed propagandistic messages via generative AI bots to social media, especially on the former Twitter, where content moderation to prevent abuse, harassment, and bigotry has largely ceased. The expectation from those disseminating this material is that uninformed internet users will absorb their message through continual, repetitive exposure to this content, and for every item they realize is artificial, an unknown number will be accepted as legitimate. Additionally, this material creates an information ecosystem where truth is impossible to define or prove, neutralizing good actors and their attempts to cut through the\u00a0noise.<\/p>\n A small minority of the AI-generated content online will be actual attempts to create appealing images just for enjoyment, or relatively harmless boilerplate text generated to fill out corporate websites, but as we are all well aware, the internet is rife with scams and get-rich-quick schemers, and the advances of generative AI have brought us into a whole new era for these sectors. (And, these applications have massive negative implications for real creators, energy and the environment, and other\u00a0issues.)<\/p>\n I\u2019m painting a pretty grim picture of our online ecosystems, I realize. Unfortunately, I think it\u2019s accurate and only getting worse. I\u2019m not arguing that there\u2019s no good use of generative AI, but I\u2019m becoming more and more convinced that the downsides for our society are going to have a larger, more direct, and more harmful impact than the positives.<\/p>\n I think about it this way: We\u2019ve reached a point where it is unclear if we can trust what we see or read, and we routinely can\u2019t know if entities we encounter online are human or AI. What does this do to our reactions to what we encounter? It would be silly to expect our ways of thinking to not change as a result of these experiences, and I worry very much that the change we\u2019re undergoing is not for the\u00a0better.<\/p>\n The ambiguity is a big part of the challenge, however. It\u2019s not that we know that we\u2019re consuming untrustworthy information, it\u2019s that it\u2019s essentially unknowable. We\u2019re never able to be sure. Critical thinking and critical media consumption habits help, but the expansion of AI generated content may be outstripping our critical capabilities, at least in some cases. This seems to me to have a real implication for our concepts of trust and confidence in information.<\/p>\n In my next article, I\u2019ll discuss in detail what kind of effects this may have on our thoughts and ideas about the world around us, and consider what, if anything, our communities might do about\u00a0it.<\/p>\n Read more of my work at www.stephaniekirmer.com.<\/a><\/p>\n Also, regular readers will know I publish on a two week schedule, but I am moving to a monthly publishing cadence going forward. Thank you for reading, and I look forward to continuing to share my\u00a0ideas!<\/p>\n https:\/\/www.theverge.com\/2024\/2\/2\/24059955\/samsung-no-such-thing-as-real-photo-ai<\/a><\/p>\n https:\/\/arxiv.org\/pdf\/2401.05749<\/a>\u200a\u2014\u200aBrian Thompson, Mehak Preet Dhaliwal, Peter Frisch, Tobias Domhan, and Marcello Federico of\u00a0AWS<\/p>\n https:\/\/www.europol.europa.eu\/cms\/sites\/default\/files\/documents\/Europol_Innovation_Lab_Facing_Reality_Law_Enforcement_And_The_Challenge_Of_Deepfakes.pdf<\/a><\/p>\n https:\/\/www.404media.co\/ai-generated-child-sexual-abuse-material-is-not-a-victimless-crime\/<\/a><\/p>\n https:\/\/www.404media.co\/fbi-arrests-man-for-generating-ai-child-sexual-abuse-imagery\/<\/a><\/p>\n Instagram Advertises Nonconsensual AI Nude Apps<\/a><\/p>\n https:\/\/www.smithsonianmag.com\/innovation\/history-spirit-photography-future-deepfake-videos-180979010<\/a><\/p>\n https:\/\/www.brennancenter.org\/our-work\/research-reports\/generative-ai-political-advertising<\/a><\/p>\n The Cultural Impact of AI Generated Content: Part 1<\/a> was originally published in Towards Data Science<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>\n<\/div>\nHistory<\/h3>\n
Current Day<\/h3>\n
Why?<\/h3>\n
Where we\u00a0Are<\/h3>\n
Further Reading<\/h3>\n
<\/p>\n
\n