• @flango@lemmy.eco.br
    link
    fedilink
    English
    844 minutes ago

    […] I read through dozens of the AI comments, and although they weren’t all brilliant, most of them seemed reasonable and genuine enough. They made a lot of good points, and I found myself nodding along more than once. As the Zurich researchers warn, without more robust detection tools, AI bots might “seamlessly blend into online communities”—that is, assuming they haven’t already.

  • @Ensign_Crab@lemmy.world
    link
    fedilink
    English
    123 minutes ago

    Imagine what the people doing this professionally do, since they know they won’t face the scrutiny of publication.

  • @perestroika@lemm.ee
    link
    fedilink
    English
    8
    edit-2
    1 hour ago

    The University of Zurich’s ethics board—which can offer researchers advice but, according to the university, lacks the power to reject studies that fall short of its standards—told the researchers before they began posting that “the participants should be informed as much as possible,” according to the university statement I received. But the researchers seem to believe that doing so would have ruined the experiment. “To ethically test LLMs’ persuasive power in realistic scenarios, an unaware setting was necessary,” because it more realistically mimics how people would respond to unidentified bad actors in real-world settings, the researchers wrote in one of their Reddit comments.

    This seems to be the kind of a situation where, if the researchers truly believe their study is necessary, they have to:

    • accept that negative publicity will result
    • accept that people may stop cooperating with them on this work
    • accept that their reputation will suffer as a result
    • ensure that they won’t do anything illegal

    After that, if they still feel their study is necesary, maybe they should run it and publish the results.

    If then, some eager redditors start sending death threats, that’s unfortunate. I would catalouge them, but not report them anywhere unless something actually happens.

    As for the question of whether a tailor-made response considering someone’s background can sway opinions better - that’s been obvious through ages of diplomacy. (If you approach an influential person with a weighty proposal, it has always been worthwhile to know their background, think of several ways of how they might perceive the proposal, and advance your explanation in a way that relates better with their viewpoint.)

    AI bots which take into consideration a person’s background will - if implemented right - indeed be more powerful at swaying opinions.

    As to whether secrecy was really needed - the article points to other studies which apparently managed to prove the persuasive capability of AI bots without deception and secrecy. So maybe it wasn’t needed after all.

  • @conicalscientist@lemmy.world
    link
    fedilink
    English
    192 hours ago

    This is probably the most ethical you’ll ever see it. There are definitely organizations committing far worse experiments.

    Over the years I’ve noticed replies that are far too on the nose. Probing just the right pressure points as if they dropped exactly the right breadcrumbs for me to respond to. I’ve learned to disengage at that point. It’s either they scrolled through my profile. Or as we now know it’s a literal psy-op bot. Already in the first case it’s not worth engaging with someone more invested than I am myself.

    • @skisnow@lemmy.ca
      link
      fedilink
      English
      62 hours ago

      Yeah I was thinking exactly this.

      It’s easy to point to reasons why this study was unethical, but the ugly truth is that bad actors all over the world are performing trials exactly like this all the time - do we really want the only people who know how this kind of manipulation works to be state psyop agencies, SEO bros, and astroturfing agencies working for oil/arms/religion lobbyists?

      Seems like it’s much better long term to have all these tricks out in the open so we know what we’re dealing with, because they’re happening whether it gets published or not.

  • @Knock_Knock_Lemmy_In@lemmy.world
    link
    fedilink
    English
    283 hours ago

    The key result

    When researchers asked the AI to personalize its arguments to a Redditor’s biographical details, including gender, age, and political leanings (inferred, courtesy of another AI model, through the Redditor’s post history), a surprising number of minds indeed appear to have been changed. Those personalized AI arguments received, on average, far higher scores in the subreddit’s point system than nearly all human commenters

    • @thanksforallthefish@literature.cafe
      link
      fedilink
      English
      231 minutes ago

      While that is indeed what was reported, we and the researchers will never know if the posters with shifted opinions were human or in fact also AI bots.

      The whole thing is dodgy for lack of controls, this isn’t science it’s marketing

    • @taladar@sh.itjust.works
      link
      fedilink
      English
      73 hours ago

      If they were personalized wouldn’t that mean they shouldn’t really receive that many upvotes other than maybe from the person they were personalized for?

      • @the_strange@feddit.org
        link
        fedilink
        English
        63 hours ago

        I would assume that people in a similar demographics are interested in similar topics. Adjusting the answer to a person within a demographic would therefore adjust it to all people within that demographic and interested in that specific topic.

        Or maybe it’s just the nature of the answer being more personal that makes it more appealing to people in general, no matter their background.

    • Geetnerd
      link
      fedilink
      English
      94 hours ago

      Yes. Much more than we peasants all realized.

      • @CBYX@feddit.org
        link
        fedilink
        English
        73 hours ago

        Not sure how everyone hasn’t expected Russia has been doing this the whole time on conservative subreddits…

        • @skisnow@lemmy.ca
          link
          fedilink
          English
          62 hours ago

          Russia are every bit as active in leftist groups whipping them up into a frenzy too. There was even a case during BLM where the same Russian troll farm organised both a protest and its counter-protest. Don’t think you’re immune to being manipulated to serve Russia’s long-term interests just because you’re not a conservative.

          They don’t care about promoting right-wing views, they care about sowing division. They support Trump because Trump sows division. Their long-term goal is to break American hegemony.

        • Geetnerd
          link
          fedilink
          English
          52 hours ago

          Those of us who are not idiots have known this for a long time.

          They beat the USA without firing a shot.

        • @taladar@sh.itjust.works
          link
          fedilink
          English
          43 hours ago

          Mainly I didn’t really expect that since the old methods of propaganda before AI use worked so well for the US conservatives’ self-destructive agenda that it didn’t seem necessary.

  • @TheObviousSolution@lemm.ee
    link
    fedilink
    English
    445 hours ago

    The reason this is “The Worst Internet-Research Ethics Violation” is because it has exposed what Cambridge Analytica’s successors already realized and are actively exploiting. Just a few months ago it was literally Meta itself running AI accounts trying to pass off as normal users, and not an f-ing peep - why do people think they, the ones who enabled Cambridge Analytica, were trying this shit to begin with. The only difference now is that everyone doing it knows to do it as a “unaffiliated” anonymous third party.

    • @tauren@lemm.ee
      link
      fedilink
      English
      24 hours ago

      Just a few months ago it was literally Meta itself…

      Well, it’s Meta. When it comes to science and academic research, they have rather strict rules and committees to ensure that an experiment is ethical.

      • @thanksforallthefish@literature.cafe
        link
        fedilink
        English
        226 minutes ago

        You may wish to reword. The unspecified “they” reads like you think Meta have strict ethical rules. Lol.

        Meta have no ethics whatsoever, and yes I assume you meant universities have strict rules however the approval of this study marks even that as questionable

      • @FarceOfWill@infosec.pub
        link
        fedilink
        English
        62 hours ago

        The headline is that they advertised beauty products to girls after they detected them deleting a selfie. No ethics or morals at all

  • @mke@programming.dev
    link
    fedilink
    English
    6
    edit-2
    2 hours ago

    Another isolated case for the endlessly growing list of positive impacts of the GenAI with no accountability trend. A big shout-out to people promoting and fueling it, excited to see into what pit you lead us next.

    This experiment is also nearly worthless because, as proved by the researchers, there’s no guarantee the accounts you interact with on Reddit are actual humans. Upvotes are even easier for machines to use, and can be bought for cheap.

    • @vivendi@programming.dev
      link
      fedilink
      English
      1
      edit-2
      2 hours ago

      ?!!? Before genAI it was hires human manipulators. Ypur argument doesn’t exist. We cannot call edison a witch and go back in caves because new tech creates new threat landscapes.

      Humanity adapts to survive and survives to adapt. We’ll figure some shit out

  • @teamevil@lemmy.world
    link
    fedilink
    English
    44
    edit-2
    9 hours ago

    Holy Shit… This kind of shit is what ultimately broke Tim kaczynski… He was part of MKULTRA research while a student at Harvard, but instead of drugging him, they had a debater that was a prosecutor pretending to be a student… And would just argue against any point he had to see when he would break…

    And that’s how you get the Unabomber folks.

    • Geetnerd
      link
      fedilink
      English
      114 hours ago

      I don’t condone what he did in any way, but he was a genius, and they broke his mind.

      Listen to The Last Podcast on the Left’s episode on him.

      A genuine tragedy.

  • @ImplyingImplications@lemmy.ca
    link
    fedilink
    English
    6610 hours ago

    The ethics violation is definitely bad, but their results are also concerning. They claim their AI accounts were 6 times more likely to persuade people into changing their minds compared to a real life person. AI has become an overpowered tool in the hands of propagandists.

    • Joe
      link
      fedilink
      English
      73 hours ago

      It would be naive to think this isn’t already in widespread use.

    • ArchRecord
      link
      fedilink
      English
      73 hours ago

      To be fair, I do believe their research was based on how convincing it was compared to other Reddit commenters, rather than say, an actual person you’d normally see doing the work for a government propaganda arm, with the training and skillset to effectively distribute propaganda.

      Their assessment of how “convincing” it was seems to also have been based on upvotes, which if I know anything about how people use social media, and especially Reddit, are often given when a comment is only slightly read through, and people are often scrolling past without having read the whole thing. The bots may not have necessarily optimized for convincing people, but rather, just making the first part of the comment feel upvote-able over others, while the latter part of the comment was mostly ignored. I’d want to see more research on this, of course, since this seems like a major flaw in how they assessed outcomes.

      This, of course, doesn’t discount the fact that AI models are often much cheaper to run than the salaries of human beings.

  • @vordalack@lemm.ee
    link
    fedilink
    English
    -83 hours ago

    This just shows how gullible and stupid the average Reddit user is. There’s a reason that there’s so many meme’s mocking them and calling them beta soyjacks.

    It’s kind of true.

  • @TwinTitans@lemmy.world
    link
    fedilink
    English
    86
    edit-2
    6 hours ago

    Like the 90s/2000s - don’t put personal information on the internet, don’t believe a damned thing on it either.

    • @taladar@sh.itjust.works
      link
      fedilink
      English
      33 hours ago

      I never liked the “don’t believe anything you read on the internet” line, it focuses too much on the internet without considering that you shouldn’t believe anything you read or hear elsewhere either, especially on divisive topics like politics.

      You should evaluate information you receive from any source with critical thinking, consider how easy it is to make false claims (e.g. probably much harder for a single source if someone claims that the US president has been assassinated than if someone claims their local bus was late that one unspecified day at their unspecified location), who benefits from convincing you of the truth of a statement, is the statement consistent with other things you know about the world,…

    • @mic_check_one_two@lemmy.dbzer0.com
      link
      fedilink
      English
      5411 hours ago

      Yeah, it’s amazing how quickly the “don’t trust anyone on the internet” mindset changed. The same boomers who were cautioning us against playing online games with friends are now the same ones sharing blatantly AI generated slop from strangers on Facebook as if it were gospel.

      • @Serinus@lemmy.world
        link
        fedilink
        English
        2910 hours ago

        Back then it was just old people trying to groom 16 year olds. Now it’s a nation’s intelligence apparatus turning our citizens against each other and convincing them to destroy our country.

        I wholeheartedly believe they’re here, too. Their primary function here is to discourage the left from voting, primarily by focusing on the (very real) failures of the Democrats while the other party is extremely literally the Nazi party.

        • queermunist she/her
          link
          fedilink
          English
          -18
          edit-2
          8 hours ago

          Everyone who disagrees with you is a bot, probably from Russia. You are very smart.

          Do you still think you’re going to be allowed to vote for the next president?

          • @Serinus@lemmy.world
            link
            fedilink
            English
            158 hours ago

            Everyone who disagrees with you is a bot

            I mean that’s unironically the problem. When there absolutely are bots out here, how do you tell?

            • queermunist she/her
              link
              fedilink
              English
              -17 hours ago

              Sure, but you seem to be under the impression the only bots are the people that disagree with you.

              There’s nothing stopping bots from grooming you by agreeing with everything you say.

      • 🇰 🌀 🇱 🇦 🇳 🇦 🇰 🇮
        link
        fedilink
        English
        1010 hours ago

        I feel like I learned more about the Internet and shit from Gen X people than from boomers. Though, nearly everyone on my dad’s side of the family, including my dad (a boomer), was tech literate, having worked in tech (my dad is a software engineer) and still continue to not be dumb about tech… Aside from thinking e-greeting cards are rad.

        • Alphane Moon
          link
          fedilink
          English
          56 hours ago

          e-greeting cards

          Haven’t even thought about them in what seems like a quarter of a century.