I wonder if Generative AIs are like Ouija boards. Every time I hear someone get freaked out about something that an AI wrote, they ascribe human characteristics to it. They anthropomorphize the computer program and act as if there’s a thinking being expressing human-like desires and feelings. But what the AI produces exists in dialog with the humans who read it. It both comes from human thought (because it has access to a vast quantity of human writing) and it gets filtered through human brains.
We interpret works of art the same way, as if if each piece has a specific, inherent, immutable meaning. But each of us brings something to a work of art. We bring our experiences and biases and we project our own identities on the work in front of us. We create meaning in dialog with the art. The art doesn’t mean anything without an observer. A blob of AI text doesn’t mean anything until we invest it with meaning.
The current crop of AIs are really good at what they do, but they don’t think any more than a Ouija board thinks. A Ouija board isn’t controlled by a supernatural being, but we can convince ourselves that it is if we want to. The AIs are a reflection of ourselves, and that reflection can often fool us into thinking there’s something in there. Like a parakeet with a mirror. It’s a really cool and useful trick, but let’s not give the really clever software more credit than it deserves.