In Pursuit of Understanding and Connection: Three Experiments with Flawed Technologies
by Carrie Sijia Wang
As a child, I believed being understood through words was a default. I thought people could effortlessly comprehend my inner thoughts and feelings by listening to what I say, as if my words could grant them direct access to a part of my mind. No deception. No misunderstanding. I also took people’s words at face value, unaware of the translation process occurring between my mind and theirs through the medium of language. And it wasn’t just the words themselves; the context, the entire history of the person I was conversing with, the situation at the moment, their emotional state, and whether they had a specific intention, a goal they wanted to achieve through their words—all stood in the middle of what I genuinely thought and what they genuinely thought.
But is there such a thing as a genuine thought? Can I trust my own written words in the journal from twenty years ago when I expressed: “I despise all adults, they are all hypocrites.” Or maybe I simply wanted to sound cool for my personal reading experience.
Sometimes, in emotionally charged situations, despite my full intention to be sincere, to bare my soul, the very closeness between me and the person I’m talking to becomes a setback. My feelings get in the way, making it impossible for me to articulate the most fitting words or for the other person to properly understand my sentiments.
When I moved from a Mandarin-speaking environment to an English-speaking one years ago, the layer of translation grew thicker. I now not only need to interpret minds and words, but switch between languages.
Through it all, I’ve somehow managed to communicate with myself and others, finding joy in the rare moments of genuine connection. But how does it all work? Is there a way to enhance interpersonal understanding?
As an artist, I use digital technologies both as tools to make interactive work and as subjects of critique. At some point, I found myself wondering: can these technical tools, often accused of driving us apart from ourselves and others, be utilized in unexpected ways to bring us closer?
Below are three experiments I conducted with tools like Speech Recognition, Google Translate, Markov Chain Model, and ChatGPT. All of the technologies are flawed and biased in one way or another. But the experiments nevertheless offered insights into the nuanced relationship between mind and language, between technology and the quest for understanding and connection.
| Experiment #1 | Automated Thoughts |
| What I did: | • For a whole week, I wrote three pages of my random thoughts every morning (inspired by the Morning Pages exercise). • I fed all my written words into a simple Markov Chain language model, a mathematical model that can predict what word comes up next based on existing probabilities in a text. • I made a program to generate “new thoughts” based on the model built on my old thoughts. |
| Tools used: | • p5.js – A free, open-source JavaScript library for creative coding. I used it to create the interface of my program. • RiTa – A free, open-source library for writing in computational environments. I used its “RiMarkov” functionalities to build the Markov Chain model. |
| What I observed: | • The generated sentences mostly feel like random mix-and-matches of the text I fed into the model, and do not make much sense. • Occasionally, there were new sentences that made interesting connections between one thought and another, inspiring me to think in different ways. • A few of the generated sentences even revealed how I thought on a subconscious level. • Compared to using a Large Language Model like ChatGPT—whose data sources and training process are beyond my control, I felt safe experimenting with a mathematical model that I could explain. |
| Try this out if: | • Sometimes you feel trapped in your own thoughts. • You want to invite randomness into your thinking and writing. • You want to connect to your subconscious in unexpected ways. |
| To repeat this experiment: | 1. Do a lot of free writing, or type up your journal entries. Prepare at least 1,000 words (the more the better) of your unfiltered thoughts that can be copied and pasted. 2. Use this browser-based program I made: https://editor.p5js.org/re7l/full/u-OXKDm23 3. Copy and paste your writing, all of it, into the input field, and press “PROCESS MY WORDS.” 4. Now your writing has been made into a Markov Chain model that can generate more sentences based on probabilities in your original text. 5. Press “GENERATE A THOUGHT” to see a generated sentence—a new thought, based on your old thoughts. |
| Experiment #2 | Bad Translation |
| What I did: | • I invited friends, acquaintances, and strangers to have 15-minute conversations with me on Zoom. • I connected p5.speech to Google Translate and made a customized program that live-translates Chinese to English. • During the calls, I spoke Mandarin while my English-speaking visitor could only hear the machine translated English. |
| Tools used: | • p5.js – A free, open source JavaScript library for creative coding. I used it to create the interface of my program. • p5.speech – A p5 extension that provides Web Speech Synthesis and Recognition functionalities. I used it to turn my speech into text for Google Translate to process. • Google Translate – A multilingual neural machine translation service developed by Google to translate text from one language to another. • Zoom – A proprietary video conferencing software developed by Zoom Video Communications. I used it to conduct my 15-minute remote conversations. |
| What I observed: | • Inaccuracies and biases present in both Web Speech Recognition and Google Translate contributed to some mistranslations. _____• One notable instance is that in Chinese, there is no difference between the pronunciation of “he, she, or it,” they’re all pronounced as “ta.” In my experiment, Google Translate translated “ta” into “he” by default, no matter what the context was. • The mistranslations, frustrating at times, also had an upside. They brought the conversations to unexpected places, where I was able to learn more about my visitors from unplanned perspectives. In one instance, my words were mistranslated as “are you new song,” this grammatically incorrect sentence ended up sparking a philosophical conversation about the progression of music and the trajectory of our lives. • I felt safe and comfortable hiding behind the technological mask, talking freely in my first language without worrying about how I sound. |
| Try this out if: | • You speak more than one language. • You are a little too self-aware when you have to communicate in a language that you did not grow up speaking. • You wonder what it’d be like to hide behind technologies beyond your control when you converse with others. You have the option to attribute any regrettable statements to translation errors. • You are open to inviting some unforeseen twists and turns into your conversations. |
| To repeat this experiment: | 1. Find a few friends who are willing to do this experiment with you. 2. Go into a Zoom meeting with your friend. 3. Open Google Translate. (I used a custom translation program to enhance the aesthetics for my project documentation. Google Translate provides the same functionalities.) _____• Select the language you are going to speak, and the language you want your friend to hear. 4. Use a headset in the call, so the program will not hear itself speak. 5. Mute yourself, and share computer audio in Zoom. 6. On the Google Translate page, click on the microphone icon on the left to speak, and the speaker icon on the right to play the translation. _____• Remember to click on the stop button on the left after playing the translation of each utterance. |
| Experiment #3 | GPT Counselor |
| What I did: | • For about a week’s time, I asked ChatGPT to give me advice on how to respond to my parents’ text messages, and what to say to them during phone calls. • I asked it to be specific about what to say, and to be concise. • I communicated with my parents, mostly following ChatGPT’s suggestions, with small tweaks to make sure my responses reflected my personal voice. |
| Tools used: | • ChatGPT 3.5 – A Large Language Model developed by OpenAI that can generate human-like text based on existing patterns in its training data, as well as the user’s input. It outputs text token by token, assuming a confident tone of voice. However, it does not really understand what you say the same way a human does, and may produce false or misleading information. GPT 3.5, as of February 2024, is the model that powers the free version of ChatGPT. |
| What I observed: | • Introducing an AI intermediary made it less daunting to call my parents or respond to their texts. • ChatGPT maintained its composed, generic, and distant tone when I was overtaken by emotions like anger, shame, and anxiety. • For the most part, ChatGPT seemed to prioritize my parents’ expectations—what they wanted to hear, over what I wanted to express—my authenticity. • Saying what ChatGPT told me to say brought my parents comfort, or at least, reduced the likelihood of triggering negative reactions from them. • I did feel like I lost a little bit of myself in the process. Despite the increased communication facilitated by ChatGPT, I experienced a growing emotional distance from my parents. • Fortunately, upon concluding this experiment, I have now managed to strike a balance between considering my own need to express and their need for comfort and reassurance. _____• It is uncertain how long this balance will last. |
| Try this out if: | • You are in a situation where regular communication with someone you care about has become challenging due to various factors. • During conversations, your own urge to pour out your unfiltered feelings and thoughts overwhelms your ability to understand their needs. _____• What starts out as a normal conversation often turns into hurtful exchanges. _____• It takes a long time to recover from the emotional impact of such a conversation. • At times, you wish for the presence of a third party, acting as a mediator between you and the other person, a buffer between your strong emotions and the words you utter. |
| Stop doing this if: | • This experiment has lasted more than a week. • You find it increasingly challenging to engage in a conversation without ChatGPT on the side. • You start to rely on ChatGPT for interactions with individuals you previously had no particular issues communicating with. Generative AI presents a lot of unique challenges and risks despite its promises of efficiency and convenience. Over-relying on it for interpersonal communication is precarious and may lead to feelings of detachment. Exercise moderation when conducting this experiment. |
| To repeat this experiment: | 1. Identify one or two individuals with whom you could implement this approach. 2. During audio or text-based conversations with them, log into OpenAI’s ChatGPT platform. 3. To protect your privacy, click on your user name at the bottom left corner and go into “Settings.” In “Data controls,” turn off “Chat history & training.” 4. When you are uncertain about how to respond, consult ChatGPT: _____• Reiterate what the other individual says, state your standpoint, and ask for an appropriate response. _____• Request ChatGPT to provide examples of responses that sound casual and conversational. 5. Use screenshots to document your experiment. |
In Pursuit of Understanding and Connection: Three Experiments with Flawed Technologies is part of Knottings, a series developed in collaboration with Pioneer Works
