Mu Mirror
A Zen Koan in p5.js
Mu Mirror is a modern form of an ancient technology. For thousands of years, Zen has been using koans, or riddles, to encode mental hacks that help people see reality in a new way. usually, you have to meditate on a koan for a while – and eventually, something just clicks, and you see life differently.
But most people don’t have the time or patience to sit on a cushion long enough or to silently chant mu in their minds for months – so I wanted to try and make a piece of digital art that might give them a shortcut.
Background Intention
Where does your body end? Is your hair you? What about when you cut a strand of it off? Let’s say your body is your contiguous body, right now, not including clothes. What if we try to draw a 3D boundary around that body, with a large plastic bag – we wrap you up in it and seal it shut. You’d suffocate, die, and cease being “you” in a few minutes. So “you” must include some air, then. And water. And food. And a circulation system.
As we expand the plastic bag’s boundary outwards more and more to try and encapsulate all the things that make you you, we realize that any line we draw between you and not you is arbitrary. It’s an illusion to believe we are in any way separable from all of existence. A more accurate description might be that you are whatever the stuff is, right where you are, right now. You’re an address, in time. We’re calling that address you. Because it’s handy to label things as discreet, even though in reality, they’re not.
Eastern mystics call this non-duality. There is no “self” and “other” if we think about it hard enough. And modern particle physics agrees. The challenge is that it’s very difficult to shatter the illusion that we don’t have a discreet self. People meditate for hours, days, decades to exist in the state of mind where the true nature of reality is not just logically known, but embodied and felt. I’ve experienced it a handful of times. It’s a rush. And it’s incredibly fragile – the moment you see reality this way, it collapses. Because our bodies evolved to believe themselves discreet – it’s evolutionarily advantageous to try to sustain your corner of reality.
But what if there were an art installation that could trigger this insight in viewers? How much sweeter would the world be if we realized we weren’t in it, we were it?
Iterations
Iteration 1: 2D Prototype
Created in Processing. Tried to see what it might feel like to move through a grid of dots and have them be attracted to you.
Insight: this effect will only work if the mouse isn’t visible?
Iteration 2: Attractor Dots 3D
Created a 3D grid of dots that attract to your hands.
To test the effect, your right hand remains visible – and the illusion that you’re made of these dots completely fails. Interestingly, when your hand is invisible, the illusion that you’re made of dots works.
Decided to keep the dots misaligned after attraction, as if you’ve left a mark on the world.
Issues: This works on PC-VR, but is extremely laggy on Quest standalone headset because there’s so much computation (even just with 6x6x6 grid of cubes). Unclear if this is communicating the core message.
Iteration 3: Dense Dots
Created a UX interactor, a “shake button” to start. 3D grid of dots appears around you. Dots can be moved by picking them up.
Issues: Fun to play with, but too chaotic to truly sense the idea that reality is made up of cube-like "dots" like this.
Iteration 4: Building Blocks
Dots spaced further apart makes them feel more playful and inviting to move around a space. Something compelling here – everyone “gets it” and wants to pick up the dots.
Issues: the message, or takeaway of the experience, has nothing to do with my intent.
Iteration 5: Voxel Dots
Built a prototype in Premiere Pro envisioning a world where these "dots" only appear when an object is moved, or interacted with. Could pixellation be both beautiful and help the user un-label the forms around them, by revealing that it's all made up of the same "stuff"?
Iteration 6: Blur Mirror
Restart the project in p5.js to try this pixellation idea in 2D.
Read in data from webcam, draw a circle with size based off its brightness and color.
Iteration 7: Mu Mirror
Built in p5. Receive webcam data. Take two photos. Compare the pixels between them based on color. Find the closest pixel of the most similar color. Move the current pixel over to the target pixel’s location. Lerp both location and color over time. Set dot dimensions parametrically, based on user input.