Alien Communicator

Intro

For this project we were tasked with the assignment of creating a simple input and output device:

Come up with a simple application using digital and/or analog input and output to a microcontroller. Make a device that allows a person to control light, sound, or movement using the components you’ve learned about (e.g. LEDs, speakers, servomotors)

I was paired with a couple classmates (Ahmed and John) and we talked a bit about different things we were interested in doing.

Ahmed was really inspired by close encounters of the third kind and the communication between two parties with sound and light:

Close incounters final scene… with a weird crossover at the end…

At first I thought Ahmed was thinking about just creating a musical instrument that used light and sound. In responding to that John came up with some basic ideas about a Hand Theremin. However soon it became clear that he was envisioning something a little more complex: an actual encounter with a machine that felt organic. John and I found this idea really intriguing. How do we create a system that feels like an organic interaction? How do you balance input and output? With this new understanding John created a new page for us to collaborate on and bounce ideas back and forth:Group Project

Thoughts

The key thing that we were detracting from the nearby experience thing is this idea of Dialog. How would you have a real conversation with a machine? In pondering it, John and I were reminded of a key idea of information theory- the information content of a system. Envision a page. In the event that it is completely loaded up with arbitrary letters it is totally indiscernible. Same as well in the event that it is totally loaded up with one letter. All things considered, all things considered the example is truly straightforward, yet it is easy to the point that it has almost no significance and is unbelievably exhausting. The secret to language is a fine harmony among shock and redundancy. A discourse is likewise a fascinating focal thought. In an exchange there is a call and a reaction. One gathering expresses, the different tunes in and afterward reacts while the primary party tunes in. In a discourse there is a sure measure of redundancy that is important so as to cause it to appear as though you are being heard. That is clear about the nearby experiences model: the people produce a yield, the outsiders emulate it and afterward develop it. I can’t help thinking about how I can start to implement that.

Summary

With all that sort of blabbed about I think now would be a good time to land on a goal for the project:GOAL: to create an organic dialogue between a human and a machine Let me break that goal down into it’s components:– A dialogue consists of a back and forth, a call and response.– Organic means alive i.e. not random and not mechanical.That is a pretty tall order for something due in a week! But I think it’s fairly manageable if we stay on target. Below are some strategies I want to try to try and make that happen:

Strategies

In my mind the two things that should work out are how to have a dialogue, and how to make that dialogue organic. Here’s an brief rundown of thoughts on the best way to accomplish each of those. Strangely the methodologies for planning a decent dialogue seems to be more hardware related and the techniques for causing the interaction organic to appear to be more programming related.

Dialogue Strategies

How to make a device that is like a conversation? I think it needs an input and a output clearly. Remaining straightforward, I’ll attempt to make the input buttons. Starting essential with state 3 buttons. At that point you need a type of feedback which tells you the device registered your input. Those could be lights or tones. Other expected inputs for later are the ultrasonic ranger which would be fun or copper tape or potentiometers or photoresistor. Were avoiding FSR since they give me random numbers.

Here’s a quick list of those necessary components:– Simple input– Feedback– Simple output – Similar to or the same as feedback– Turn indicator

Organic Strategies

Then once we actually make the physical device, there arise a whole bunch of questions about how to make an organic or semi intelligent interaction. It’s harder to break that down at the outset to see what is necessary, but I’ll give it a shot. The secret probably lies somewhere in the response. There probably needs to be a fine balance between surprise and predictability. An easy way to do this (I think) is to make a machine that listens to your input and records it and then spits it back out with slight variation. If we have 3 buttons you could capture which button was pressed, how long it was pressed for, and the length of time between each press. If you put all that into an array you could then either repeat it back, scramble it,or ideally do something in between. A basic example is if you did something like:At this point once we make the physical device, there goes emergeing of mysterious questions regarding how to make a organic or semi intelligent interaction.It’s harder to separate that at the beginning to perceive what is fundamental, yet I’ll try it out. The mystery likely lies some place in the reaction. There most likely should be a fine harmony among surprise and predictability. An simple approach to do this is to make a machine that listens in to your input and records it and then spits it back out with slight variety. On the off chance that we have 3 buttons we could capture which buttons was pressed, how long it was pressed for, and the length of time between each press. If you put all that into an array you could then either repeat it back, scramble it,or ideally accomplish something in between. An essential example is on the off chance that we accomplished something like: [1,2,3,1,2,3,3,3] → [1,2,3,3,3,1,2,3]In reality, since I consider it, a Markov Chain could be a decent numerical example to follow here. Other potential examples could be Conway’s game of life or Perlin Noise. However, that is kind of a digression and I’m losing trace of what’s most important. I think now it is smart to begin prototyping something and see what the connection ‘feels’ like.Final OutcomeCode#include “pitches.h”
///Music
int noties[ 4 ][ 4 ] = {{B3, C4, CS4},{D4, DS4, E4},{F4, FS4,G4},{GS4, A4, AS4}};

//int pbnoties[ 3 ][ 3 ] = {{NOTE_C4, NOTE_C5,NOTE_C6},{NOTE_E4, NOTE_E5,NOTE_E6},{NOTE_G4, NOTE_G5,NOTE_G6}};

const int buttonPin1 = 2;
const int buttonPin2 = 3;
const int buttonPin3 = 4 ;
const int buttonPin4 = 5 ;

const int ledPin = 10;
int leds[3]={13,12,11};
bool buttonState1 = false;
bool buttonState2 = false;
bool buttonState3 = false;
bool buttonLocked = false;
bool buttonJustPressed = false;

bool awake = false;
bool listening = true;

//arrays
const int maximum = 40;
int buttonsPressed[maximum];
int pausesBeforePlaying[maximum];
int durationOfPresses[maximum];
int noteCount = 0;
int playbackIndex = 0;

//data
int pressStarted = 0;
int prevPressEndedAt = 0;
int pressLength = 0;
int gapLength = 0;
int currentPressedButton = 0;

//playback

int playState = 0;
int currentPause = 0;

void setup() {
// initialize the LED pin as an output:
pinMode(ledPin, OUTPUT);
//buttons
pinMode(buttonPin1, INPUT);
pinMode(buttonPin2, INPUT);
pinMode(buttonPin3, INPUT);

pinMode(14, OUTPUT);
pinMode(15, OUTPUT);
pinMode(16, OUTPUT);

Serial.begin(9600);

//speaker
pinMode(8,OUTPUT);
}

void loop() {
//modeswitch

if(awake){
if(millis()-prevPressEndedAt>=2000&&listening){
listening = false;
playbackIndex = -1; //noteCount;;
playState = 2;
digitalWrite(13, LOW);
digitalWrite(12, LOW);
digitalWrite(11, LOW);
}
}

/////record and playback
if(listening){
digitalWrite(13, HIGH);
digitalWrite(12, HIGH);
digitalWrite(11, HIGH);
///recording
buttonState1 = digitalRead(buttonPin1);
buttonState2 = digitalRead(buttonPin2);
buttonState3 = digitalRead(buttonPin3);
digitalWrite(ledPin, LOW);

if(buttonState1||buttonState2||buttonState3){
awake = true;

if (digitalRead(buttonPin1)==HIGH){
buttonPressed(1);
delay(10);
}
if (digitalRead(buttonPin2)==HIGH){
buttonPressed(2);
delay(10);
}
if (digitalRead(buttonPin3)==HIGH){
buttonPressed(3);
delay(10);
}
}else{
if(buttonJustPressed){
buttonReleased();
}
}
}else{
///playback

playback();
if(digitalRead(buttonPin1)||digitalRead(buttonPin2)||digitalRead(buttonPin3)){
noTone(8);
listening = true;
noteCount = 0;
prevPressEndedAt = millis();
//
memset(buttonsPressed, 0, sizeof(buttonsPressed));

memset(pausesBeforePlaying, 0, sizeof(pausesBeforePlaying));

memset(durationOfPresses, 0, sizeof(durationOfPresses));

};
}
}

void buttonPressed(int button){
if(!buttonLocked){
currentPressedButton = button;
pressStarted = millis();
tone(8,noties[button-1][0]);
// Serial.println(noties[button][0]);
buttonLocked = true;
buttonJustPressed = true;
gapLength = pressStarted – prevPressEndedAt;
Serial.println(pressStarted);

}

void buttonReleased(){
noTone(8);
buttonLocked = false;
pressLength = millis()-pressStarted;

if(pressLength&&gapLength){
Serial.print(noteCount);
Serial.println(” —————————–“);
Serial.print(“button-“);
Serial.println(currentPressedButton);
Serial.print(“length-“);
Serial.println(pressLength);
Serial.print(“gap- “);
Serial.println(gapLength);

//writing to arrays
buttonsPressed[noteCount]= currentPressedButton;
pausesBeforePlaying[noteCount]= gapLength;
durationOfPresses[noteCount]=pressLength;
noteCount++;
if(noteCount==maximum){
noteCount = 0;

}

buttonJustPressed = false;
prevPressEndedAt = millis();
}

void playback(){

digitalWrite(ledPin, HIGH);
switch (playState) {
case 0://init
currentPause = millis();
playState = 1;
break;

case 1://waiting
if(millis()-currentPause>=pausesBeforePlaying[playbackIndex]){
playState = 2;
currentPause = millis();
}
break;

case 2: //emit
if(buttonsPressed[playbackIndex]){
Serial.print(playbackIndex);
Serial.println(” ||||||||||||||||||||||”);
Serial.print(“button-“);
Serial.println(buttonsPressed[playbackIndex]);
// Serial.print(noties[playbackIndex-1][1]);
Serial.print(“length-“);
Serial.println(durationOfPresses[playbackIndex]);
Serial.print(“gap- “);
Serial.println(pausesBeforePlaying[playbackIndex]);

tone(8, noties[(buttonsPressed[playbackIndex])-1][random(3)]); /// 0 this is getting weird
digitalWrite(leds[(buttonsPressed[playbackIndex])-1], HIGH);

playState = 3;
}else{playState = 4;}

break;

case 3: //hold
if(millis()-currentPause>= durationOfPresses[playbackIndex]){
digitalWrite(leds[(buttonsPressed[playbackIndex])-1], LOW);
playState= 4;
}

break;

case 4: //release
noTone(8);
// digitalWrite(leds[(buttonsPressed[playbackIndex])-1], LOW);
playbackIndex++;
if(playbackIndex==maximum){
delay(700);
awake = false;
listening = true;
playbackIndex = 0;
} ///it would be nice to make this cut the playback off
playState = 0;
break;

}

Date:

May 24, 2021

Minjun Kim