Android Wearable

You can create some simple wearable projects with just your Android phone, an arm band and a set of headphones with a microphone. A wearable solution like this could be used by anyone that has their hands busy, for example recipes for chefs, instruction manuals for mechanics or  directions for a cyclist.

In this article we will look at creating an Android application that does speech recognition of a keyword. The keyword will be searched in a simple CSV file and text-to-speech will be used for the result.  We will create the Android app using MIT’s AppInventor package. AppInventor is a free Web based Android app development tool, that allows you to create applications in a graphical environment.

AppInventor

To get started with AppInventor, you will need a Google user account and a PC, Mac or Linux computer. AppInventor is a Web based Android app creation tool  (http://appinventor.mit.edu), that uses a graphical programming environment.

AppInventor has two main screens. The Designer screen is used for the layout of the Android app and the Blocks screen is used to build the logic.

app_2buttons

On the Designer screen, an app is created by dragging a component from the Palette window onto the Viewer window.

For the visuals on this application we will a Button, a Label and a ListView from the User Interface Palette window. The button will be used to initiate the speech recognition. The label will show the result from the speech recognition, and the listview component will show the CSV file data.

Also some non-visual components will as be used. In the Media section of the Palette window, select the SpeechRecognizer and TextToSpeech components and drag them into the Viewer window. As well select add the File component from the Storage heading .

app_layoutcomponents

The Components window is used to rename or delete components.  When a component is selected the Properties window is used to change its editable features. In this example we renamed the button to BT_Speak, and we changed the backgroundColor, FontSize, Width and Text.

app_prop

The Logic

Once the layout design is complete, logic can be added by clicking on the Designer  button (on the top menu bar).

Logic is built by selecting an object in the Blocks window, and then click on the specific block that you would like to use.

add_block

The entire program only requires 1 variable and 4 when blocks.

file_logic

The first step is to load the text file when Screen1.Initialize block is called. The when File1.GotText block loads the text file data into the global variable, (THELIST), and it populates the ListView component.

The when BT_Speak.Click block is activated on a button push and it starts the speech recognition block.

The final block, when SpeechRecognizer1.AfterGettingText, shows the result of the speech in a label and it checks if the result is in the global variable. If the result is found a text-to-speech message is generated with the full line of text.

The Data File

For our test file we placed the key words at the starting of each line.

"Hope Bay has a sandy beach with good hiking and..."
"The Glen is a horseshoe shaped valley with ..."
"Isaac Lake is a bird sanctuary with ..." 
"Oliphant is a great for kite surfing..."
...

Our data file used some local landmarks, but there are lots of other choices like: friends addresses, recipes ingredients or favorite restaurants.

The file was saved as places.txt in the phones download directory, this should match up with the File1.ReadFrom block definition (/Download/places.txt).

Compiling and Running the App

After the screen layout and logic is complete, the menu item Build will compile the app. The app can be made available as an APK downloadable file or as a QR code link.

app_build

Once the app is install in the phone, pushing the “Talk” button will open the Google speech recognition dialog.  If you’ve spoken a valid keyword then you should hear the line from the data file. The data file can be updated without any changes to the app.

app_arm

Final Thoughts

This example used a simple text file, but it could be enhanced to support multi-field CSV files, Cloud Services, HTTP requests or Google Maps.

 

 

 

 

 

 

 

 

 

Android Language Translator

Using MIT’s free AppInventor package you can create your own custom language translation app.

AppInventor

To get started with AppInventor, you will need a Google user account and a PC, Mac or Linux computer. Next log into the Web site, http://appinventor.mit.edu. AppInventor is Web based Android app creation tool, that uses a graphical programming structure so no coding is required.

AppInventor has 2 main screens. The Design screen is used to layout the presentation or look-and-feel of the Android app and the Blocks screen is used to build the app logic.

appinv_menu1

On the Design screen, an app is created by dragging a component from the Palette window onto the Viewer window.

For visuals on this application we are using 2 buttons, a text box and a label.

talk_setup1

This application also requires some non-visual components. In the Media section of the Palette window, the SpeechRecognizer, TextToSpeech and YandexTranslate components are added by dragging and dropping them on to the Viewer window.

The Components window is used to rename or delete components. When a component is selected the Properties window is used to change visual features such as its sizing, color,  font size etc.

app_inv_prop

After the layout design is complete, click on the Blocks menu button to start building logic.

Translation Logic

The logic is built by selecting an object in the Blocks window, and then select the specific block that you would like to use.

talk_logic_drop

For the translation app we use a  when Screen1.Initialize block to define the language to translate to. For this example “ESP” and “es” are used for Spanish. Other language such as French (“FRA”, “fr”), Germany (“DEU”,”de”), or Italian (“ITA”,”it”)  could also be used.

On our app we will use the top button to start the speech recognition. This is done by inserting a when BT_Speak.click block and put a call SpeechRecognizer1.GetText block inside.

A when SpeechRecognizer1.AfterGettingText block will return our spoken text. This result is shown in a textbox and passed to the YendexTranslation1.RequestTranslation block.

Next a when YandexTranslate1.GotTranslation block is used to show and speak the translated text.

A second button click is included so that you can repeat the translation.

The full logic is shown below.

talk_logic

Compiling and Transferring the App

After the screen layout and logic is complete, the menu item Build will compile the app. The app can be made available as an APK downloadable file or a QR link can be used.

app_build

Final Thoughts

This translator app is pretty basic but it has a lot of potential for enhancements. Some useful features could include adding multiple languages and 2 way translations.

Control with Texts (Node-Red on Android)

In places where Internet connections are not possible or too expensive,  SMS text messaging can be a simple approach for monitoring and controlling your remote systems. Many of the mobile providers offer IoT packages for low data throughput, where you’d be looking at spending $1 to $5  per month for 1-5 MB of data.  From the hardware standpoint there are many options such digital modem modules that come in either a Raspberry Pi top or a USB form factor.

If you are looking at doing some prototyping, using an Arduino phone and Node-Red is a great way to get jump started.

Node-Red on Android

Node-Red is a graphical programming system that is used for Internet of Things (IoT) projects. Node-Red is installed in the base Raspberry Pi images, but it can also be loaded on Linux, Windows, MacOS and Android systems.

To load Node-Red on Arduino, you will first need to load Termux,  an Android Terminal Emulator app, that is available at Google Play. After Termux is loaded, enter the following commands to install  and run Node-Red:

apt update
apt upgrade
apt install coreutils nano nodejs
npm i -g --unsafe-perm node-red
node-red

Node-Red will start a Web graphical interface that is accessed by:  http://the_ip_address:1880.

nodered_pc

Extra features can be added and removed from Node-Red by selecting the “Manage Palette” menu option. For this project I needed terminux-api for the texting support, and bigtimer to scanning the text message buffer.

manage_palette

A Test Setup

For a basic setup I used:

  • 1 Android phone with Termux and Node-Red
  • 1 Raspberry Pi with Node-Red
  • 1 Powerswitch Tail II connected to a light
  • 1 Android phone for texting in

sms_pi_setup

Scanning for Text Messages

To create a simple text message application on the Android Note-Red system, the following components are used:

  • A Big Timer node is used to define how often the SMS inbox is scanned. Without doing any configuration, the second output will offer a 1/minute cycle time.
  • An SMS Inbox node will read in a defined number of text messages. To get the last message select the buffer limit to be 1.
  • A function node is used with some JavaScript to check for valid text messages
  • A TCP out node will send a message to another Node-Red system.

A basic logic setup will be as follows:

sms_in_logic

The function node needs some logic to interpret the latest text message and send out the required logic to the Raspberry Pi GPIO pins. The example logic uses a text message of  “LIGHTS ON” or “LIGHTS OFF” to control the output on GPIO pin 12.


// look for new action texts
smslast = context.get('smslast')|| 0;

// Do an action if there is a new SMS message
if ( msg.payload[0].received != smslast) {
    context.set('smslast', msg.payload[0].received )
    smsbody = msg.payload[0].body;
    if (smsbody.toUpperCase().trim() == "LIGHTS ON") {
        msg.payload = 1;  // this is sent to the Pi via TCP
        return msg;
    }
    if (smsbody.toUpperCase().trim() == "LIGHTS OFF") {
        msg.payload = 0;  // this is sent to the Pi via TCP
        return msg;
    }
}

The first time that the code runs you will be prompted on the phone to “Allow Termux:API to send and view SMS messages?”. After you acknowledge this message your Node-Red logic will run cleanly.

Reading TCP Inputs

Android phones can not be directly connected I/O devices, so TCP connections can used to pass commands to Raspberry Pi’s which have the I/O support. Node-Red is pre-installed on most Raspberry Pi images so no added installation is required.

On the Raspberry Pi Node-Red side only a couple of nodes are required:

  • A TCP in node is used to read the incoming communications. For the configuration of this node it is important to have the port (8888 in this case) match the TCP out node’s port. Also the output should be set as: single, and string.
  • An RPI GPIO out node is used set Raspberry Pi General Purpose (GPIO) pins.

sms_tcpin

Texting Out from Android

If your text scheme is simple, for example one light that is turned on and off, then you could manually just output an “ON” or “OFF” message. However if you are controlling multiple lights, heaters and other devices then manually typing the text messages gets awkward.

To manage multiple commands, I wrote a simple Android app in MIT’s App Inventor. App Inventor is a free online Android development environment, that only requires a Google login for access. In about 10 minutes I was able to get a prototype going with multiple inputs. The first step is to drag and drop some buttons from the User Interface Palette onto the viewer pane. Then drag and drop a Texting component from the Social Palette onto the view. Note, the Texting component will appear as a non-visible item.

text2red_gui

After you have a basic layout, select the Blocks button on the menu bar, and this will open a logic page. Logic is created by clicking on the objects in the Block panel, and then dragging and dropping the block onto the Viewer panel.

To have a button sent a text message the following pieces are required:

  • “when Button.Click”  will be called when the users touches the button object
  • “set Texting1.PhoneNumber”  defines the remote phone number
  • “set Texting1.Message” defines the text message
  • “call Texting.SendMessage”  will send the message

text2red_blocks

To build the application use the Build menu item on the menu bar.

app2qr

For my test project I configured four devices and eight commands.text2sms_app

Final Comments

I found that using Node-Red on Android to be a lot faster than I expected, however I noticed that some of the added features (like Bluetooth support) only worked on the Raspberry Pi/Linux version of the Node-Red.

For a final solution I would definitely move to dedicated SMS hardware, but I found it nice to be able to do proof of concept testing with just some basic Android phones. Also don’t forget to setup Node-Red for automatic startup on a power up.

Android Step Counter/Pedometer

You can make a super simple Android step counter or pedometer application using App Inventor. App Inventor is a free Web based Android programming environment.

In the Designer screen we used the User Interface palette to layout 4 labels and 2 buttons. To help with the layout a TableArrangement component can be used.

Most of today’s smart phones have a built pedometer. In the Sensor palette the non-visible pedometer component is the software piece that connects to the phone’s pedometer hardware.

AI2_Step_Setup

The next step is to use the Blocks page to add the required logic. The smart phones pedometer hardware is running in the background, so we added 2 buttons. One button to reset the pedometer count and the second button to pause and resume the pedometer counter.

The pedometer component has a WalkStep event, this event is used to update the distance and steps labels. Below is the full logic for our application.

AI2_Step_blocks

The final application should look something like the screenshot below. With our basic step counter logic we thought that some future features could include:

  • Daily summaries
  • GPS and Map data
  • Hiking information

Set_Screenshot

Tow Truck

The goal for our tow truck was to have a 4-axis crane and a movable vehicle that we could remotely control with an Android smart phone.

tow_truck2

The parts we used for this project were:

Hardware Setup

The tow truck project used a camera mount for up/down/left/right crane motion and an Arduino car chassis for mobility. The controls were done using Bluetooth.

Meccano was used to build a box for the main structure. Wire was used to secure everything together. We laid a folded piece of paper under the Arduino Mega to ensure that none of the Arduino solder connections shorted on the metal Meccano base.

tow_truck_inside

The motor and servo shield that we used did not expose any of the extra Arduino pins, so we needed to use the Mega board. We then wired the Bluetooth module to the exposed pins on the end of the Mega.

tow_truck_circuit

Arduino Code

The Arduino code will vary a little based on the motor/servo shield that is used. Our shield was an older version 1 (V1) board that used direct pin connections (no I2C or SDA/SCL connections).

Also because Tx/Rx (Tx0/Rx)) were not available once our motor/servo shield was installed we used Tx1/Rx1 and so our Bluetooth connection was on Serial1 and not Serial.

For the Bluetooth communications we used the following command letters:

  • R = drive right
  • L = drive left
  • f = drive forwards
  • b = drive backwards
  • s = stop driving
  • r = move crane right
  • l = move crane left
  • u= move crane up
  • d = move crane down

Our Arduino code is below:

#include <Servo.h> 

Servo servo1;
Servo servo2;

char thecmd;
int xpos = 90;
int ypos = 90;

AF_DCMotor motor1(1); 
AF_DCMotor motor2(2);

void setup() {
  pinMode( 19, INPUT_PULLUP );
  Serial1.begin(9600);
  Serial1.println("Crane Controls");
  Serial1.println("r = right, l = left, u= up, d = down");
  Serial1.println("Driving Controls");
  Serial1.println("R = right, L = left, f = forwards, b = backwards, s = stop");
  servo1.attach(9); // attaches the servo on pin 9 to the servo object 
  servo2.attach(10); // attaches the servo on pin 9 to the servo object 
  
  servo1.write(xpos);
  servo2.write(ypos);

  motor1.setSpeed(255);
  motor2.setSpeed(255);

}
void loop() {
  
  if (Serial1.available() > 0) {
        // read the incoming byte: 
       thecmd = Serial1.read();
       Serial1.println(thecmd);
       if (thecmd =='l') { move_crane(servo1, 5); }
       if (thecmd =='r') { move_crane(servo1, -5); }
       if (thecmd =='d') { move_crane(servo2, 5); }
       if (thecmd =='u') { move_crane(servo2, -5); }
       if (thecmd =='f') { 
            motor1.run(FORWARD); 
            motor2.run(FORWARD); 
       }
        if (thecmd =='b') { 
            motor1.run(BACKWARD); 
            motor2.run(BACKWARD); 
       }
       if (thecmd =='L') { 
           motor1.run(BACKWARD); 
           motor2.run(FORWARD); 
       }
       if (thecmd =='R') { 
            motor1.run(FORWARD); 
            motor2.run(BACKWARD); 
       }
       if (thecmd =='s') { 
           motor1.run(RELEASE); 
           motor2.run(RELEASE); 
       }
  }
}

void move_crane(Servo theservo, int direction) {
  int minpos = 50;
  int maxpos = 220;
  if (direction < 0) {
    if (ypos > minpos) {
      ypos = ypos + direction;
      theservo.write(ypos);
    }
  }
  else {
    if (ypos < maxpos) {
      ypos = ypos + direction;
      theservo.write(ypos); 
    }   
  }
}

Android Program

To communication to an Android smart phone we used MIT’s App inventor. This is a free Web based Android development tool.

There are many ways to layout a control screen, for us we used a 10×3 table and then populated it with buttons. Our layout is shown below:

ai_layout

The button logic will pass the required letter command to the Bluetooth component:

ai_logic

Our final running App looked like:

Screenshot_tow_truck

Android Controlled Gondola

fullsetupUsing the Lego Mindstorms NXT robotics kit we built a remote control gondola. The MIT App Inventor package was used to create an Android phone app that controlled the gondola.

For this project we used:

• 1 Lego Mindstorms NXT brick with 1 motor and cable
• 1 Android phone
• lots of Lego bricks
• some string
• 2 adjustable mens or womens belts

The Lego Mindstorms motor should be connected in either the B or C connection on the brick.

You will need to make two gondola platforms. One with the Lego Mindstorms brick and motor, and a second platform with a wheel that can spin freely. You can have a lot of fun creating different gondola platforms. We found that some of the important points in the construction to be:

the platforms needed to be firmly secured. We put our platforms on chairs and we used belts to stop them from moving (Figure 2). A carpeted floor will help stop the chairs from slipping.

Small gondola. It’s tempting to want to make a big gondola, but we found that big gondolas had more string slippage and you had to have the gondola stations closer together.

Use wheels with grooves or rims. If you use wheels with grooves or rims the gondola string is less likely to slip out.station2

gondola

The Lego NXT brick needs to have Bluetooth turned on. Check the top left corner of the Brick’s screen to see if the Bluetooth symbol is showing. If it is not showing go into the Bluetooth options and turn visibility on.

For the Bluetooth pairing the Lego NXT needs to be turned on and your Android phone will need to have Bluetooth enabled. From your Android Bluetooth setting select “Search for Devices”. The Lego NXT brick will appear as NXT on your Android phone. On the phone select “pair with device”, and the phone pairing dialog will come up. The pairing request dialog may look different for different versions of Android, but the key is to enter the correct pairing PIN.

When the pairing is started the NXT will beep and prompt you for the pairing passkey. For the pin code (passkey) use the default of: 1234, and finish with the check mark. The pin code of 1234 is used on both the NXT and on the Android phone.

AppInventor has a set of components that will talk to the Lego Mindstorm NXT (or EV3) bricks. Below are pictures showing the our screen layouts and the key code elements.

blocks

app_layout