Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

NAO RobotsGetting Started Guide

Arizona State universityCyber Physical Systems LabProfessor Georgios Fainekos

Author: Ramtin Raji KermaniSummer 2013

1

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Table of Contents:

PrefaceIntroduction*How NAO works

How to Program NaoSimulationActual NAONAO Robots

NAO hardwareNAO Internals and Operating System

NAO operating systemNAOqi SoftwareHow NAOqi Works

Programming NAO1.1.4 Choregraphe® and Monitor®1.1.5 qiBuild system1.1.6 NAOqi SDK

1.1.6.1 C++ APIAs one might have noticed, an object of class ALTextToSpecchProxy is createdand a method on that object is called with appropriate arguments and that is allyou need to code to make NAO “say” something. However you need to go throughthe project creation and build using qiBuild build system to be able to compile thiscode.1.1.6.2 Python API1.1.6.3 Matlab API

Implementing simple behaviors using NAOqi Matlab® API1.1 NAO Matlab API installation

1.1.1 NAO Matlab API troubleshooting1.1 QiBuild and SDK Installation

1.1.1 qiBuild Installation1.1.2 Creating a initializing a Work tree1.1.3 Creating a toolchain1.1.4 Status of available toolchains1.1.5 Creating, configuring and building a new project1.1.6 Using Microsoft Visual Studio to build the projects

1.2 Connecting to OpenNAO through SSHConnecting to the NAOChoregraphe and MonitorProgramming NAO

Remote Controlled NAO

2

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Python SDKC++ SDKMatlab SDK

Stand alone Code on NAOInstalling Webots for Linux

3

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

1. PrefaceThis is a getting Started Guide for users who are new to the NAO world. NAO documentation on Aldebaran’s website is a very complete one, covering all aspects of NAO robots, SDK, Software, Frameworks etc. However the documentation is too much and too complicated for someone who is just getting started.This tutorial is written based on the personal experience on the Cyber Physical Systems lab at Arizona State University under supervision of my advisor Professor Georgios Fainekos.

The main source of documentation for you should be the following website:

NAO Documentation

This is a great, well­written and comprehensive guide for NAO. However navigation through this documentation is a little bit difficult for the beginners. This tutorial tries to organize the learning steps and the way you are going to use the NAO documentation.

2. Introduction

NAO is an educational humanoid robot used as the main agent in Robocup Standard League platform.Software in and out of the robotNaoqi FrameworkNaoqi Modules APIs

Remote Connection to NAO­ SSHDebugging NAO programs

4

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

5

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

6

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

3. How NAO works

Read this: Software in and out of the robot

7

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

4. How to Program NaoThere are various ways you can exploit the capabilities of your NAO and program it for specific tasks.Tip!:First of all note that you can either use a simulated environment or an actual real­world NAO.Tip!:Second, if you decide to work with actual NAO, again you have two choices: Remote execution of your code from a desktop computer OR building and downloading your code on the NAO and execute a local stand­alone program.

Either simulation or using actual NAO, you can use any of the following options to program the robot:1­ Using the high­level programming environment of Choregraphe2­ Using NAOqi SDK to write your own applications in many languages such as C++, python, Matlab, Java, etc.In both cases, you need to connect to NAO using a TCP/IP connection. So you need an IP and a port. Default TCP port for NAO is 9559.

The following summarises the above programming paths and is a chart that illustrates various ways of testing your algorithms for NAO:

Simulation (Webots, RoboCup Soccer Server 3D­ RCSS3D, etc.) High level software

Choregraphe Programming using SDK

Matlab Python C++ Java

Actual NAO Remote execution

High level software Choregraphe

Programming using SDK Matlab Python C++ Java

Local execution (On the NAO) Python C++

8

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

a. SimulationBefore executing your program on the robot, you can test the behavior on a simulator. There are many choices out there but the following are the most widely used simulators for NAO robots:­ Webots­ Webots for NAO (Comes with NAO on the DVD)­ Simple embedded Simulator of Choregraphe­ RoboCup Soccer Server 3D: RCSSS3D

b. Actual NAO

9

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

NAO RobotsNAO is an autonomous programmable humanoid robot designed and manufactured by Aldebaran Robotics. It offers a rapid development environment for humanoid robotics. Since 2007 NAO robots are being used in Robocup Standard Platform League (SPL) competitions succeeding AIBO four‑legged robots from Sony. There are various models of NAO developed, each suiting a specific application area. The one used in academia and SPL competitions, NAO H25, has 25 degrees of freedom. In the following sections features of this robot based on which this thesis is written, are portrayed.

NAO hardware

Table 1 lists brief technical specification for NAO H25 version. For a full detailed specification please refer to NAO Specification [33]:

Table 1 ) NAO H25 technical specification

CPU Intel Atom Processor 1.6 GHz

Sensors IR Sensor(2x), Sonar Sensor(2x), Gyrometer, Accelerometer,

Microphones 4x

Force Sensitive Resistors Head, feet

HD Cameras 2x

Loud Speakers 2x

Operating System: Embedded GNU/Linux: OpenNAO, based on Gentoo distribution

Architecture: X86

Programming SDK Embedded: C++ / Python

Remote: C++ / Python / .NET / Java / Matlab / Urbi

Degrees of freedom 25 (Head(x2), Arm (in each)(x5), Pelvis (x1), Leg (in each) (x5), Hand

(in each)(x1)

10

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

NAO Internals and Operating SystemIn this section we briefly explore the software internals of the NAO robots. Understanding

how a program is executed within NAO, helps users to better understand and use the APIs as well as programming and configuring the robot.An embedded software called NAOqi is always running on the robot on top of the operating system and is responsible for managing robot modules, communications and scheduling. NAOqi Software Development Kit (SDK) is a set of Application Programming Interfaces (API) provided in several languages such as C++, Python, Matlab®, Java and Urbi that help programmers develop application for NAO. The SDK also is supplied alongside with a build system called qiBuild which is created based on CMake and is responsible for project management, build and debugging the applications written using the C++ SDK.

NAO operating systemNAO has a full­fledged computer embedded in its head utilizing an Atom processor and

running an embedded operating system called OpenNAO. OpenNAO is developed explicitly for NAO robots and is based on the Gentoo Linux distribution. The operating system is equipped with libraries and programs necessary to run NAO programs.There are two default Linux users defined with usernames “nao” and “root”. The later has root privileges. It is possible to connect to the robot using an SSH client getting access to the robot’s operating system command line as well as the file system.

NAOqi SoftwareNAOqi is the main program residing on the NAO that is responsible for managing

resources in a higher level and provides an abstraction for the hardware and operating system through software components called modules and also the APIs. NAOqi includes a reliable, fast, cross­platform robotic framework, NAOqi Framework, for developers to enhance the capabilities and functionalities of NAO. NAOqi supports all requirements of a robotic application such as parallelism, resource management, event handling and synchronization.

How NAOqi WorksNAOqi system provides system processes called “Modules” that are considered as main

interactive components of the robot’s functionalities. Each module is responsible for taking care of a specific capability of the robot such as motion, text or speech processing, vision, etc. and provides the interface for high level layer to interact with the hardware and implement those functionalities.

Using the provided API’s, programmer creates appropriate “Proxy” objects that are used to call the functions and services provided by the “Modules”.Different levels of abstraction presented by NAOqi SDK will be a useful feature for code generation. However, in many cases, for model based design we need to ignore many underlying details and that is why we decided to create higher level classes in C++ which basically will function as wrappers for lower level functionalities and services. ROS users can think of NAOqi as ROS core and modules as ROS packages/Nodes.

11

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Programming NAOThere are various ways to exploit the capabilities of NAO robots and program it for a

specific task. First of all note that one can either use a simulated environment or an actual real­world NAO to test the developed algorithms. In this project, both simulation environment and real robots are considered as the target for code generation. Again we have two choices for controlling NAO:∙ Remote execution of the code from a desktop computer∙ Building and downloading the code on the NAO and execute a local standalone programIn both cases we need to use the NAO API which is provided as NAOqi Software Development Kit and write the programs in either of the following languages: C++, Python, Matlab, Java, .Net. However NAO only supports local execution of C++ and Python code, the rest need to be executed on a host desktop/laptop computer and send commands to NAOqi system either via Wi­Fi or Ethernet.The following summarizes the above programming paths for programming NAO:

∙ Simulation (Webots, RoboCup Soccer Server 3D­ RCSS3D, etc.)o High level software (Choregraphe)o Programming using SDK (Matlab, Python, C++, Java, .NET)

∙ Actual NAOo Remote execution

§ High level software (Choregraphe)§ Programming using SDK (Matlab, Python, C++, Java, .NET)

o Local execution (On the NAO; Python SDK, C++ SDK)

Choregraphe® and Monitor®

Choregraphe® is a high level block based programming environment to develop applications for NAO. It is used to create and edit movement and behaviors using its intuitive graphical environment. The blocks are programmed to implement behaviors from joint movements to high level posture transitions, walking algorithms and even object tracking all in one simple to use block. As in LabVIEW® and Simulink® environments, Choregraphe® takes advantage of execution flow concepts and the block execution order is dependent to the order in which they are connected to each other. Figure below shows a screenshot of Choregraphe software connected to an actual NAO.

12

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Choregraphe programming environment

Monitor® is a software bundled with Choregraphe® that takes care of the other direction of data

flow: NAO data to you. Monitor® lets user to “monitor” two important aspects of NAO and get a data feedback from the robot. It provides two modes: Camera monitor and Memory monitor. Camera monitor provides a real­time image/video feed of what currently is seen by NAO. Using Memory monitor one can monitor the values set in NAO memory locations as well as states of the robot and all communications that are using a shared memory. Monitor® is very useful is debugging application developed either in Choregraphe or using SDK as standalone programs. Figures below show screenshots of Monitor® for monitoring the contents of memory and a live video feed from NAO.

13

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Monitor software showing the content of selected memory cells

Monitor software displaying what robot is seeing through its upper camera

14

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

qiBuild systemqiBuild is NAO’s build system created on top of CMake. It provides facilities for creating

projects, compilation, build, linking and program debugging. Using the SDK, programming NAO is easy once the programmer understands how the internals of the robot and NAOqi framework function.There are two sets of SDK provided for each operating system: One for local compilation of the C/C++ code to be run on the remote machine and another one for the Atom processor which requires cross­compilation on the development machine and transferring the executables to the NAO for execution. A toolchain using the desired SDK should be created first. By configuring a project before building it, one may specify which toolchain should be used in the build process.From model developer point of view both should have the same procedure for code generation. The only difference is what toolchain is chosen to be used. A tutorial for creating qiBuild projects, creating toolchain and building NAO application using the SDK is presented later.

NAOqi SDKNAO is supplied with a rich set of well documented APIs for different languages. These

languages include Python, C++, Java, .NET, Matlab and Urbi. Some of these APIs like C++ API are native meaning that they interact with NAOqi directly and some are basically wrappers in the target language that call functions in a native API (Matlab API using C++ API). All API’s have the same interface and function/class signature and the procedure to use them is all the same. Except that some are interpreted and some should be compiled.

C++ APIThe C++ API is probably the most complete and most efficient NAOqi API to be used.

However due to the compilation and build processes, one might face difficulties at first. An example “hello world” program using C++ API is as follows:#include <alproxies/altexttospeechproxy.h>

int main(int argc, char* argv[]) AL::ALTextToSpeechProxy tts("192.168.1.155", 9559); tts.say(“Hello World!”); return 0;

As one might have noticed, an object of class ALTextToSpecchProxy is created and a method on that object is called with appropriate arguments and that is all you need to code to make NAO “say” something. However you need to go through the project creation and build using qiBuild build system to be able to compile this code.

NAO API is composed of different sets of APIs each of which is dedicated to fulfill a specific purpose. For example ALMotionProxy class is intended to take care of most of motion­related behavior of the robot and ALRobotPostureProxy makes the robot switch smoothly between various predefined postures such as “StandUp”, “SitDown”, “LayBack”, etc. Example API

15

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

classes and some associated methods are listed in Table 2.

Table 2) Example NAO API classes and associated methods

API name Sample methods

Stiffness Control API ALMotionProxy::setStiffnesses() ALMotionProxy::getStiffnesses()

Joint Control API ALMotionProxy::setAngles() ALMotionProxy::getAngles() ALMotionProxy::closeHand() ALMotionProxy::openHand()

Locomotion control API ALMotionProxy::walkTo() ALMotionProxy::moveInit() ALMotionProxy::move()

You have to note in the motion category there are ALRobotPosture and ALRobotNavigation classes also available. There also exists other API for Core functionality, Audio, Vision, Sensors and Tracking

Python APIUsing Python API is probably the best place to start working with NAO robots since the

API installation and integration with the Python environment is very straightforward and it needs no compilation. An example usage of the API for the NAO to say “Hello World” is presented here:from naoqi import ALProxytts = ALProxy("ALTextToSpeech", "192.168.1.155”, 9559)tts.say("Hello world")

Matlab APIMost of NAO APIs are native classes and functions written and compiled in the target

language. However NAOqi Matlab API provided by Aldebaran Robotics for NAO is not a native API. NAO Matlab API is actually a series of Matlab classes and function wrappers that utilize a compiled version of C++ API through a compiled MEX file which will be described later. We should note that the MEX C++ files provided and compiled by Aldebaran is only a bridge between C++ API and Matlab wrapper functions.

a MEX file is a dynamically linked library file that is consist of a gateway function named “mexFunction()” which is the entry point by Matlab engine. This function is considered the “main” function of the program. All other C/C++ functions are called from this function. Based on the gateway nature of this type of files, there is an interface to Matlab® workspace variables.

After installation of NAOqi Matlab API, the piece of code needed to implement the ‘Hello

16

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

World” program in Matlab is basically the same as C++ and Python as shown below:tts = ALTextToSpeechProxy('192.168.1.155',9559);tts.insertData('Hello World');

Implementing simple behaviors using NAOqi Matlab® API

Here we show how to use the Matlab® API to implement simple behaviors. As with all other APIs, to make NAO do something, first we need to create an object of that type of behavior ( e.g. Motion, Speech, Vision, etc.) and then call the methods of the class on the object.In this example we show how to make a movement for the “HeadYaw” joint of NAO in Matlab®. For a detailed documentation of Joint Control API visit Aldebaran Robotics website.

Step 1) To create an object of type motion we simply need to call the class constructor with the IP address and port of the NAO as its arguments and assign the result to an object variable of that specific class, as follows:>> naoMotObj = ALMotionProxy('192.168.1.109', 9559)

If succeeded, a pointer ID will be assigned to the object. And this message will be displayed:Almotion proxy created!naoMotObj =ALMotionProxy with properties:ptrProxyNaoQi: 1432727224

Step 2) The next step would be to set the stiffness of the joint (or a group of joints using the parent name of the joints). To stiffen a specific joint (‘HeadYaw’):>> naoMotObj.setStiffnesses('HeadYaw', 1.0)

Alternatively we can stiffen a parent group of joints including “HeadYaw” and “HeadPitch” which is named as “Head”:>> naoMotObj.setStiffnesses('Head', 1.0)

Step 3) Calling the appropriate function to set the angle of a specific joint to a set value:>> naoMotObj.setAngles('HeadYaw', ­1.0, 0.2)

The first parameter is the standard name of the joint, the second is the target angle (ranges from ­2.0 to 2.0) and the last is the speed of movement (ranges from 0.0 to 1.0). After executing this last command you should see the robots head moving.You can also use angleInterpolation() function,>> naoMotObj.angleInterpolation('HeadYaw', [1.0, ­1.0, 0.0], [5.0, 15.0, 25.0], true)

The first argument specifies the joint. The second is a list of angles to be traversed based on the corresponding times list as the next argument and the last argument is a Boolean that denotes if the angles are absolute or the values are relative to the current angle. In the above example, the ‘HeadYaw’ joint will go to the angle 1.0 at time 5.0 then keeps moving “such that” it will be at angle ­1.0 at time 15.0 (10 seconds later) and then keeps moving till it is positioned at

17

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

angle 0.0 at time 25. So the whole process takes exactly 25 seconds from the beginning. This function smoothens the joint movement by using interpolations.

Step 4) When you are done using a joint or a group of joint, you should “unStiff” the joint(s) in order to avoid the motors to heat up as well as decreasing the power consumption. You “Un­Stiff” a joint or set of joints using the same function but calling it with a value of zero:>> naoMotObj.setStiffnesses('Head', 0.0)

This command “un­stiffens” the “Head” joint group which includes “HeadYaw” and “HeadPitch”. In Figure 6 and Figure 7 body skeleton of NAO and Head joint structure are displayed, respectively.

In this section, the basic concepts and tools of programming simple NAO behavior is briefly explained which will be used to understand the procedure of code generation from Simulink® models and StateFlow® chart. Complete NAO SDK reference and programming guide is available at[42].

Figure 6) NAO H25 skeleton (Courtesy of Aldebaran Robotics)

Figure 7) NAO Head joint (Courtesy of Aldebaran Robotics)

18

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

NAO Matlab API installationAldebaran Robotics provides comprehensive set of SDK in different languages.

Programmers are able to program NAO robots in languages such as C++, Java, Python, .NET and Urbi. For compiled languages there is also a build system that assists with the process of creating, configuring projects as well as compilation, build and debugging such as qiBuild for C++.NAO Matlab SDK is in fact a series of Matlab function wrappers that call two Matlab MEX functions “Matlabproxy.mexw32” and “Matlabcall.mexw32” that underneath make calls to NAO C++ APIs.Like any other Matlab toolbox, to install NAO Matlab SDK all you need to do is to download the SDK files, extract it and add the directory to Matlab path.

NAO Matlab SDK installation

NAO Matlab API troubleshootingWhen working with the NAO Matlab API, some run­time errors and exceptions might occur. The following list helps users to identify and resolve those issues.Error DescriptionUnexpected Standard exception from MEX file.What() is: ALNetwork::getModuleByNamefailed to get module ALMotion http://192.168.1.109:9559Error in ALMotionProxy (line 18)obj.ptrProxyNaoQi = matlabproxy('ALMotion',ip,port);Error Reason:

19

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

The network connection is not established. There is a problem with network connectivity between your computer and the network to which NAO is connected to. Try pressing NAO’s chest button to makes sure you are using the correct IP address and then try pinging the robot from your computer.Error Description:NAO joint is not moving after the execution of a motion related command such as “setAngel”:Error Reason:If you have successfully created a motion object using the “ALMotionProxy()” constructor, then this only has two reason:1­ You have not “Stiffened” the joints. Before performing any motion related command, you need to make sure you have turned on and activated the corresponding motors of the joints. This procedure is called stiffening and is a method of ALMotionProxy class. The following command activates the motors of joint “joint name” id n equals 1 and deactivates it if n equals 0; you can also provide a set of joints using a “joint group name” [reference] or providing a list of joints (list in Python, a vector in C++);motionObj.setStifnesses(<joint name>, n)2­ You are setting the angle of a joint to its current angle which obviously wouldn’t make it move.

20

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

qiBuild and SDK InstallationqiBuild is a build system by Aldebaran Robotics based on CMake that facilitates creating

cross­platform projects, compilation and build

qiBuild Installation∙ Install CMake from the website and modify the path to C:/CMake and when prompted to add to PATH, say ok.∙ Install Python 2.7+∙ Download the C++ SDK from the DVD (naoqi­sdk­1.14.1­win32­vs2008.zip) and copy it to a folder on your computer and unzip it. You can find it on the DVD in Download > Software > SDK > NAOqi SDK > C++∙ Also do the same for qibuild­1.14.1.zip. unzip it and run install­qibuild.bat, when installation is complete you get the following message

Figure 45) Final stage of qiBuild installation∙ Next you need to configure the qiBuild by entering:> qibuild config ­­wizardqiBuild finds the available IDEs and compilers on your system and asks to choose one as depicted in Figure 46.

21

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Figure 46) Configuring qiBuild

Creating a initializing a Work treeA Work tree is a top level folder for creating all your NAO projects. A Work Tree should be created, initialized and configured first. All new projects should be created inside that directory so that it would be accessible by qiBuild.To create a work tree and a project, go to your workspace directory and run:> qibuild initThis command creates a work tree that you can add your projects under it. The result is a folder names “.qi”.For your whole working directory you only need to perform “qibuild init” once which will cover all projects created inside that directory.

Creating a toolchainAs mentioned before, there are two SDK sets available for compiling NAO applications, one for desktop remote applications (either in Windows, MAC or Linux) and one for Atom processor to be run on the robot locally. A Toolchain is technically a configuration that is hooked to a specific

22

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

SDK and in compilation process by choosing a toolchain, you implicitly choose which SDK to use.The following line creates a toolchain named naoCPPtoolchain which points to the SDK for desktop application running on Windows 32 bit.> qitoolchain create naoCPPtoolchain C:\NAOqi\naoqi­sdk­1.14.1­win32­ àà vs2008\naoqi­sdk­1.14.1­win32­vs2008\toolchain.xml ­­default

Status of available toolchainsDuring work one might create multiple toolchains for different projects. It may become confusing. Check the status of your available toolchains using the following command. The result in Figure 47 shows the two available toolchains.> qitoolchain info

Figure 47) checking available toolchains

Creating, configuring and building a new projectTo create a new project in the current Work Tree, use the following command:> qibuild create newProjectsName

To configure the project to use a specific toolchain, use the following command:> qibuild configure ­c NameOfTheToolchain ProjectName

23

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Note) you have to be at the same directory as “.qi” folder to be able to configure or make a project, not inside the project folder.At this point you should have the following project hierarchy:

You can simply open the main.cpp file, modify and compile it. To make and build a project, execute the following command:> qibuild make ­c NameOfTheToolchain ProjectName

To rebuild the project (even if it is up to date), you can use the “­r” flag.

24

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

1.1.6 Using Microsoft Visual Studio to build the projectsNow if you check the projects directory you should notice that make files and even a

Visual Studio Solution file is created. That is because we chose “Visual Studio 2008” as our IDE and compiler. Now you can even open the solution in Visual Studio and modify your code and run it there. As it will be explained in section 8.4, that is what is needed to be done when we want to integrate the automatically generated code from our Simulink models with our qiBuild project.

Figure 48) The file strcuture of qiBuild project after initial buildYou can open the solution and simply use your designated IDE to write the code and build it. In Figure 49, a qiBuild project is opened, built and run in Visual studio.Note) When trying to run the project, after successful compilation, you need to specify an executable name. Choose Browse and point to the following address (of course if your project name is helloWorld):“<NAOWorkspace>\helloWorld\build­naocpptoolchain\sdk\bin\helloWorld.exe”

25

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Opening the qiBuild solution in VS and running it

Note) You can write and compile simple C/C++ applications at this point. However if you want to use the NAO SDK, for example including a header file by using the following directive#include <alproxies/almotionproxy.h>and actually work with the robot, you have to make one modification in the “CMakeLists.txt” file by going to the project’s root directory, opening the CMakeList.txt file and adding this line:qi_use_lib(ProjectName ALCOMMON)

after this line:qi_create_bin(ProjectName "main.cpp")

Finally remake the project using qiBuild make command as described previously.

26

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Connecting to OpenNAO through SSHThere are times you need to connect to the OpenNAO operating systems running on

NAO’s motherboard. One reason is to download the compiled application that is built using the Atom toolchain on the NAO. By modifying “autorun.ini” file, one can request execution of that compiled application after the robot is booted.You can use an SSH client such as putty to connect to the NAO via wi­fi or cable and providing its IP address.As mentioned before, there are two default Linux users defined in OpenNAO. One is the “root” and the other is “nao”. After version 14.1, OpenNAO doesn’t allow users to remotely connect SSH using the “root” user. You need to first connect using “nao” user and then switch to “root” using “sudo ­s” command.

Connecting to the NAONAO can use either Ethernet and Wi­Fi to connect to the outside world. Ethernet is initially used to bring the NAO up and setup the Wi­Fi connection. After that, it is recommended to use Wi­Fi to connect to NAO.We have used a private local Wi­Fi network in our lab to connect our desktop systems with NAOs. Since the Cisco router uses DHCP, it assigns your robots a new IP address every time it

27

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

is turned on. Using a simple procedure, you can use the router’s control panel, to assign each NAO a fixed IP address using it’s MAC address, that’s what we did! It makes life much easier!

Choregraphe and Monitorif you are new to NAO programming I strongly recommend you to first get a sense of it by starting to use Choregraphe.Choregraphe is a high­level block­based programming environment that helps you easily start programing NAO robots. It provides a simple procedure to connect to the actual robot or the simulated NAO, design a behavior or set of actions and remotely execute that behavior.

Monitor is the other software by Aldebaran that is used to actually “Monitor” the activities of a NAO either in a simulated environment or on the actual robot. Using Monitor you can see what the robot is observing through it’s Camera. So basically it’s a gateway to running NAO’s camera. Another application of Monitor is to check the Memory of NAO and the values that are stored in it.

To start learning Choregraphe, you can follow this tutorial from Aldebaran:Your first steps in Choregraphe

When you finished this tutorial, you can follow this page for further instructions and tutorials to fully exploit the capabilities of Choregraphe.Choregraphe User Guide

Even though Choregraphe seems at first like a primitive application, it has much more to offer, spend more time to explore and learn it and you can create very complex behaviors using this software very quickly.­ You can add your own custom blocks written in Python and usnig the Python SDK so extending the block library.­ You can use the Timeline box and to create complex behaviors on the robot.

Installing Choregraphe for Linuxgo to http://www.aldebaran­robotics.com/Downloads/85­Software/View­category.html anddownlaod Choregraphe. we choose Choregraphe software suite 1.14 for Linux 32 bits.Note) You have to first create a free account to be able to download the software anddocumentation.Download the file and save it in an easily accessible directory like /home/ramtin/software. thenusing any unzip tool, unzip the package. you can run it using the scripts available in its rootdirectory (eg. choregraphe or monitor). create a shortcut (make link) of each and copy it on yourdesktop for easy access.

Now you can use the Choregraphe with webots. You create a robot algorithm usingChoregraphe blocks and connect to webots as simulator and simulate the behavior or you caneasily use Choregraphe to run the algorithm directly on the actual NAO (without downloading the

28

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

code! that is another process which requires cross­compilation, discussed later)

To simulate a behavior in webots:Open Webots. from File ­> open world... , navigate to “webots/projects/robots/nao/worlds/”and open nao.wbtIt starts running the world automatically. if not, click on the play button to start simulation. Nowopen Choregraphe and create a behavior (simplest would be choosing the Tai Chi Chuan dancemodule) and place it in the area. as follows:

click on the red antenna logo to disconnect and then the green one to connect to the robotrunning in Webots. Choose the robot based on its IP:port and connect. Then by clicking on the“green gears” icon, disable the “stiffen all joints” to make it red. Then click on the play button inChoregraphe. It starts the program and you see the simulation running in Webots. You can alsoopen “monitor” utility (that comes with Choregraphe) to monitor the status of memory or see therobots camera view.

using the above guide you can use Choregraphe with Webots. Now if you want to writeprograms in C++ or Python and run as an stand­alone application that commands the actual orsimulated NAO, you need to install the NAOqi SDK, and use it in your favorite IDE with providedAPIs. This is how you can do it.

5. Programming NAOYou can write pieces of code to work with NAO, get sensory data, process those data and actuate it’s joints or make it say a word. There are two approaches for doing so.One is is to develop your application on your desktop/laptop computer and remotely execute the

29

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

application on the robot using a Wi­Fi connection. Obviously no code or program is downloaded onto the robots but the process is faster for testing and development purposes.Second approach which is good for achieving the ultimate performance, needs you to download the code and let it execute on the NAO’s local processor, in a stand alone fashion. The following describes each of these methods:

a. Remote Controlled NAOYou can

i. Python SDKUsing Python SDK is very easy and comfortable since Python is a scripting languages and need not be compiled. You can just write your Python code and execute it instantly.Of course you need to install Python language itself on your system before moving forward. Take a look at Python Installation Guide for Windows and Mac.To install Python on Debian based Linux systems such as ubuntu, simply:$ sudo apt­get install python

Follow the instruction on Python SDK Install guide to be able to program NAO using Python languages remotely.For Linux you just need to download the SDK from the DVD based on your system’s specification (either 32bit or 64bit). For example:pynaoqi­python­2.7­naoqi­x.x­linux32.tar.gz

Unzip it in a folder and change the Python path using the following command:$ export PYTHONPATH=$PYTHONPATH:/path/to/python­sdk

Error!When trying the above command, if you get the following error:bash: export: `=/home/ramtin/software/pynaoqi­python­2.7­naoqi­1.14­linux32/': not a valididentifiertry first setting the variable PYTHONPATH and then exporting it:$ PYTHONPATH=$PYTHONPATH:/path/to/python­sdk$ export PYTHONPATH

make sure you don’t put any spaces around the equal sign, that also causes errors.

Tip!Note that the exportcommand changes the Python Path variable for the current terminal session. To make a permanent change, you have to add the above command to the end of your bashrc file:$ gedit ~/.bashrcYou can use gedit, vi or any other editor.

30

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Now to check if now Python has access to NAOqi SDK, just run Python and try importing Naoqi modules. If you didn’t get any errors, then you are good to go!$ Python (This is bash to start Python Shell)>> import naoqi (This is in Python Shell)

Example 1) Remote Programming ­ PythonThis short piece of code, remotely connects to the NAO (either the simulated NAO or the real NAO) and turns it head around for the specified angle. To run your first example, Turn on the NAO, run python in the terminal (on your desktop) and enter the following commands in Python shell:>> from naoqi import *>> tts = ALProxy(“ALTextToSpeech”, “192.168.1.126”, 9559)>> tts.say(“Hello World!”)

if successful, NAO should say something! Now let’s describe what each line does:>> from naoqi import *This line imports the NAO SDK libraries to you program. If you got an error after executing thisline, go back and try to check your PYTHOPATH. Note that this specific example loads the wholelibrary which is not efficient. It’s better to only load libraries that you need, like the following:>> from naoqi import ALProxywhich only loads the ALProxy library to your program environment.

>> tts = ALProxy(“ALTextToSpeech”, “192.168.1.126”, 9559)This statement instantiates an object of class ALProxy for the NAO with the specified IP addressand the default TCP port of 9559.

Example 2) Remote Programming ­ PythonThis example uses ALMotion proxy directly without the use of ALProxy module and moves thehead of the NAO.>> from naoqi import ALProxy>> motionObject = ALProxy(“ALMotion”, “192.168.1.126”, 9559)>> motionObject.setStiffnesses(“Head”, 1.0)>> motionObject.setAngles(["HeadYaw"], [1.0], 0.5)

Note!if the code worked without issuing any error but nothing moved, maybe the head is already atthat goal position. So try changing the argument values of setAngle function. Note that the firstone in the bracket is the target angle which could be in the range [­1.0, 1.0]. The last argument isthe speed of movement which should be in the range [0.0, 1.0].

Now lets see how this piece of code works.

31

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

The first two lines work the same way as the first example. The only difference is that we arecreating a proxy object for the ALMotion module instead of ALTextToSpeech module.ALMotion module provides functionality for working with NAO’s joins and locomotion.The third line is calling the setStiffnesses() function for the “Head” body part and sets it to one.

But what is stiffness?Stiffness in mechanical robots with joints (and NAO of course!) is referred to as the motors of a joint of group of joints being on or off. When a joint’s stiffness is off (or set to Zero), the motors are off, so you can modify the position of the joints “by hand” and if you try to move the joint by programming, you cannot! but when you change the stiffness of a joint to “On” or set it’s value to One, you turn the motors “On”. Now the Joints are kind of locked and you cannot (and should not) try to move it by hand. However you can move the joints by programming.So, to summarize, to start working with a joint, first you need to set the stiffness of that joint, or group of join to which that specific joint is a member of, to “One”. Then make the movements and change the angles etc. Make sure to set the stiffness to “zero” or “Off” when you are done since keeping the motors on for a long time causes overheating of the motors and NAO joints.

The last line of the code, simply takes a list of joints (here just “HeadYaw”) and a list of associated goal angles and finally a speed of rotation and sets the angle of the joint to the goal angle with the specified speed.

Note!You can run both previous examples either line by line in Python shell or you can put all the code in one python script file such as naoCommands.py and execute it directly as a whole program:http://www.aldebaran­robotics.com/documentation/naoqi/index.html$ python naoCommands.py

More on Joint ControlJoint control APIHello World using Python

ii. C++ SDKFor programming NAO using C++ SDK, you need to install the build tools provided by Aldebaran which is called qiBuild. To install follow the instructions in the following pages:C++ SDK InstallationUsing qiBuild with Aldebaran packages

For Ubuntu, you can simply follow these instructions:

Installing NAOqi SDK

Note)

32

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

to install cmake:sudo apt­get install cmake

for step 5 you can use the following to get the qiBuild from its GIT source:(You might need to install GIT utility before that, using sudo apt­get install git)

git clone git://github.com/aldebaran/qibuild

for step 6, to make sure the directory /home/ramtin/.local/bin is in your PATH, do the following:> PATH=$PATH:/home/ramtin/.local/bin> export PATH

ERROR) In step 4 of this page if you received an error regarding not finding the BOOST library,you need to install the library before you proceed:

sudo apt­get install libboost­all­devBut its not always the case. sometimes you don’t need to do this.

After finishing step 5, follow the link on using an IDE:http://www.aldebaran­robotics.com/documentation/qibuild/guide/overview/building_a_project.html#using­an­ideand follow that tutorial.

Using qiBuild and starting a new project:http://www.aldebaran­robotics.com/documentation/qibuild/qibuild_in_five_minutes.html

Installing Webots for Linuxusing the installation procedure available at page 22 of:http://www.cyberbotics.com/guide.pdfinstall the webots. You need to first add the repository to the resources list by opening:

sudo vi /etc/apt/sources.list

and add this line to the end of it:

deb http://www.cyberbotics.com/debian/ binary­i386/ (if using a 32 bit machine)

then you should download the Cyberbotics.asc key file from http://www.cyberbotics.com/linuxand adding it to the system by:

sudo apt­key add /home/ramtin/software/Cyberbotics.asc

33

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

then update apt list:

sudo apt­get update

and finally install webots using:

sudo apt­get install webots

You can either use the trial or by going to the preferences provide a license key:

Starting a new C++ project using qiBuildqiBuild in five minutes­ $ create a new directory­ $ qibuild init // to create a new worktree­ $ qibuild create newProject­ If no toolchain is created yet: $ qitoolchain create nameOfToolchain /path/to/cpp/sdk/toolchain.xml ­ to check if there is any toolchain created before: $ qitoolchain status­ To use an already created toolchain: $ qibuild configure ­c NameOfTheToolchain ProjectName $ qibuild make ­c NameOfTheToolchain ProjectName

­ You shoudl get success at this point. to run the compiled binary file, go to/Project/build­ProjectName/sdk/bin and execute the binary file: $ ./BinaryFile

­ Still if you change main.cpp and add any of the Aldebaran include headers (e.g #include<alproxies/almotionproxy.h>), you get a compile error! to resolve this: ­ go to the project root ­ open CMakeList.txt ­ Add this line:qi_use_lib(ProjectName ALCOMMON) // project name is the name of the executable

after: qi_create_bin(ProjectName "main.cpp")

and re­make the project using qibuld make command as described above

34

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

iii. Matlab SDKMatlab SDK is actually a Matlab toolbox which provides the same API as in other languages. The difference is the functions are actually wrappers that call C++ API functions in a precompiled MEX file.

b. Stand alone Code on NAOi. Python SDKii. C++ SDK

6.

Tip!if you have worked with ROS, just think of NAOqi as ROSCore and think of NAOqi Modules as ROS packages. It’s not exactly the same but this helps you understand how NAO works if you are already familiar with ROS.

NAOqi:NAOqi which runs on OpenNAO OS on the NAO Robots is like ROS running on Ubuntu.

OpenNAO:The Linux distribution installed on NAO’s hardware. A descendant of Gentoo Linux.

Modules:NAO Moduels in NAO are like Packages in ROS. They provide services and functions to be called.

35

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

You can create a proxy and use the functionalities provided by those modules. Examples fo these modules are ALTextToSpeech, ALMotion, ALRobotPosture etc.You can create a Proxy object of class ALProxy with the target module as a parameter to the constructor of ALProxy class and create an object which can access the services and functions provided by those modules.

36

Cyber Physical Systems Lab @ Arizona State University ­­ Author: Ramtin Raji Kermani

Installing Webots for Linuxusing the installation procedure available at page 22 of:http://www.cyberbotics.com/guide.pdfinstall the webots. You need to first add the repository to the resources list by opening:

sudo vi /etc/apt/sources.list

and add this line to the end of it:

deb http://www.cyberbotics.com/debian/ binary­i386/ (if using a 32 bit machine)

then you should download the Cyberbotics.asc key file from http://www.cyberbotics.com/linuxand adding it to the system by:

sudo apt­key add /home/ramtin/software/Cyberbotics.asc

then update apt list:

sudo apt­get update

and finally install webots using:

sudo apt­get install webots

You can either use the trial or by going to the preferences provide a license key:

37