Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro
-
Upload
facultad-de-informatica-ucm -
Category
Engineering
-
view
112 -
download
3
Transcript of Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro
![Page 1: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/1.jpg)
Desarrollo de robots sociales con RoboComp
Pablo BustosRoboLab
Universidad de Extremadura
UCM - 2016
![Page 2: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/2.jpg)
A quick introduction to
2000-
![Page 3: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/3.jpg)
Social and Service Robotics
● Robots that help humans in everyday activities
![Page 4: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/4.jpg)
Social and Service Robotics
● Robots that help humans in everyday activities
Educational
![Page 5: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/5.jpg)
Social and Service Robotics
● Robots that help humans in everyday activities
Educational Rehabilitation
![Page 6: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/6.jpg)
Social and Service Robotics
● Robots that help humans in everyday activities
Educational Rehabilitation Advertisement
![Page 7: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/7.jpg)
Social and Service Robotics
● Robots that help humans in everyday activities
Educational Rehabilitation Advertisement Affective
![Page 8: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/8.jpg)
Social and Service Robotics
● Robots that help humans in everyday activities
Educational Rehabilitation Advertisement Affective Butler
![Page 9: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/9.jpg)
Educational robots - Robex
● 2005 - now● college, comp. sci.● low-cost● versatile● robust● classroom● adaptable
![Page 11: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/11.jpg)
Educational robots in school - Dulce
● 2011● 6th grade primary school● 1 semester experiment
![Page 13: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/13.jpg)
Educational robots in school - Learnbot 1
● 2012● Initiation to Python
programming for kids in 5th to 12th grades
● Build with RaspberryPi and Arduino
● WiFi connection to laptop running Python
![Page 14: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/14.jpg)
Educational robots in school - LearnBot 2
● Odroid only● WiFi, battery, USs and USB camera● under 150€ for one prototype
![Page 15: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/15.jpg)
Educational robots in school - LearnBot 2
def code(self):# 0 back 3 left 2 right 1 frontactions = ((150,-3), (450, 0), (150, 3)) while(True):
frame = self.getImage()rois = self.getROIS(frame)maxIndex = np.argmax(rois)self.setRobotSpeed(actions[maxIndex])
sum of black pixels in three ROIs
![Page 16: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/16.jpg)
Rehabilitation Robots
![Page 17: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/17.jpg)
Rehabilitation robots - Ursus
● 2010● Dynamixel servos● Visual mark detection
![Page 18: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/18.jpg)
Rehabilitation robots - Ursus
![Page 19: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/19.jpg)
Rehabilitation robots - Ursus
● Technology validation experiment● 5 patients in 7-10 age range● mild lesions affecting the shoulder (brachial or
cerebral paralysis)
![Page 22: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/22.jpg)
Advertisement robots - Gualzru
● a product for INDRA Spanish tech company.
● completely designed and built our group of research teams.
● differential base, batteries, tactile panel, leds for the eyes, micro, speaker, laser, RGBD.
● runs RoboCog on top of RoboComp framework.
● dozens of hours of operation.
![Page 24: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/24.jpg)
Gualzru building process
![Page 27: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/27.jpg)
Affective robots - Muecas
● 13 motors● Stereo vision● IMU● Great for HRI
○ affective robotics○ visual attention○ mouth synchronization○ generation of facial gestures○ speech augmented with facial
expressions
![Page 32: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/32.jpg)
Social robots - Loki
● too expensive● Schunk motors and Barrett hand, hard to integrate in open source designs
![Page 35: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/35.jpg)
Social robots - Shelly
![Page 37: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/37.jpg)
Social robots - Shelly
2014 Toulouse2015 Lisboa
![Page 38: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/38.jpg)
Social Robotics - Autonomy Lab
● 70 m²● almost ready to live
apartment
![Page 39: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/39.jpg)
Group of groups
robotics
planning
health vision
audio
natural language processing
![Page 40: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/40.jpg)
A family of robots
![Page 41: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/41.jpg)
RoboComp
![Page 42: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/42.jpg)
Why is robotics software hard?
Drivers People Failure Algorithms
![Page 43: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/43.jpg)
The complexity trap
![Page 44: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/44.jpg)
Robotics frameworks
2002 2003 2006 2008 201020092004 2005 2007
![Page 45: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/45.jpg)
RoboComp
● Distributed, component-oriented ● Formal component model● pub/sub and RPC comm models● Industrial grade communications middleware (Ice) with public
interfaces● Code generation using DSLs technology● Tool set: deploy, monitor, log, simulator● C++ and Python● > 100 components● Talks to ROS using DSL tech.
![Page 46: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/46.jpg)
Component model
SpecificWorker
GenericWorker
+ compute()=0
GenericMonitor
Main
thread
SpecificMonitor
thread
- SpecificWorker w
Ice App
user code
automatically generated code
INTE
RN
ET
![Page 47: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/47.jpg)
Communication modelspush / pull pub / subscribe
thre
ad
publications
both can be synchronous or asynchronous
thre
ad
requests
thre
ad
thre
ad
![Page 48: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/48.jpg)
Communication models (code)pub / subscribe
while(True):c = proxyA.getData(a,b)d = proxyB.update(c)do_something_useful( c, d )
push / pull
while(True):a,b =check_is_something_arrived()do_something_useful( a, b )
## Running in middleware’s threaddef subscribe_method1(a):
safe_copy_to_local(a)
def subscribe_method2(b):safe_copy_to_local(b)
![Page 49: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/49.jpg)
Communication models (comp view)
middleware code
requests
subscriptions
requests
publications
while(True):c = proxyA.getData(a,b)STORM.publish( topic )a,b =check_is_something_arrived()do_something_useful( a, b )
## Running in middleware’s threaddef subscribe_method1(a):
safe_copy_to_local(a)
def interfaz_method(a):safe_copy_from_local(a)
![Page 50: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/50.jpg)
Component model: ComponentDSLimport "/robocomp/interfaces/IDSLs/interface1.idsl";import "/robocomp/interfaces/IDSLs/interface2.idsl";
Component mycomponent{ Communications { implements interface1; requires interface2; subscribesTo topicToSubscribeTo; publishes topicToPublish; }; language Cpp; gui Qt(QWidget); statemachine statemachine.smdsl;};
![Page 51: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/51.jpg)
Component model: InterfazDSLmodule RoboCompDifferentialRobot {
exception HardwareFailedException { string what; };
struct TBaseState {
float x;
float correctedX;
float z;
float correctedZ;
float alpha;
float correctedAlpha;
float advV;
float rotV;
float adv;
float rot;
bool isMoving;
float voltage;
};
.idsl file
interface DifferentialRobot
{
void getBaseState(out TBaseState state) throws HardwareFailedException;
void getBasePose(out int x, out int z, out float alpha) throws HardwareFailedException;
void setSpeedBase(float adv, float rot) throws HardwareFailedException;
void stopBase() throws HardwareFailedException;
void resetOdometer() throws HardwareFailedException;
void setOdometer(TBaseState state) throws HardwareFailedException;
void setOdometerPose(int x, int z, float alpha) throws HardwareFailedException;
void correctOdometer(int x, int z, float alpha) throws HardwareFailedException;
};
};
Ice provides binaries to generate proxies for different languages
![Page 52: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/52.jpg)
Component model: StateMachineDSLName_machine { [ states name_state *[, name_state];] [ initial_state name_state;] [ end_state name_state;] [ transition { name_state => name_state *[, name_state];
*[name_state => name_state *[, name_state];] };]};
[:parent_state [parallel] { states name_state *[, name_state]; [ initial_state name_state;] [ end_state name_state;] [ transition { name_state => name_state *[, name_state]; *[name_state => name_state *[, name_state];] }; ]};]
![Page 53: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/53.jpg)
Component model: StateMachineDSLMyMachine { states State2, State3, State4, State5; initial_state State1; end_state State6; Transition { State1 => State1, State2; State2 => State3, State5, State6; State3 => State3, State4;
State4 => State5;State5 => State6;
};};:State1 parallel { states State11, State12; Transition { State11 => State11; State12 => State12; };};………..
![Page 54: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/54.jpg)
DSL Code Generator
import "/robocomp/interfaces/IDSLs/AprilTags.idsl";import "/robocomp/interfaces/IDSLs/AprilBasedLocalization.idsl";import "/robocomp/interfaces/IDSLs/Laser.idsl";import "/robocomp/interfaces/IDSLs/NewAprilPose.idsl";
Component MyComp{ Communications { requires Laser;
implements NewAprilPose; subscribesTo AprilTags;
publishes AprilBasedLocalization; };
language Cpp; statemachine statemachine.smdsl;};
.cdsl file
.idsl files
communication primitives
user@machine: robocompcdsl mycomponent.cdsl
![Page 55: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/55.jpg)
… generates a code directory
MyComp
myComp.cdsl bin CMakeLists.txt DoxyFileetcREADME.mdsrc
myComp main.cpp
CMakeLists.txt
genericmonitor.cppspecificmonitor.cpp
genericworker.cpp
specificworker.cpp
config
1500 lines of code in C++
![Page 56: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/56.jpg)
RoboComp Tools
![Page 57: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/57.jpg)
RoboComp’s other tools: RCManager
![Page 58: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/58.jpg)
RoboComp other tools: 3D Simulator
Example: testing generalized inverse kinematics using LM non-linear optimization
Based on the Open Scene Graph engine
![Page 59: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/59.jpg)
CORTEX
![Page 60: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/60.jpg)
CORTEX: a proposal for a cognitive architecture
A graph-based representation of the robot and the world, shared by a set of reactive-deliberative agents
![Page 61: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/61.jpg)
RoboComp tools: DeepStateRepresentation
![Page 62: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/62.jpg)
RoboComp tools: DeepStateRepresentation
3D view
![Page 63: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/63.jpg)
CORTEX: a proposal for a cognitive architecture
![Page 64: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/64.jpg)
CORTEX: a proposal for a cognitive architecture
● ¿What is the proper division of functionalities?
![Page 65: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/65.jpg)
CORTEX: a proposal for a cognitive architecture
● ¿What is the proper division of functionalities?
● ¿What is the correct choice of agents?
![Page 66: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/66.jpg)
CORTEX: a proposal for a cognitive architecture
● ¿What is the proper division of functionalities?
● ¿What is the correct choice of agents?
● ¿Is developmental cognition the only way?
![Page 67: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/67.jpg)
… a real world problem
The BringMe(X) functionWhat the hell!
Bring me the cup, please
![Page 69: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/69.jpg)
Bring me the cup, please
cup is in the kitchen
Thank you!
Decision Making
Navigation & Manipul. Objects Interaction
Paps is in the living room
You are here
Reinforcement
Planning /Replanning
Monitoring
Object detection and learning
Trajectory control
Path planning
Annotated path
Path planning
ASR + NLP + TTSCORTEX
ASR + NLP + TTS
Monitoring
Monitoring
Monitoring
Monitoring
Monitoring
Monitoring
Monitoring
You are here
Lifelong learning
Lifelong learning
Lifelong learning
Lifelong learning
Lifelong learning
Lifelong learning
Lifelong learning
Lifelong learning
TRANSFER & MULTI MODAL HRI
INCREM. OBJECT LEARNING REACH & GRASP
NATURAL LANGUAGE DIALOG SEMANTIC PATH PLANNING ANNOTATED NAVIGATION COGNITIVE SUBTRACTION
3D OBJECT FITTING
POLICY LEARNING
Grasping
3D registration to known models
The problem
![Page 70: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/70.jpg)
New project
LifeBots: Lifelong Technologies for Social Robots in Smart Homes
MINECORetos de la Sociedad2016-2019
![Page 71: Desarrollo de robots sociales con RoboComp - Dr. Pablo Bustos García de Castro](https://reader031.fdocuments.mx/reader031/viewer/2022022411/58ee13171a28ab24118b466d/html5/thumbnails/71.jpg)
Thanks