TOOLBOX PVM - ugr.esjfernand/investigacion/papers/Users99_pres.pdf · 3 TOOLBOX PVM 3 He then spent...
Transcript of TOOLBOX PVM - ugr.esjfernand/investigacion/papers/Users99_pres.pdf · 3 TOOLBOX PVM 3 He then spent...
1
TOOLBOX PVM
Javier Fernández [email protected]. de Arquitectura y Tecnología de ComputadoresUniversidad de Granada
>> 1999
>> congreso usuarios MATLAB
2
2TOOLBOX PVM
Esquema de la PresentaciónIntroducción: MATLAB Paralelo
Otras Toolboxes Paralelas
Paso de Mensajes: PVM
Implementación MEX
Ejemplo de aplicación: Wavelets
Conclusión
3
3TOOLBOX PVM
He then
spentfiveyears
with
two
computerhardw
are manufacturers
theIntel H
ipercubeorganization
andA
rdentCom
puter
4
4TOOLBOX PVM
Ittookfarlongerto
distributethe
data thanit
didto
do thecom
putation.
...parser, interpreter, andgraphics
routines, where
anyparallelism
isdifficultto
find.
outerloop
Thereare notenough
potentialcustomers
with
parallelmachines.
5
5TOOLBOX PVM¿Por qué no hay MATLABParalelo?
Modelo Memoria distribuida: Overhead comunicación
Granularidad MATLAB: parser, intérprete, gráficos...
Situación comercial: poco uso,optimizar monoprocesador
¿Por qué hay tantos ToolboxesParalelos?• Diversos métodos de comunicar datos/instr.• Diversa funcionalidad• Diversos grados integración con MATLAB
6
6TOOLBOX PVM
Otras Toolboxes ParalelasToolbox Mecanismo llamadas Fecha Centro
Paralize Disco Común 2 enero-99 Chalmers, SwedenPMI MPI 19 marzo-99 [email protected] TCP/IP 12 abril-99 Mitthögskolan, SwedenTMath Engine Interface 9 ago-97 [email protected] PVM 3.3.7 24 abril-95 Wake Forest Univ.DP-TBox PVM 3.4.0 56 marzo-99 Rostock Univ.MultiMATLAB MPICH 27 nov-97 Cornell Univ/MathWorks
PVM-TB PVM 3.4.b7 93 junio-99 Univ. Granada
7
7TOOLBOX PVM
Paralizecomandos:
• serve.m• paralize.m
servidores:• SGI Origin2000 4CPU
• HP740• 9 Pentium II 450MHz
códigofor I=1:12N=25*I; a=rand(N,N,20); tic;x=paralize('eig',a); t(I)=toc;
end
8
8TOOLBOX PVM
MultiMATLABpseudo-espectro
• de una matriz 64x64• implica 1024 SVD’s• O(1013) flops
fácil paralelización• muchos flops• comunicación: máx/mín
IBM RS/6000 SP2, 4 CPU
código:http://www.tc.cornell.edu/...~anne/projects/MM.html
9
9TOOLBOX PVM
PVMTBInterfaz completa de PVM⇒ Util como herramienta didácticaEnlace dinámico PVM ↔ MATLAB
Paso de mensajes de cualquier tipo MATLAB
cells, structs, double, single, char, int8...
Interfaz con notificador PVMaddmhf, delmhf, recvf
Interfaz con ganchos (hooks) PVMreg_hoster, reg_tasker, reg_rm
Entorno de alto nivel disponible (MM-TB)
10
10TOOLBOX PVM
PVM
Estándar de-facto para Paso de Mensajes
Portable, interoperable:• PC bajo Win/NT, Linux, Solaris, FreeBSD, OS/2...• SPARC bajo SunOS, Solaris, Linux...• I860 HyperCube, Titan, Paragon, SGIMP64, CRAYSMP...
Escalable a centenares de máquinas, config. dinámica
addhost, delhost, config, notify...
Notificador, hooks, señales, política balance
11
11TOOLBOX PVM
Implementación MEX
Obligatoria (PVM es una librería C)
Enlace dinámico• obligatorio en Fichero MEX• ventajoso en librería PVM
Clasificación sistemática de llamadas PVM
• int=f(int), int=f(string, int)...• 16 clases distintas
Uso de nuevos tipos MATLAB 5.x• string cell-arrays, structs...
i lifi ió i t f MATLAB PVM
13
13TOOLBOX PVM
Algoritmo SecuencialU
sers99SecRG
B.m
%%%%%%%%%%%%%%%%%%%%
% Imagen y Filtros %
%%%%%%%%%%%%%%%%%%%%
lena=imread('tif/chuck','tif'); load h, load g
gd=[ g(2:length(g));0];
gr=[0;g(1:length(g)-1)];
record('Load');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Descomposición 3 niveles, 3 BitPlanes %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
RED =double(lena(:,:,1));
[a1 dh1 dv1 dd1]= mydwt2 ( RED,h,gd,'spec');
record('RD1');
[a2 dh2 dv2 dd2]= mydwt2 ( a1,h,gd,'spec');
record('RD2');
[a3 dh3 dv3 dd3]= mydwt2 ( a2,h,gd,'spec');
record('RD3');
tR=math(a3,dh3,dv3,dd3,dh2,dv2,dd2,dh1,dv1,dd1);record('Rmth');
GREEN=double(lena(:,:,2));
[a1 dh1 dv1 dd1]= mydwt2
(GREEN,h,gd,'spec');
record('GD1');
[a2 dh2 dv2 dd2]= mydwt2
( a1,h,gd,'spec');
record('GD2');
[a3 dh3 dv3 dd3]= mydwt2
( a2,h,gd,'spec');
record('GD3');
tG=math(a3,dh3,dv3,dd3,dh2,dv2,dd2,dh1,dv1,dd1);record('Gmth');
BLUE =double(lena(:,:,3));
[a1 dh1 dv1 dd1]= mydwt2
(BLUE,h,gd,'spec');
record('BD1');
[a2 dh2 dv2 dd2]= mydwt2
( a1,h,gd,'spec');
record('BD2');
[a3 dh3 dv3 dd3]= mydwt2
( a2,h,gd,'spec');
record('BD3');
tB=math(a3,dh3,dv3,dd3,dh2,dv2,dd2,dh1,dv1,dd1);record('Bmth');
%%%%%%%%%%%%
% Gráficas %
%%%%%%%%%%%%
figure,
record('fig');
imshow(lena), title('Original'),
record('show');
transform=cat(3,tR,tG,tB);
figure,
record('fig');
imshow(transform), title('Transformada'),
record('show');
14
14TOOLBOX PVM
Algoritmo Secuencialm
ydwt2.m
function [ahav, ahdv, dhav, dhdv]=mydwt2(img, h,g, varargin)
%%%%%%%%%%%%
% Longitudes
%%%%%%%%%%%%
lih =size(img,2); liv =size(img,1);% Length image horz/vert
lih2=ceil(lih/2); liv2=ceil(liv/2);% decimar puede salir 1 más
%%%%%%%%%%%%%%%%%%%
% Filtrar por filas
%%%%%%%%%%%%%%%%%%%
ah=zeros(liv,lih2);
% Pre-allocation
dh=zeros(liv,lih2);
for row=1:liv
%%%%%%%%%%%%%%%%%%%%
r = double(img(row,:));
% Obtener img->ah,dh
[ra rd] = mydwt
(r',h,g, varargin{:});%%%%%%%%%%%%%%%%
ah(row,:) = ra';
dh(row,:) = rd';
end
%%%%%%%%%%%%%%%%%%%%%%
% Filtrar por columnas
%%%%%%%%%%%%%%%%%%%%%%
ahav=zeros(liv2,lih2);
% Pre-allocation
ahdv=zeros(liv2,lih2);
dhav=zeros(liv2,lih2);
% Pre-allocation
dhdv=zeros(liv2,lih2);
for col=1:lih2
c = ah(:,col);
%%%%%%%%%%%%%%%%%%%%%%%%%%%
[ca cd] = mydwt
(c,h,g,varargin{:}); % Obtener ah->ahav,ahdv
ahav(:,col)= ca;
%%%%%%%%%%%%%%%%%%%%%%%%%%%
ahdv(:,col)= cd;
c = dh(:,col);
%%%%%%%%%%%%%%%%%%%%%%%%%%%
[ca cd] = mydwt
(c,h,g,varargin{:}); % Obtener dh->dhav,dhdv
dhav(:,col)= ca;
%%%%%%%%%%%%%%%%%%%%%%%%%%%
dhdv(:,col)= cd;
end
15
15TOOLBOX PVM
function [a, d]=mydwt(x,h,g, varargin)
%% se¤al x, approx. h, detail g, 'spec'|'zero'%
%%%%%%%%
% ArgChk
%%%%%%%%
lx=length(x); lh=length(h); lg=length(g);
if lh~=lg, error('debe coincidir longitud filtros'), end
lh2=floor(lh/2);
lg2=floor(lg/2);
mode='zero';
if ~isempty(varargin), if isstr(varargin{1})
switch varargin{1}
case {'spec','zero'}, mode=varargin{1};
otherwise, warning('modo no reconocido')
end
else, warning('argumentos adicionales no string')
end
end
%%%%%%%%%%%%%%% Aproximación %%%%%%%%%%%%%%%%%%%%%
% 2 reflejos de tama¤o lh2 y Downsample
% ly == lx+2lh2 %% la == ly+lh-1 == lx+2lh2 + lh-1
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if strcmp(mode,'spec')
y=[x(lh2+1:-1:2); x ; x(lx-1:-1:lx-lh2)];
else, y=[zeros(lh2, 1); x ; zeros(lh2,1)]; end;
a= conv
(y,h);
a=a(lh:lx+lh-1);
a=a(1:2:lx);
%%%%%%%%%%%%%%% Detalle %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if strcmp(mode,'spec')
y=[x(lg2+1:-1:2); x ; x(lx-1:-1:lx-lg2)];
else, y=[zeros(lg2, 1); x ; zeros(lg2,1)]; end;
d= conv
(y,g);
d=d(lg:lx+lg-1);
d=d(1:2:lx);
Algoritmo Secuencialm
ydwt.m
18
18TOOLBOX PVM
Algoritmo ParaleloU
sers99ParRG
B.m
NUMSLV=2;
% =3;
[numt tids]= pvm_spawn
('/opt/kde/bin/kvt',...
{'-display' 'ox0:63' '-e' '/usr/local/bin/matlab'},...
33,'oxigeno.ugr.es',NUMSLV);
if numt~=NUMSLV, error('no consigo arrancar MATLABs'), end
TAG=7; RAW=1; NUMCMDS=1; QUIT=0;
% Enviaremos 1 descomposición
for i=1:numt, pvm_initsend
(RAW);
pvme_pack
(NUMCMDS,QUIT);
pvm_send
(tids(i),TAG);
end
% Ignorar respuesta confirmación
for i=1:numt, pvm_recv (tids(i),TAG); end,
record('PVM');
%%%%%%%%%%%%%%%%%%%%% Imagen y Filtros %%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%% Descomposición 3 niveles %%%%%%%%%%%%%%%%
TAG=7; RAW=1;
img = double(lena(:,:,1)); cmd = ['dwt2ParRGB
(img,h,g, 1);'];
pvm_initsend
(RAW);
pvme_pack
(img,h,g,cmd);
pvm_send
(tids(1),TAG);
img = double(lena(:,:,2)); cmd = [ 'dwt2ParRGB
(img,h,g, 1);'];
pvm_initsend
(RAW);
pvme_pack
(img,h,g,cmd);
pvm_send
(tids(2),TAG);
img = double(lena(:,:,3));
if NUMSLV==3,
cmd = [ 'dwt2ParRGB
(img,h,g, 1
);'];
pvm_initsend
(RAW);
pvme_pack
(img,h,g,cmd);
pvm_send
(tids(3),TAG);
record('Send');
else,
record('Send'); trBLUE = dwt2ParRGB
(img,h,g, 0
);
end
#####%%%%%%% Recibir ahora que habrán acabado %%%%%%%%%%%%%%%%%
pvm_recv
(tids(1),TAG); pvm_unpack;
trRED =transform;
pvm_recv
(tids(2),TAG); pvm_unpack;
trGREEN=transform;
if NUMSLV==3
pvm_recv
(tids(3),TAG); pvm_unpack;
trBLUE =transform;
end,
record('Recv');
%%%%%%%%%%%%% Gráficas %%%%%%%%%%%%%
19
19TOOLBOX PVM
Algoritmo Paralelofunction transform=dwt2ParRGB(img,h,gd, slvflg )
% señal img, app. h, det g, slvflg==0/1 indica si es esclavo
% img es un bit plane (R/G/B)
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Preallocation tiempos/flops en ordenadores esclavos %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if slvflg
global tm ix msg
tm=zeros(50,6); ix=0; msg={};
record('');
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Descomposición 3 niveles %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
[a1 dh1 dv1 dd1]= mydwt2
(img,h,gd,'spec');
record('D1');
[a2 dh2 dv2 dd2]= mydwt2
( a1,h,gd,'spec');
record('D2');
[a3 dh3 dv3 dd3]= mydwt2
( a2,h,gd,'spec');
record('D3');
transform=[[[zeros(size(a3)) dh3;...
dv3 dd3] dh2;...
dv2 dd2] dh1;...
dv1 dd1];
M = mean(transform(:));
st = std (transform(:));
% normalizar transf 0..1
transform = (transform-M)/(2*st) + 0.5; % porque si no no se ve
transform=transform*mean(a3(:));
% según peso canal RGB
[x,y] = size(a3); transform(1:x,1:y)=a3;% Pegarle aproximación
transform(transform< 0)= 0;
% corregir detallillos
transform(transform>255)=255;
% porque imshow protesta
transform=uint8(transform);
record('math');
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Esclavos: dejar enviado resultado %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if slvflg
TAG=7; RAW=1; pvm_initsend(RAW);
pvme_pack
(transform); pvm_send
(pvm_parent,TAG);
recgraph, title([hostname ': Segundos para Users99ParRGB'])
pvm_exit;
end dw
t2ParRG
B.m
22
22TOOLBOX PVM
Conclusiones
PVMTB: herramienta didáctica• diseño sistemático• interfaz completa (incluso hooks)• respeta prototipo llamadas PVM
PVMTB: herramienta de investigación• prototipado rápido bajo MATLAB
• soporta todos los tipos de datos MATLAB
• aprovechamiento paralelismo PVM• también entorno alto nivel disponible (MM-TB)