Home > SaliencyToolbox > estimateShape.m

estimateShape

PURPOSE ^

estimateShape - estimates the shape of the attended proto-object region.

SYNOPSIS ^

function shapeData = estimateShape(salmap,saliencyData,winner,params)

DESCRIPTION ^

 estimateShape - estimates the shape of the attended proto-object region.

 shapeData = estimateShape(salmap,saliencyData,winner,saliencyParams)
    Estimates the shape of the attended proto-object region from the saliencyData:

    salmap: the saliency map as returned by makeSaliencyMap.
    saliencyData: the saliencyData as returned by makeSaliencyMap.
    winner: the winning location in saliency map coordinates.
    saliencyParams: the necesary parameters.

    shapeData: structure containing information about the shape of
    the attended regions, with the following fields:
       origImage: the Image structure for the source image.
          winner: the winning location in saliency map coordinates.
      winningMap: the map for the most salient feature at the winner location.
         iorMask: the mask used for shape-based inhibition of return.
       binaryMap: a binary map of the attended region.
    segmentedMap: the winning map segmented by the binary map.
        shapeMap: a smoothed version of segmentedMap.
            date: the time and date of the creation of this structure.

    If finding an appropriate map for segmentation failed, an empty
    shapeData structure is returned.

 The possible params.shapeModes for shape estimation are:
     'None': no shape processing.
     'shapeSM': use the saliency map.
     'shapeCM': use the conspicuity map with the largest contribution
                to the saliency map at the attended location.
     'shapeFM': use the feature map with the largest contribution
                to that conspicuity map.
    'shapePyr': use the pyramid level (center or surround level) with
                the largest ontribution to that feature map.

 For details of this method see:
      Walther, D., and Koch, C. (2006). Modeling attention to salient 
      proto-objects. Neural Networks 19, pp. 1395-1407.

 See also makeSaliencyMap, evolveWTA, dataStructures, runSaliency,
          applyIOR, shapeIOR, defaultSaliencyParams.

CROSS-REFERENCE INFORMATION ^

This function calls: This function is called by:

SOURCE CODE ^

0001 % estimateShape - estimates the shape of the attended proto-object region.
0002 %
0003 % shapeData = estimateShape(salmap,saliencyData,winner,saliencyParams)
0004 %    Estimates the shape of the attended proto-object region from the saliencyData:
0005 %
0006 %    salmap: the saliency map as returned by makeSaliencyMap.
0007 %    saliencyData: the saliencyData as returned by makeSaliencyMap.
0008 %    winner: the winning location in saliency map coordinates.
0009 %    saliencyParams: the necesary parameters.
0010 %
0011 %    shapeData: structure containing information about the shape of
0012 %    the attended regions, with the following fields:
0013 %       origImage: the Image structure for the source image.
0014 %          winner: the winning location in saliency map coordinates.
0015 %      winningMap: the map for the most salient feature at the winner location.
0016 %         iorMask: the mask used for shape-based inhibition of return.
0017 %       binaryMap: a binary map of the attended region.
0018 %    segmentedMap: the winning map segmented by the binary map.
0019 %        shapeMap: a smoothed version of segmentedMap.
0020 %            date: the time and date of the creation of this structure.
0021 %
0022 %    If finding an appropriate map for segmentation failed, an empty
0023 %    shapeData structure is returned.
0024 %
0025 % The possible params.shapeModes for shape estimation are:
0026 %     'None': no shape processing.
0027 %     'shapeSM': use the saliency map.
0028 %     'shapeCM': use the conspicuity map with the largest contribution
0029 %                to the saliency map at the attended location.
0030 %     'shapeFM': use the feature map with the largest contribution
0031 %                to that conspicuity map.
0032 %    'shapePyr': use the pyramid level (center or surround level) with
0033 %                the largest ontribution to that feature map.
0034 %
0035 % For details of this method see:
0036 %      Walther, D., and Koch, C. (2006). Modeling attention to salient
0037 %      proto-objects. Neural Networks 19, pp. 1395-1407.
0038 %
0039 % See also makeSaliencyMap, evolveWTA, dataStructures, runSaliency,
0040 %          applyIOR, shapeIOR, defaultSaliencyParams.
0041 
0042 % This file is part of the SaliencyToolbox - Copyright (C) 2006-2013
0043 % by Dirk B. Walther and the California Institute of Technology.
0044 % See the enclosed LICENSE.TXT document for the license agreement.
0045 % More information about this project is available at:
0046 % http://www.saliencytoolbox.net
0047 
0048 function shapeData = estimateShape(salmap,saliencyData,winner,params)
0049 
0050 if strcmpi(params.shapeMode,'None')
0051   shapeData = [];
0052   return
0053 end
0054 
0055 shapeData.origImage = salmap.origImage;
0056 shapeData.winner = winner;
0057 
0058 tmp1 = cat(1,saliencyData.CM);
0059 tmp2 = cat(3,tmp1.data);
0060 [mx,CMidx] = max(tmp2(winner(1),winner(2),:));
0061 
0062 % first default: winning map is saliency map
0063 winMap = salmap;
0064 winPos = {winner};
0065 
0066 % need to go deeper into the maps?
0067 if ((mx > 0) & ~strcmp(params.shapeMode,'shapeSM'))
0068   
0069   % find the biggest contributing conspicuity map
0070   CMidx = CMidx(randi(length(CMidx)));
0071   tmp1 = cat(1,saliencyData(CMidx).FM(:));
0072   tmp2 = cat(3,tmp1.data);
0073   [mx,FMidx] = max(tmp2(winner(1),winner(2),:));
0074   
0075   % found our winning conspicuity map
0076   winMap(end+1) = saliencyData(CMidx).CM;
0077   winPos{end+1} = winner;
0078 
0079   % need to go deeper?
0080   if ((mx > 0) & ~strcmp(params.shapeMode,'shapeCM'))
0081     
0082     % our next bet are the feature maps that contribute
0083     % to the wining conspicuity map
0084     FMidx = FMidx(randi(length(FMidx)));
0085     winMap(end+1) = saliencyData(CMidx).FM(FMidx);
0086     winPos{end+1} = winner;
0087     
0088     % need to go deeper still?
0089     if (strcmp(params.shapeMode,'shapePyr'))
0090       
0091       % now we compare the contributing center and surround maps
0092       cen = saliencyData(CMidx).csLevels(FMidx).centerLevel;
0093       sur = saliencyData(CMidx).csLevels(FMidx).surroundLevel;
0094       [pyrIdx,tmp] = ind2sub(size(saliencyData(CMidx).FM),FMidx);
0095       cenMap = saliencyData(CMidx).pyr(pyrIdx).levels(cen);
0096       surMap = saliencyData(CMidx).pyr(pyrIdx).levels(sur);
0097       
0098       % extract the values of the cen and sur maps at the winner location
0099       wCM = size(saliencyData(CMidx).CM.data,2);
0100       cenWin = round(winner/wCM * size(cenMap.data,2));
0101       surWin = round(winner/wCM * size(surMap.data,2));
0102       cenWin = max(min(cenWin,size(cenMap.data)),[1 1]);
0103       surWin = max(min(surWin,size(surMap.data)),[1 1]);
0104       cenVal = cenMap.data(cenWin(1),cenWin(2));
0105       surVal = surMap.data(surWin(1),surWin(2));
0106       
0107       % compare center and surround and store the winner
0108       if (abs(cenVal) > abs(surVal) & (min(size(cenMap.data)) > 7))
0109         winMap(end+1) = cenMap;
0110         winVal = cenVal;
0111         winPos{end+1} = cenWin;
0112       else
0113         winMap(end+1) = surMap;
0114         winVal = surVal;
0115         winPos{end+1} = surWin;
0116       end
0117       
0118       % renormalize the cen or sur map values for better segmentation
0119       winMap(end).data = (1 - abs(winMap(end).data - winVal) / winVal).^2;
0120     end    
0121   end
0122 end
0123 
0124 % now we have extracted all the maps we need
0125 debugMsg(winMap(end).label)
0126 debugMsg(sprintf('Value at winning location: %g',...
0127          winMap(end).data(winPos{end}(1),winPos{end}(2))));
0128 gotMap = 0;
0129 
0130 % let's see who behaves nicely for segmentation
0131 for idx = length(winMap):-1:1
0132   switch params.segmentComputeType
0133     case 'Fast'
0134       binMap = fastSegmentMap(winMap(idx),winPos{idx});
0135     case 'LTU'
0136       binMap = LTUsegmentMap(winMap(idx),winPos{idx});
0137     otherwise
0138       error(['Unknown segmentComputeType: ' params.segmentComputeType]);
0139   end
0140     
0141   % check that we actually segmented something, but not too big (< 10%)
0142   areaRatio = sum(binMap.data(:)) / prod(size(binMap.data));
0143   if ((areaRatio > 0) & (areaRatio < 0.1))
0144     
0145     % this guy looks good - let's keep him!
0146     shapeData.winningMap = winMap(idx);
0147     shapeData.winner = winPos{idx};
0148 
0149     % for the IOR mask, we don't want to smooth the shape
0150     shapeData.iorMask = binMap;
0151     shapeData.iorMask.data = imdilate(shapeData.iorMask.data,strel('disk',2));
0152     shapeData.iorMask.label = 'IOR mask';
0153     
0154     % for the binary map, erode the shape a bit
0155     binMap.label = 'binary shape map';
0156     se = [[0 0 1 0 0];[0 1 1 1 0];[1 1 1 1 1];[0 1 1 1 0];[0 0 1 0 0]];
0157     tmp = imclose(imopen(binMap.data,se),se);
0158     newMap = [];
0159     if (tmp(winPos{idx}(1),winPos{idx}(2)) > 0)
0160       if (sum(tmp(:)) > 0)
0161         newMap = tmp;
0162       end
0163     else
0164       se = [[0 1 0];[1 1 1];[0 1 0]];
0165       tmp = imclose(imopen(binMap.data,se),se);
0166       if ((tmp(winPos{idx}(1),winPos{idx}(2)) > 0) && (sum(tmp(:)) > 0))
0167         newMap = tmp;
0168       end
0169     end
0170     if ~isempty(newMap)
0171       lab = bwlabel(newMap,4);
0172       binMap.data = double(lab == lab(winPos{idx}(1),winPos{idx}(2)));
0173     end
0174     shapeData.binaryMap = binMap;
0175     gotMap = 1;
0176     break;
0177   end
0178 end
0179 
0180 % huh - no success in segmentation? Just return empty then
0181 if (~gotMap)
0182   shapeData = [];
0183   return;
0184 end
0185 
0186 % Hurray, we have a nicely segmented map - let's compute a few more things
0187 
0188 % The segmented map is just winning map * binary map
0189 shapeData.segmentedMap.origImage = shapeData.winningMap.origImage;
0190 shapeData.segmentedMap.label = 'segmented shape map';
0191 shapeData.segmentedMap.data = shapeData.winningMap.data .* shapeData.binaryMap.data;
0192 shapeData.segmentedMap.date = clock;
0193 shapeData.segmentedMap.parameters = shapeData.winningMap.parameters;
0194 
0195 % The shape map is a smoothed version of the binary map
0196 shapeData.shapeMap.origImage = shapeData.winningMap.origImage;
0197 shapeData.shapeMap.label = [shapeData.binaryMap.label ' - rescaled'];
0198 tmp = imresize(shapeData.binaryMap.data,...
0199                shapeData.shapeMap.origImage.size(1:2),'nearest');
0200 kernel = gaussian(0,15,15);
0201 tmp = normalizeImage(sepConv2PreserveEnergy(kernel,kernel,tmp),[0,3]);
0202 shapeData.shapeMap.data = clamp(tmp,0,1);
0203 
0204 shapeData.shapeMap.date = clock;
0205 shapeData.shapeMap.parameters = shapeData.winningMap.parameters;
0206 
0207 shapeData.date = clock;

Generated on Thu 18-Jul-2013 06:10:46 by m2html © 2005