Color names are linguistic labels that humans attach to colors. We use them routinely and seemingly without effort to describe the world around us. They have been primarily studied in the fields of visual psychology,anthropology and linguistics. Within a computer vision context color naming is the action of assigning linguistic color labels to image pixels. We have investigated the possibility to automaticly learn color names from Google Image search.
Publications on this subject:
Although color is commonly experienced as an indispensable quality in describing the world around us,state-of-the art local feature-based representations are mostly based on shape description,and ignore color information. The description of color is hampered by the largeamount of variations which causes the measured color values to vary significantly. A change in illuminant color, viewpoint, and acquisition material, all influence the color values of the scene. Wehave investigated extending local shape description with color descriptors which are robust with respect to photometric varations.
Publications on this subject:
Most of my thesis research was on the subject of color feature detection and photometric invariant feature detection. A brief description and some matlab code can be foundhere.
Publications on this subject:
Color constancy is the ability to measure colors of objects independent of the color of the light source.The Grey-World assumption, which is at the basis of a well-known color constancy method, assumes thatthe average reflectance of surfaces in the world is achromatic. In our research we investigated the possibility of extending thissimple algorithm to the higher order derivative structure of images. We propose the Grey-Edge hypothesis, which assumes that the average edge difference in a scene is achromatic.Based on this hypothesis, we derive an algorithm for illuminant color estimation.The method is easily combined together with Grey-World, max-RGB and Shades of Grey into a single framework for color constancybased on low level image features (matlabcode is available).
Publications on this subject:
This higher order color constancy theory is further developed in the recent work ofArjan Gijsenij .
Most color constancy methods apply a bottom-up approach. Based on some image statistic an estimation of the illuminant color is computed. In recent work we have investigated the use ofusing high-level visual information for color constancy.We evaluate a number of illuminant color hypotheses on the likelihoodof its semantic content: is the grass green, the road grey, and the sky blue, in correspondence withour prior knowledge of the world. Based on this semantic likelihood we pick the illuminant which results in the most likely image. We use two approaches to obtain the illuminant hypotheses, one of which is the use of existing color constancy methods, such as Grey-World, and Max-RGB. Furthermore, we propose to casttop-down color constancy hypotheses, based on a semantic understanding of the image, and prior knowledge of the colors of the recognized classes.
2. J. van de Weijer, C. Schmid, J.J. Verbeek, Using High-Level Visual Information for Color Constancy , Proc. ICCV, Rio de Janeiro, Bresil, 2007.
For our research on color names we have collected two data sets. To automatically learn color names we collected a set of 100 images for each of the eleven basic color terms: black, blue, brown, grey, green, orange, pink, purple, red, white, and yellow. The images are collected with Google Image by using the color term together with the term "color", so for red the query in Google Image is "red+color".
A tar-file containing 1100 color name labelled images: google_colors.tar
To evaluate color name mappings we have collected a set containing real-world objects accompanied by a color name. The data set contains images collected from EBAY auction site (www.ebay.com). The set contains four classes: cars, shoes, dresses, and pottery. Each class contains 10 images for each of the eleven basic color terms. The color names were assigned to the images by EBAY users.For each image we have hand-segmented the object areas which correspond to the color name
A tar-file containing the ebay images: ebay_data.tar
The data set has been used in the following publication:
To test image descriptions with respect to variations of image blur we have collected a data set of 20 image pairs with variations in blur. The changes in blur are caused by relative motion between the camera and the object, and changes in focus of the camera. The images were captured by Matthijs Douze.
Here are some more examples: Blur Image Data
A tar-file containing the 20 image pairs: blur_data.tar
The data set has been used in the following publication:
This data set contains images from 7 soccer teams taken from the web, containing 40 images per class,divided into 25 training and 15 testing images per class. Although, players of other teams were allowed to appear in theimages, no players being a member of the other classes in the database were allowed.
A tar-file containing the 280 image is available at: soccer_data.tar
The data set has been used in the following publication:
Not all images were depicted. They can be downloaded here.
matlab code
Here some implementations in Matlab code and some general color image processing functions.
Color Feature Description: matlab and C code for discriminative color descriptor, color naming (2 versions) and the hue and opponent color descriptor.
Color Feature Description (old) : matlab code for the hue and opponent color descriptor.
Color Constancy: matlab code for weighted Grey-Edge color constancy algorithm.
Color Attention : example code in matlab to implement color attention.
Object Recoloring : matlab code for object recoloring including a gui.
Color Feature Detection I : some color image processing functions, including color edge detection, photometric invariant, color Canny edge detection, Harris point detection, and color boosting.
Color Feature Detection II : implementations of color Harris, color Laplacian feature detection.
Color Constancy: matlab code for edge-based color constancy. The file also includes implementations of Grey-World, max-RGB and Shades of Grey.
Top-Down Color Constancy : matlab code for computation of color constancy based on the semantic lieklihood of the image (ICCV 2007).
Color Naming: matlab code and color name assignment data for color naming.
Color Naming : a data set of ebay-images labelled with color names useful for evaluation of color name algorithms.
Some data used in my research can be found here .
On this page Matlab code for some of my research and some general color image processing functions are available. It has not been optimized for speed, so feel free to adapt.
New website !!:
In 2014 I started the Learning and Machine Perception (LAMP) group . See my new website for the latest publications and projects.
These papers are made available for personal use only, subject to author's and publisher's copyright.
An overview of my papers is also provided at Google scholar.
Journal Publications
, IEEE Transaction in Image Processing (TIP), vol 24(3):1153-1163, 2015. ( project page)
Accurate stereo matching by two-step energy minimization, Pattern Recognition Letters (PRL), vol 51(1):16-22, January 2015.
Compact Color-Texture Description for Texture Classification, IEEE Transaction in Image Processing (TIP), vol 23(8):3633-3645, August 2014.
Semantic Pyramids for Gender and Action Recognition, Machine and Vision Application (MVAP), 25(6):1385-1397,2014. ( project page)
Painting-91: A Large Scale Database for Computational Painting Categorization, IEEE Transaction in Image Processing (TIP), vol 23(1):83-95, january 2014. ( project page)
Multi-Illuminant Estimation with Conditional Random Fields, International Journal in Computer Vison (IJCV), 105(3):205-221, 2013. ( project page)
Coloring Action Recognition in Still Images, Pattern Recognition (PR), vol. 45(4): 1627-1636, 2012.( project page+code)
Discriminative Compact Pyramids for Object and Scene Recognition, International Journal of Computer Vision (IJCV), vol 98(1), 49-64, 2012.( project page+code)
Modulating Shape Features by Color Attention for Object Recognition, IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) , vol. 34(5):918-929, 2012.( code)
Improving Color Constancy by Photometric Edge Weighting, International Journal of Computer Vision (IJCV), vol. 96(1), 83-102, 2012.
Harmony Potentials: Fusing Global and Local Scale for Semantic Image SegmentationIEEE Transaction in Image Processing (TIP), vol. 20(9): 2475-2489, 2011.( project page)
Computational Color Constancy; Survey and Experiments ,IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) , vol. 33(5): 917-930, May 2011. ( project page)
, Describing Reflectances for Colour Segmentation Robust to Shadows, Highlights, and Textures,, Journal of the Optical Society of America A (JOSA), vol. 27(3):1-20, march 2010.
, International Journal of Computer Vision (IJCV), vol. 86(2-3): 140-151, january 2010.
IEEE Transaction in Image Processing (TIP), vol 18 (7):1512-1524, July 2009.( + matlab code)( + data )
IEEE Trans. Image Processing (TIP), vol. 16 (9): 2207-2214, September 2007. ( + matlab code)( + database-image-list )
IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) , vol. 28 (1): 150-156, January 2006. ( + matlab code )
IEEE Trans. Image Processing (TIP), vol. 15 (1): 118-127, January 2006. ( + matlab code )
Int. J. Computer Vision (IJCV). 64(2/3): 143-155, September 2005.
IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), vol. 27 (4): 1520-1526, April 2005. ( + matlab code )
IEEE Trans. Image Processing (TIP), 12(8): 938-943, August 2003. ( + matlab code )
Curvature estimation in oriented patterns using curvilinear models applied to gradient vector fields , IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), vol. 23(9): 1035-1042, September 2001. ( + matlab code )
,J. van de Weijer, F. Khan and M. Masana Castrillo, Interactive Visual and Semantic Image Retrieval , In A. Sappa and J. Vitria (Eds.): Multimodal Interaction in Image and Video Applications, Springer, Berlin, 2013.
, In R. Lukac and K.N. Plataniotis (Eds.): Color Image Processing: Methods and Applications, CRC Press, 2006.
, In J. Weickert, H. Hagen (Eds.): Visualization and Processing of Tensor Fields. Springer, Berlin, 2006, 17-47.
, University of Amsterdam, PhD Thesis, March 2005.
, Color Features and Local Structure in ImagesAdria Ruiz, Joost van de Weijer, Xavier Binefa, Regularized Multi-Concept MIL for weakly-supervised facial behavior categorization, Proc. (BMVC)
( project page+code)Martin Danelljan, Fahad Shahbaz Khan, Michael Felsberg, Joost van de Weijer, Adaptive color attributes for real-time visual tracking, Proc. (CVPR)
( project page+code)Carlo Gatta, Adriana Romero, Joost Van de Weijer, Unrolling loopy top-down semantic feedback in convolutional deep networks, Proc. (CVPR Workshop on Deep Vision: Deep Learning for Computer Vision)
( project page+code)Fahad Shahbaz Khan, Joost Van de Weijer, Andrew D. Bagdanov, Michael Felsberg, Scale Coding Bag-of-Words for Action Recognition, Proc. (ICPR)
Rahat Khan, Joost van de Weijer, Fahad Khan, Damien Muselet, Christophe Ducottet, Cecile Barat, Discriminative Color Descriptors, Proc. (CVPR)
( project page+code)Shida Beigpour, Marc Serra, Joost van de Weijer, Robert Benavente, Maria Vanrell, Olivier Penacchio, Dimitris Samaras, Intrinsic Image Evaluation On Synthetic Complex Scenes , Proc. Int. Conf. on Image Processing(ICIP)
( project page+data)Rahat Khan, Joost van de Weijer, Dimosthenis Karatzas, Damien Muselet, Towards multispectral data acquisition with hand-held devices , Proc. Int. Conf. on Image Processing(ICIP)
Fahad Shahbaz Khan, Joost van de Weijer, Sadiq Ali, Michael Felsberg, Evaluating the impact of color on texture recognition, Proc. Int. Conf. on Computer Analysis of Images and Patterns(CAIP)
Christophe Rigaud, Dimosthenis Karatzas, Joost Van de Weijer, Jean-Christophe Burie, Jean-Marc Ogier, An active contour model for speech balloon detection in comics, Proc. International Conference on Document Analysis and Recognition (ICDAR)
Joost van de Weijer, Fahad Khan, Fusing Color and Shape for Bag-of-Words Based Object Recognition, Invited paper at Computational Color Imaging Workshop(CCIW)
Christophe Rigaud, Dimosthenis Karatzas, Joost Van de Weijer, Jean-Christophe Burie, Jean-Marc Ogier, Automatic text localisation in scanned comic books, Proc. International Conference on Computer Vision Theory and Applications (VISAPP’13)
Fahad Shahbaz Khan, Rao Muhammad Anwer, Joost van de Weijer, Andrew Bagdanov, Maria Vanrell, Antonio M. Lopez, Color Attributes for Object Detection, Proc. (CVPR)
( webpage+code)Fahad Shahbaz Khan, Joost van de Weijer, Andrew Bagdanov, Maria Vanrell, Portmanteau Vocabularies for Multi-Cue Image Representation, Proc. (NIPS)
( webpage+code)Shida Beigpour, Joost van de Weijer, Recoloring based on Intrinsic Image Estimation, Proc. (ICCV)
( webpage+code+video)Joost van de Weijer, Sida Beigpour, The Dichromatic Reflection Model: Future Research Directions and Applications, invited paper at Int. Conf. on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP)
David Rojas Vigo, Fahad Shahbaz Khan, Joost van de Weijer, Theo Gevers, The Impact of Color on Bag-of-Words based Object Recognition, Proc. ICPR, Istanbul, Turkey,
( webpage)David Rojas Vigo, Joost van de Weijer, Theo Gevers, Color Edge Saliency Boosting using Natural Image Statistics, Proc. CGIV, Joensuu, Finland
( webpage)Harmony Potentials for Joint Segmentation and Classification, Proc. CVPR, San Fransisco, USA ( webpage)
Top-Down Color Attention for Object Recognition , Proc. ICCV, Kyoto, Japan ( webpage + code)
, Physics-based Edge Evaluation for Improved Color Constancy , Proc. CVPR, Miami, USA
Proc. ECCV, Marseille, France, 2008.( webpage + code )
, Image Segmentation in the Presence of Shadows and Highlights,IS&T's European Conference on Colour in Graphics, Imaging and Vision (CGIV), 2008.
, Edge Classification for Color Constancy ,Workshop on Photometric Analysis for Computer Vision (PACV'07) in conjuncture with the ICCV 2007.
, Color Constancy by Derivative-based Gamut Mapping, Proc. ICCV, Rio de Janeiro, Brazil, 2007. ( + matlab code )
Proc. ICIP, San Antonio, USA, 2007.
Proc. CVPR, Minneapolis, Minnesota, USA, 2007. ( webpage )
Corners Detectors for Affine Invariant Salient Regions: Is Color Important?, Proc. CIVR, Phoenix, USA, 2006.
Proc. ICIP, Atlanta, USA, 2006.
, Beyond Patches Workshop, in conjunction with CVPR 2006.
Workshop on Dynamical Vision, in conjunction with ECCV 2006.
Proc. ECCV, Part II, 334-348, Graz, Austria, 2006.( webpage )
Proc. ICIP, Genua, Italy, October 2005.
Proc. CVPR, San Diego, CA, USA, 2005.
Proc. IS&T/SID's CIC, The SunBurst Resort, Scottsdale, Arizona, November 2004.
Proc. ICIP, Singapore, October 2004.
Proc. ICCV, pages 1520-1526, Nice, France, 2003. ( + matlab code )
Proc. Scale-Space, 237-254, Isle of Skye, Scotland, UK, 2003
Proc. ICPR, Quebec city, Canada, Aug 11-15, 2002.
Texture, The 2nd international workshop on texture analysis and synthesis 1 June 2002 in conjuncture with ECCV, Copenhagen. 2002.
Proc. ECCV, volume 1, pages 99-112. Springer Verlag (LNCS 2350), Copenhagen, Denmark.
Proc. CVPR, Kauai, Hawaii, USA, December 2001.
Proc. ICIP, Thessaloniki, Greece, October 2001.
Proc. BMVC, Bristol, September, 2000.
Proc. SPIE, San Jose, January, 2000.
Proc. SCIA, Kangerlussuaq, Greenland, 7-11 june, 1999.
Proc. ICPR, Brisbane, Australia, 17-20 August, 1998.
, Improved Curvature and Anisotropy Estimation for Curved Line Bundles ,
from: http://lear.inrialpes.fr/people/vandeweijer/research
http://lear.inrialpes.fr/people/vandeweijer/data
http://lear.inrialpes.fr/people/vandeweijer/blur_data/blur.html
http://www.cat.uab.cat/~joost/
http://www.cat.uab.cat/~joost/software.html
http://www.cat.uab.cat/~joost/research.html
http://www.cat.uab.cat/~joost/publications.html
http://lear.inrialpes.fr/people/vandeweijer/software
http://lear.inrialpes.fr/people/vandeweijer/pubs
http://www.cat.uab.cat/~joost/publications.html