: Henning Müller, Paul Clough, Thomas Deselaers, Barbara Caputo
: Henning Müller, Paul Clough, Thomas Deselaers, Barbara Caputo
: ImageCLEF Experimental Evaluation in Visual Information Retrieval
: Springer-Verlag
: 9783642151811
: 1
: CHF 87.10
:
: Informatik
: English
: 544
: Wasserzeichen/DRM
: PC/MAC/eReader/Tablet
: PDF
The pervasive creation and consumption of content, especially visual content, is ingrained into our modern world. We're constantly consuming visual media content, in printed form and in digital form, in work and in leisure pursuits. Like our cave- man forefathers, we use pictures to record things which are of importance to us as memory cues for the future, but nowadays we also use pictures and images to document processes; we use them in engineering, in art, in science, in medicine, in entertainment and we also use images in advertising. Moreover, when images are in digital format, either scanned from an analogue format or more often than not born digital, we can use the power of our computing and networking to exploit images to great effect. Most of the technical problems associated with creating, compressing, storing, transmitting, rendering and protecting image data are already solved. We use - cepted standards and have tremendous infrastructure and the only outstanding ch- lenges, apart from managing the scale issues associated with growth, are to do with locating images. That involves analysing them to determine their content, clas- fying them into related groupings, and searching for images. To overcome these challenges we currently rely on image metadata, the description of the images, - ther captured automatically at creation time or manually added afterwards.
Foreword6
Preface8
Acknowledgements10
Contents12
List of Contributors24
Introduction28
Seven Years of Image Retrieval Evaluation30
Introduction30
Evaluation of IR Systems32
IR Test Collections33
Cross--Language Evaluation Forum (CLEF)36
ImageCLEF36
Aim and Objectives36
Tasks and Participants38
Data sets39
Contributions39
Organisational Challenges41
Conclusions42
References43
Data Sets Created in ImageCLEF46
Introduction46
Collection Creation47
Requirements and Specification48
Collection Overview50
Image Collections for Photographic Retrieval51
The St. Andrews Collection of Historic Photographs51
The IAPR TC--12 Database53
The Belga News Agency Photographic Collection55
Image Collections for Medical Retrieval56
The ImageCLEFmed Teaching Files57
The RSNA Database61
Automatic Image Annotation and Object Recognition62
The IRMA Database62
The LookThatUp (LTU) Data set63
The PASCAL Object Recognition Database64
The MIR Flickr Image Data Set65
Image Collections in Other Tasks65
The INEX MM Wikipedia Collection66
The KTH--IDOL2 Database67
Conclusions68
References69
Creating Realistic Topics for Image Retrieval Evaluation71
Introduction71
User Models and Information Sources74
Machine--Oriented Evaluation74
User Models75
Information Sources for Topic Creation76
Concrete Examples for Generated Visual Topics in Several Domains79
Photographic Retrieval79
Medical Retrieval80
The Influence of Topics on the Results of Evaluation81
Classifying Topics Into Categories82
Links Between Topics and the Relevance Judgments83
What Can Be Evaluated and What Can Not?83
Conclusions84
References85
Relevance Judgments for Image Retrieval Evaluation88
Introduction88
Overview of Relevance Judgments in Information Retrieval89
Test Collections89
Relevance Judgments90
Relevance Judging for the ImageCLEF Medical Retrieval Task97
Topics and Collection97
Judges98
Relevance Judgment Systems and the Process of Judging99
Conclusions and Future Work103
References104
Performance Measures Used in Image Information Retrieval106
Evaluation Measures Used in ImageCLEF106
Measures for Retrieval107
Measuring at Fixed Recall108
Measuring at Fixed Rank110
Measures for Diversity112
Collating Two Measures Into One113
Miscellaneous Measures113
Considering Multiple Measures114
Measures for Image Annotation and Concept Detection115
Use of Measures in ImageCLEF116
Conclusions117
References117
Fusion Techniques for Combining Textual and Visual Information Retrieval120
Introduction120
Information Fusion and Orthogonality122
Methods123
Results123
Early Fusion Approaches123
Late Fusion Approaches124
Inter--media Feedback with Query Expansion129
Other Approaches130
Overview of the Methods from 2004--2009130
Justification for the Approaches and Generally Known Problems130
Conclusions133
References133
Track Reports140
Interactive Image Retrieval142
Interactive Studies in Information Retrieval142
iCLEF Experiments on Interactive Image Retrieval144
iCLEF Image Retrieval Experiments: The Latin Square Phase145
iCLEF Experiments with Flickr148
The Target Collection: Flickr149
Annotations149
The Task150
Experiments152
Task Space, Technology and Research Questions159
Use Cases for Interactive Image Retrieval159
Challenges: Technology and Interaction160
References162
Photographic Image Retrieval165
Introduction165
Ad hoc Retrieval of Historic Photographs: ImageCLEF 2003--2005166
Test Collection and Distribution167
Query Topics168
Relevance Judgments and Performance Measures171
Results and Analysis171
Ad hoc Retrieval of Generic Photographs: ImageCLEFphoto 2006-2007173
Test Collection and Distribution174
Query Topics175
Relevance Judgments and Performance Measures176
Results and Analysis177
Visual Sub--task178
Ad hoc Retrieval and Result Diversity: ImageCLEFphoto 2008--2009179
Test Collection and Distribution179
Query Topics180
Relevance Judgments and Performance Measures182
Results and Analysis182
Conclusion and Future Prospects184
References185
The Wikipedia Image Retrieval Task187
Introduction187