iFDO content section#

The iFDO content is one section of the complete iFDO file. See its description to learn about its file formats, parts and sections. All fields in this section are optional!

Motivation#

Image data is inherently unstructured and obtaining a glimpse of its content is hard to achieve for humans as well as machines. The iFDOs content fields are a mechanism to encode the content of image data by means of visual, textual or other data proxies (annotations, previews, descriptions, categorisations, etc.). These can take various forms as described below. Simple examples of visual proxies are thumbnails for images or the average intensity along a video.

File format#

All iFDO content fields shall be stored alongside the core metadata in your iFDO file! It does not take up a specific section of the file, rather the values are intermixed into the image-set-header and image-set-items section!

iFDO content fields#

Further domain-specific iFDO content fields#

Field	Comment
image-entropy	Information content of an image / frame according to Shannon entropy.
image-particle-count	Counts of single particles/objects in an image / frame
image-average-color	The average colour for each image / frame and the `n` channels of an image (e.g. 3 for RGB)
image-mpeg7-colorlayout	An nD feature vector per image / frame of varying dimensionality according to the chosen descriptor settings.
image-mpeg7-colorstatistic	An nD feature vector per image / frame of varying dimensionality according to the chosen descriptor settings.
image-mpeg7-colorstructure	An nD feature vector per image / frame of varying dimensionality according to the chosen descriptor settings.
image-mpeg7-dominantcolor	An nD feature vector per image / frame of varying dimensionality according to the chosen descriptor settings.
image-mpeg7-edgehistogram	An nD feature vector per image / frame of varying dimensionality according to the chosen descriptor settings.
image-mpeg7-homogeneoustexture	An nD feature vector per image / frame of varying dimensionality according to the chosen descriptor settings.
image-mpeg7-scalablecolor	An nD feature vector per image / frame of varying dimensionality according to the chosen descriptor settings.

iFDO content fields for annotations#

Field	Format / Values / Unit	Comment
image-annotation-labels	[{id:`<LID>`,name:string,info:`<description>`},…]	All the labels used in the image-annotations. Specified by an id (e.g. AphiaID), a human-readable name and an optional description.
image-annotation-creators	[{id:`<ORCID/UUID>`,name:string,type:string (expert, non-expert, AI)},…]	All the annotators that created image-annotations. Specified by an id (e.g. ORCID), a human-readable name and an optional type specifying the annotator’s expertise.
image-annotations	[{coordinates:…,labels:…,shape:,…,frames:…},`<ANNOTATION-2>`,…]	This field is mighty powerful! It stores all annotations as a list of dictionaries of 3-4 fields: shape, coordinates, labels and (optional) frames. See further explanations below. The list of labels specifies the IDs or names of objects and annotators and their confidence. These should be specified in an `image-annotation-labels` and `image-annotation-creators` field (see above) to provide more information on the values used in these fields.
image-annotations:shape	single-pixel, polyline, polygon, circle, rectangle, ellipse, whole-image	The annotation shape is specified by a keyword (allowed values: see previous column).
image-annotations:coordinates	[[p1.x,p1.y,p2x,p2.y,…]..]	The pixel coordinates of one annotation. The top-left corner of an image is the (0,0) coordinate. The x-axis is the horizontal axis. Pixel coordinates may be fractional. Coordinates can be given as a list of lists in case of video annotations that change coordinates over time (see `image-annotation:frames` below). Otherwise it is just a list of floats. The required number of pixel coordinates is defined by the shape (0 for whole-image, 2 for single-pixel, 3 for circle, 8 for ellipse/rectangle, 4 or more for polyline, 8 or more for polygon). The third coordinate value of a circle defines the radius. The first and last coordinates of a polygon must be equal.
image-annotations:labels	[{label: `<LID>`, annotator: `<ORCID/UUID>`, created-at: `<datetime>`, confidence: `<float>`},…]	A list of labels and annotators for one annotation. Use label IDs and annotator IDs here. The optional confidence in an annotation can be given as a float between 0 (100% uncertain) and 1 (100% certain). The confidence can be given independently for each label. Labels are independent of frames. The `created-at` field contains an ISO8601 string like `2011-10-05T14:48:00.000Z`
image-annotations:frames	[f1,…]	(only required for video annotations) Frame times (in seconds from the beginning of a video) of a video annotation. Each frame time is linked to one entry in `image-annotations:coordinates` at the same position in the list, which specifies the current coordinates of the annotation at that frame.