View
14
Download
0
Category
Preview:
Citation preview
FACULDADE DE C IÊNCIAS E TECNOLOGIA DA
UNIVERSIDADE DE COIMBRA
MESTRADO INTEGRADO EM
ENGENHARIA B IOMÉDICA
Segmentation Processes
and Pattern Recognition in
Retina and Brain Imaging
Pedro Guimarães Sá Correia
2010/2011
Dissertação apresentada à Universidade de Coimbra
para cumprimento dos requisitos necessários à obtenção do
grau de Mestre em Engenharia Biomédica. Este trabalho foi
realizado sob a orientação do Professor Doutor Rui Manuel
Dias Cortesão dos Santos Bernardes e do Professor Doutor
Miguel de Sá e Sousa de Castelo-branco, na Associação para
investigação Biomédica e Inovação em Luz e Imagem e no
Instituto Biomédico de Investigação da Luz e Imagem.
Segmentation Processes
and Pattern Recognition in
Retina and Brain Imaging
FACULDADE DE C IÊNCIAS E TECNOLOGIA DA
UNIVERSIDADE DE COIMBRA
Pedro Guimarães Sá Correia
2010/2011
Aos meus pais, irmã e namorada;
Aos amigos, quer feirenses quer ‘conimbricenses’;
Aos meus orientadores;
Às ‘segmentadoras’;
A todos aqueles que directa ou indirectamente
possibilitaram este trabalho;
Obrigado.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
i
Contents
Abbreviations ....................................................................................................................... iv
Images Table ........................................................................................................................ vi
Introduction .......................................................................................................................... 1
The eye .........................................................................................................................................1
Retina ...........................................................................................................................................3
Retinal Layers ...........................................................................................................................5
Glaucoma .....................................................................................................................................7
Optical Coherence Tomography .....................................................................................................8
Time-Domain OCT ....................................................................................................................9
Spectral-Domain OCT .............................................................................................................. 10
Retinal Thickness ........................................................................................................................ 11
From Retina to the Cortex ........................................................................................................... 11
The Cerebral Cortex ..................................................................................................................... 14
Primary Visual Cortex ............................................................................................................. 14
Extrastriate Cortex.................................................................................................................. 15
Magnetic Resonance Imaging ...................................................................................................... 16
Cortical thickness ................................................................................................................... 17
Functional MRI ....................................................................................................................... 18
Retinotopic Mapping ................................................................................................................... 19
ii Pedro Guimarães Sá Correia
Methods ............................................................................................................................. 21
Retinal Thickness Measurements ................................................................................................. 21
MRI Data Acquisition ................................................................................................................... 22
Surface Model ........................................................................................................................ 23
Functional MRI data .................................................................................................................... 23
Retinotopic Mapping ................................................................................................................... 25
Stimuli ................................................................................................................................... 25
Segmentation of Visual Areas ...................................................................................................... 26
Segmentation Algorithm ......................................................................................................... 26
Algorithm Validation .............................................................................................................. 33
Visual Areas Cortical Thickness .................................................................................................... 34
Correlating the Retina to the Visual Cortex .................................................................................. 35
Interpolation .......................................................................................................................... 35
Pruning .................................................................................................................................. 37
Results ................................................................................................................................ 41
Magnetic Resonance Imaging ...................................................................................................... 42
Data Importing ............................................................................................................................ 43
Segmentation ............................................................................................................................. 44
Data Filtering.......................................................................................................................... 45
Gradient Gathering and Visual Areas Delineation .................................................................... 45
Visual Areas Classification and Final Results ............................................................................ 47
Algorithm Validation ................................................................................................................... 48
Inter-Subject Validation .......................................................................................................... 49
Intra-Subject Validation .......................................................................................................... 51
Validation Analysis ................................................................................................................. 53
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
iii
Correlating the Retina to the Visual Cortex .................................................................................. 57
Pruning .................................................................................................................................. 58
Interpolation .......................................................................................................................... 59
Discussion and Conclusion ................................................................................................... 63
References .......................................................................................................................... 67
iv Pedro Guimarães Sá Correia
Abbreviations
BOLD - Blood Oxygenation Level-Dependent
CCD – Charge Coupled Device
ELM - External Limiting Membrane
fMRI - Functional Magnetic Resonance Imaging
FOV - Field-of-View
GCL - Ganglion Cell Layer
GM - Grey Matter
ILM - Internal Limiting Membrane
INL - Inner Nuclear Layer
IPL - Inner Plexiform Layer
LGN - Lateral Geniculate Nucleus
mm – Millimeter
MPRAGE - Magnetization Prepared Rapid Acquisition Gradient Echo
MRI - Magnetic Resonance Imaging
ms – Millisecond
MT - Middle Temporal
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
v
NFL - Nerve Fiber Layer
OCT - Optical Coherence Tomography
ONL - Outer Nuclear Layer
OPL - Outer Plexiform Layer
PET - Positron Emission Tomography
PL – Photoreceptor Layer
poi – patch-of-interest
RPE - Retinal Pigment Epithelium
TE - Echo-Time
TR - Repetition-Time
VFS - Vector Field Sign
WM - White Matter
μm – Micrometer
vi Pedro Guimarães Sá Correia
Images Table
FIG. 1 – THE ANATOMY OF THE HUMAN EYE.(EDITED FROM
(WWW.ANGIOEDUPRO.COM/SHARPOINT/ANATOMY/ANATOMY_OF_THE_EYE.PNG) 3
FIG. 2 – REPRESENTATION OF THE RETINA AND ITS LAYERS. (EDITED FROM [30]) 5
FIG. 3 – SIMULATION OF GLAUCOMA PROGRESSION: PATIENT PERSPECTIVE. 7
FIG. 4 - B-SCAN OBTAINED FROM:
B) A TIME DOMAIN MACHINE; (FROM [24]) 10
A) A SPECTRAL DOMAIN MACHINE. (FROM [24]) 10
FIG. 5 – OCT IMAGE: RPE REPRESENTED IN RED AND ILM REPRESENTED IN GREEN. 11
FIG. 6 - PROJECTIONS OF RETINAL GANGLION CELLS TOWARDS THE LATERAL
GENICULATE NUCLEUS. (EDITED FROM [39]) 12
FIG. 7 - M AND P PATHWAYS. (EDITED FROM [30]) 13
FIG. 8 – VISUAL INFORMATION PATHWAY AT THE STRIATE CORTEX. (CREATED FROM TEXT IN [3]) 15
FIG. 9 – DORSAL AND VENTRAL STREAMS REPRESENTATION. (CREATED FROM TEXT IN [3]) 16
FIG. 10 – DIAGRAM ILLUSTRATING CORTICAL THICKNESS ESTIMATION BY:
A)SURFACE NORMALS AND MINIMUM DISTANCE; (CREATED FROM TEXT IN [19]) 18
B)LAPLACE’S EQUATION WITH EQUIPOTENCIAL FIELD LINES. (FROM [19]) 18
FIG. 11 – VISUAL FIELD MAPS IN THE HUMAN CORTEX. (FROM [40]) 20
FIG. 12 - POLAR ANGLE AND ECCENTRICITY STIMULI. 25
FIG. 13 – FILTERED DATA AND RESPECTIVE DIRECTIONAL GRADIENT WITH DIFFERENT KERNEL SIZES. 27
FIG. 14 – LABELLED IMAGE AND MASK. 30
FIG. 15 – CANDIDATE VISUAL AREAS. 31
FIG. 16 – INTER-REGIONAL SPACE SKELETON AND LOOSE ENDS REMOVAL. 31
FIG. 17 – ENDPOINTS REPRESENTATION 31
FIG. 18 - ALGORITHM SCHEMATICS 32
FIG. 19 – REGION BOUNDARY AND SCHEMATICS OF R AND Θ AND BOUNDARY POINTS INTERPOLATION. 33
FIG. 20 – POLAR ANGLE AND ECCENTRICITY MAPPING. 35
FIG. 21 - DELAUNAY TRIANGULATION WITH ITS CIRCUMCIRCLES REPRESENTED. 36
FIG. 22 – POLAR ANGLE AND ECCENTRICITY, BEFORE AND AFTER FILTERING,
IN V2 REGION OF THE VISUAL CORTEX. 36
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
vii
FIG. 23 – POLAR ANGLE AND ECCENTRICITY REPRESENTATIVE LINES BEFORE AND AFTER FILTERING. 37
FIG. 24 – BRANCHPOINTS REPRESENTATION. 37
FIG. 25 – NEIGHBORHOOD. 38
FIG. 26 – CONTINUITY CRITERION. 39
FIG. 27 – CORTICAL THICKNESS MAP ON A FLATTENED SURFACE MODEL. 41
FIG. 28 – POLAR ANGLE AND ECCENTRICITY FMRI RESULTS FOR TWO DIFFERENT SUBJECTS; 42
FIG. 29 – THE SAME DATA PRESENTED IN FIG. 28 AFTER BEING IMPORTED
AND PREPROCESSED IN MATLAB. 43
FIG. 30 - POLAR ANGLE REPRESENTATIONS OF THE TWO SUBJECTS AFTER FILTERING. 44
FIG. 31 - POLAR ANGLE DIRECTIONAL GRADIENT MAPS. 45
FIG. 32 - POLAR ANGLE REPRESENTATIONS ANF OBTAINED BUNDARIES OF THE TWO SUBJECTS. 45
FIG. 33 – FINAL SEGMENTATION RESULTS FOR BOTH SUBJECTS. 46
FIG. 34 – VALIDATION DATA FOR VENTRAL V1. 53
FIG. 35 – INTER- AND INTRA-SUBJECT VALIDATION DATA FOR DORSAL REGION V2. 54
FIG. 36 – INTER- AND INTRA-SUBJECT VALIDATION DATA FOR DORSAL REGION V2. 55
FIG. 37 – INTER- AND INTRA-SUBJECT VALIDATION DATA FOR VNTRAL REGION V2. 55
FIG. 38 – MANUAL SEGMENTATION OF V1, V2 AND V3. 56
FIG. 39 – POLAR ANGLE AND ECCENTRICITY REPRESENTATIVE LINES BEFORE AND AFTER PRUNING. 57
FIG. 40 – RESULTING SCATTERED POINTS. 58
FIG. 41 – ECCENTRICITY AND POLAR ANGLE SCALE ON RESULTING SCATTERED POINTS. 58
FIG. 42 – INITIAL MAPS AND INTERPOLATION RESULTS. 59
FIG. 43 – INITIAL MAPS AND INTERPOLATION RESULTS. 60
FIG. 44 – POLAR ANGLE MAP OF SUBJECT S3 (LEFT HEMISPHERE)
AND THE MANUAL SEGMENTATIONS PROVIDED BY DIFFERENT RESEARCHERS. 61
FIG. 45 – POLAR ANGLE MAP OF SUBJECT S3 (LEFT HEMISPHERE)
AND THE MANUAL SEGMENTATIONS PROVIDED BY DIFFERENT RESEARCHERS. 62
FIG. 46 – POLAR ANGLE MAP OF SUBJECT S1 (RIGHT HEMISPHERE)
AND THE MANUAL SEGMENTATIONS PROVIDED BY DIFFERENT RESEARCHERS. 63
FIG. 47 - POLAR ANGLE MAP. 63
viii Pedro Guimarães Sá Correia
Abstract
We purposed to establish an automatic correlation between retinal and cortical
thickness.
In 2009 Christine C. Boucard et al. [6] showed that acquired long-standing retinal
pathologies, such as age-related macular degeneration and glaucoma, appear to be related to
grey matter density reduction roughly in the defect projections at the visual cortex. As
mentioned in *6+, “this indicates that long-term cortical deprivation, due to retinal lesions
acquired later in life, is associated with retinotopic-specific neuronal degeneration of visual
cortex”.
Also on that year, Gupta et al. [14] confirmed the in vivo atrophy of LGN in glaucoma
patients, “(…) consistent with ex vivo primate and human neuropathological studies (…)”.
Optical coherence tomography (OCT) allows imaging the ocular fundus in vivo and is
herewith applied as the technique of choice for measuring the retinal thickness. We resort to
the high-definition spectral domain system from Zeiss (Cirrus HD-OCT, Carl Zeiss Meditec,
Dublin, CA, USA) to compute maps of retinal thickness covering the central 20 degrees of the
human macula.
Similarly, we resort to the Siemens Trio 3T (Siemens AG, Healthcare Sector, Erlangen,
Germany) to gather MRI (Magnetic Resonance Imaging) from the human cortex and the Brain
Voyager QX® (Brain Innovation B.V., Maastricht, The Netherlands) software to derive cortical
thickness from the human visual cortex.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
ix
While with the former technique a fundus reference is made available, therefore the
thickness map of the macula can be easily located within the ocular fundus. The latter does not
provide per se a reference, thus requiring proper visual stimuli in order to establish to which
location within the ocular fundus a given visual cortex location refers to.
Each visual area contains a representation of the visual space and therefore a
representation of each retinal point. In order to achieve the intended correlation we need to
map the retina in each visual area, thus visual area segmentation is mandatory. An algorithm
was developed and validated against manual segmentation, using a total of 6 data sets from
healthy subjects.
In order to correlate the retina to the brain it is mandatory to find a continuous
transformation function that links each retinal point to the cortex and vice-versa. Interpolation
method used is based on Delaunay triangulation.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
1
Introduction
The visual system is a complex system which allows us to describe our world in terms of
color, size, texture, shape or even location and movement. It is the ultimate responsible for our
ability to see. Light that is reflected by the environment enters the eye through specialized
transparent tissue. Its energy is transduced into electrical signals by photoreceptors located at
the retina and transmitted to the brain where it still has to be processed… A beautiful and
complex process where brain and eye work together, achieving a better understanding of our
world.
The eye
The purpose of the eye is to capture and focus light into a thin layer of photoreceptors
and convert it to refined electrical signals that are transmitted to the visual cortex through the
optic disk. Its anatomy will be shortly described and is represented in figure 1.
Light beams enter the eye through the cornea, a transparent avascular structure that
provides ideal light transmission. The cornea has two main functions: to transmit light
minimizing scattering and distortion and to reflect light that can be harmful to the eye. It is
continuous with sclera which is an opaque white tissue layer that surrounds the whole eye
acting as its limiting “wall”. Sclera is able to maintain globe shape, present resistance to internal
and external forces, and serves also as an attachment for the ocular muscles. [3, 30, 31]
2 Pedro Guimarães Sá Correia
Adjacent to sclera lays a second layer of tissue, the uveal tract (or simply Uvea), which is
formed by three continuous structures: the iris, the ciliary body and, the most posterior one,
the choroid:
Before reaching the retina, the amount of light entering the eye is controlled by the iris. It is
a round structure positioned anterior to the lens and it contains sets of muscles with
different functions that can vary the size of its center aperture, the pupil; [30,31]
The ciliary body is composed by two components, a muscle component, strong smooth
muscles called ciliary muscles are connected to the lens and are able to adjust its refractive
power and, a vascular component, responsible for the production of fluid present in the
anterior chamber (aqueous humor); [30, 39]
The choroid is mainly composed by blood vessels and it plays an important role in
nourishing the retina;
Before reaching the retina light beams also pass through two fluid-filled chambers: the
anterior chamber, located between the cornea and the lens, which contains the aqueous
humor that supplies nutrients to both these structures, and the vitreous chamber, between the
lens and the retina, containing the vitreous humor that accounts for the maintenance of the
eye shape and contains phagocytic cells that are able to eliminate debris that might obstruct
light transmission.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
3
Retina
The retina is a multilayered structure that, despite its location, is still part of the central
nervous system.
The direct pathway of visual information in the retina starts at the light sensitive
photoreceptor cells which are located at the outermost layers of the retina. Light beams must
first pass through several non-sensitive elements before photons can be absorbed. However,
these elements are mainly transparent and since distortion in the final image is minimal, this
counterintuitive arrangement allows a better nourishment and maintenance of these essential
cells. Photoreceptor cells are able to convert photons into neural signals through a process
called phototransduction. [3, 8, 30, 31]
Fig. 1 – The anatomy of the human eye.
4 Pedro Guimarães Sá Correia
Two types of photoreceptor can be identified: rods and cones. Rods are more sensitive
to light but have lower spatial resolution. They are accountable for vision at lower levels of
light. Due to their lack of resolution, it is common to have problems discriminating what we see
at this conditions. At higher levels of luminance the rod system starts to saturate while cones
become more and more dominant. [8]
While rods contain a single photo-pigment, there are three types of cones that contain
different photo-pigments sensible to different wavelengths. They are referred as short, medium
and long wavelength cones or more commonly as blue, green and red cones, and through their
relative levels of activation we perceive color. [8, 30]
Action potentials formed at the photoreceptors are transmitted by the bipolar neurons
in the central layers to the ganglion cells of the inner retina. The axons of the ganglion cells exit
the eye and form the optic nerve that carries retinal information to the rest of the central
nervous system. Ganglion cells are also involved in the detection of luminance changes, i.e. how
we perceive brightness variations between adjacent regions. [3, 8, 30]
Other neurons in the retina are the horizontal and amacrine cells which modify and
integrate visual information before it leaves the eye. While horizontal cells obtain synaptic
input from photoreceptors and project laterally (parallel to retinal surface) to influence
neighbor bipolar cells and photoreceptors, amacrine cells receive their input from bipolar cells
terminals and project themselves to adjacent bipolar cells and ganglion cells. Resulting from
these and other retinal organizations, the visual signals that leave the retina to the brain
contain already preprocessed visual information that is able to highlight or discriminate
different aspects of the visual space. [30, 39]
Neuroglial cells play an important role in maintaining the normal function of the retina.
They are not neurons and therefore do not participate in the transmission or modulation of the
visual information. However, they provide support and structure, and act as defense
mechanism against injuries and infections. [31, 37]
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
5
Three main types of neuroglial cells can be distinguished in the retina: Müller cells,
essential in the maintenance of retinal structure, microglial cells, phagocytic cells which number
increases in response to tissue inflammation and/or injury, and astrocytes, which provide
support and structure to neurons. [5, 31, 37]
Retinal Layers
Based on several imaging technics, retina is considered to be organized in ten layers
(Fig. 2) which will be shortly described:
RPE - The most external layer (farther to the
center of the eye) is the retinal pigment
epithelium. It’s a layer of pigmented cells
(melanin) that is able to reduce light
backscattering. It also plays an important role
in renewing photoreceptor cells and
phagocytosing its debris, and is the outer
blood-retinal barrier. [30, 31]
PL - The photoreceptor layer contains the
outer and inner segments of photoreceptor
cells. [12, 31]
ELM - Despite its name, the external limiting
membrane (also called outer limiting
membrane) is not a real membrane. It
consists in intercellular junctions between
photoreceptor cells and Müller cells. This
RPE
PL
ONL
OPL
IN
IPL
GCL
NFL
ILM
ELM
Fig. 2 – Representation of the retina and its layers
6 Pedro Guimarães Sá Correia
layer has the ability to block passage of large molecules and thereby acts as a metabolic
barrier. [12, 30, 31]
ONL - Cell bodies of rod and cone cells lie in the outer nuclear layer. This layer is thicker in
fovea region due to cone distribution (much larger in this area). [30, 31]
OPL - In the outer plexiform layer occur the synaptic contacts between photoreceptor
terminals, bipolar cells and horizontal cells.
INL - The inner nuclear layer contain the cell bodies of horizontal, bipolar, Müller and
amacrine cells, and some displaced ganglion cells [12]. Horizontal and amacrine cells are
located closer to where their synapses occur, i.e. while horizontal cells can be found next to
the outer plexiform layer, amacrine cells are positioned nearer to the inner plexiform layer.
[31]
IPL - Bipolar cells make synaptic contacts on the dendrites of ganglion cells in the inner
plexiform layer. Amacrine cells contact with this two set of cells also in this layer. [30]
GCL - Ganglion cell layer is mainly constituted by ganglion cell bodies. However, it can also
contain some displaced amacrine and astroglial cells. [31]
NFL - In the nerve fiber layer ganglion cell axons run parallel to retinal surface. This axons
exit the eye at the optic disk to form the optic nerve.
ILM - The most internal layer (closer to the center of the eye) is the internal limiting
membrane. As its name may indicate, this membrane limits the retina, i.e. it separates the
retina from the vitreous.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
7
Glaucoma
Glaucoma is the world second most common cause for blindness and the primary cause
for irreversible visual loss. It was estimated that 130 thousand people in the United States of
America alone and 7 million worldwide were blind due to primary glaucoma. [22, 29]
Despite technological advances, glaucoma processes are still poorly comprehended.
Several factors may contribute to the development of this optic neuropathy but ultimately it
leads to damage and loss of ganglion cells and axons at the retinal nerve fiber layer and
therefore optic nerve damage that causes progressive and irreversible loss of peripheral vision
(Fig. 3). Damage is often linked to increased ocular pressure caused by insufficient drainage of
the aqueous humor. As pressure inside the eye begins to rise to unhealthy levels, it starts to
damage the retinal nerve fiber layer, either directly by applying mechanical pressure or
indirectly by obstructing the normal blood flow. [11, 30, 31, 35, 38]
Two main types of glaucoma can be distinguished: open-angle glaucoma, where
drainage channels become gradually clogged through months or even years, and the less
common closed-angle glaucoma, where drainage channels block due to narrowing of the angle
between the cornea and the iris. In closed-angle glaucoma channels blockage can be
instantaneous or gradual. [22, 29]
Fig. 3 – Simulation of glaucoma progression: patient perspective.
8 Pedro Guimarães Sá Correia
Several diagnosing and monitoring methods can be applied when dealing with
glaucoma. Imaging technics like scanning laser tomography, scanning laser polarimetry and
optical coherence tomography can provide important knowledge on disease progression by
measuring the nerve fiber layer thickness and optic disk contour. [17, 38]
Optical Coherence Tomography
Optical Coherence Tomography (OCT) has gained increasingly importance in areas such
as biomedical engineering and medicine. It provides high-resolution cross-sectional images that
allow us to see tissue microstructure, thereby becoming an important tool in diagnose and
research for several optical diseases in ophthalmology. It provides images in vivo and in situ of
the ocular fundus with resolutions one or two orders of magnitude higher than conventional
ultrasound technics.
OCT imaging method uses the interferometer principle. It can be compared to an
echography since it uses echoes to visualize through biological tissue. Rather than sound waves,
it relies on light, which provides higher spatial resolution and allows non-contact examination.
As mentioned, retinal layers are transparent to a certain range of wavelengths. However, some
of the light projected into it is back reflected by the internal microstructure of the retina. [7, 9]
In OCT, light passes through a beam splitter therefore being divided into sample and
reference beams. As the light beam is directed onto the eye, OCT machines measure the echo
time delay of the back reflected light in the internal structures, which can be used to compute
its travel distance. [7, 9, 23]
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
9
A reflected sample beam comprises multiple echoes from different structures.
Reference beams are reflected in a reference mirror that is positioned at a previously known
distance. Reflected reference and sample beams are once again combined at the beam splitter.
When the reference beam and the sample beam travel the same distance, a phenomenon
called interference occurs. Its detection allows inferring the echo time delay for each reflecting
structure.
A single depth scan is called an A-scan and it provides measurements of the
backscattering versus depth. Combining several A-scans obtained at different transverse
positions along a line, we get a representation of the backscattering along a cross-sectional
plane (B-scan). These images can also be combined to form a three-dimensional representation.
[2]
Time-Domain OCT
The first generation of commercially available systems was able to envision tissue
microstructure in the time domain, i.e. depth information was acquired as a function of
distance or time [33]. The most widely in use machine of this type is the Stratus OCT (Carl
Zeiss-Meditec, Dublin, California, USA) with an axial resolution of 10 μm and a scan velocity of
400 axial scans per second [21].
A time-domain OCT sequentially measure echo time delays. It uses a movable reference
mirror, adjusting echo time delays for the reference beam by changing the mirror position. The
interference pattern is measured by a light sensitive detector. The pre-known distance of the
mirror and the detection pattern provide the distance of the reflecting structure. [23, 24]
10 Pedro Guimarães Sá Correia
Spectral-Domain OCT
Spectral-domain OCT is also called Fourier-domain or even frequency-domain OCT. In
these systems all echo time delays are measured simultaneously by a spectrometer and a high-
speed charge coupled device (CCD), leading to increased speed when acquiring data (Fig. 4).
[15, 23, 35]
An interference spectrum is obtained which undergoes Fourier transformation making it
possible to acquire A-scan measurements and therefore remove the requirement to adjust the
reference mirror. [15, 23, 33]
Two machines are in use at AIBILI: the Cirrus high definition OCT (Carl Zeiss-Meditec,
Dublin, California, USA), with speed of 27.000 A-scans per second and 5 μm axial resolution,
and the Spectralis Heidelberg Retinal Angiography (HRA)-OCT (Heidelberg Engineering,
Heidelberg, Germany), able to get 40.000 A-scans per second with 7 μm optical and 3.5 μm
digital axial resolution.
Studies have showed that retinal thickness measurements obtained from Spectral-
Domain OCT systems are consistently superior than those obtained from Time-Domain OCT
systems. The reason may be that the lower resolution provided by time domain machines,
results in different calculations of the reference retinal structures by the system software. [15,
21]
a) Fig. 4 - B-scan obtained from: b) a time domain machine; a) a spectral domain machine.
b)
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
11
Retinal Thickness
Several retinal pathologies, such as macular edema or glaucoma, may be expressed by
changes in retinal thickness. This measurement allows inferring on the integrity of the retina,
thus being an important tool in diagnosis and monitoring of pathology progress and treatment
response. [10, 36]
Imaging technics such as slit-lamp biomicroscopy and stereo fundus photography were
used to evaluate retinal thickness. However, these technics did not provide the necessary
accuracy to track slight alterations in the degenerative processes. Since its appearance, OCT
technology has been extensively applied in this field and studies have shown the good
reproducibility of its measurements. [26]
Measuring retinal thickness by OCT
imaging depends on the identification of the
internal limiting membrane and the retinal
pigment epithelium (Fig. 5). The distance
between these two layers is regarded as the
retinal thickness.
From Retina to the Cortex
The visual pathway between the retina and the cortex starts with the ganglion cell axons
leaving the retina through a circular region in its nasal part, the optic disc, and forming the optic
nerve. No photoreceptors can be found in this region, thus giving rise to the blind spot.
Fig. 5 – OCT image: RPE represented in red and ILM represented in green.
12 Pedro Guimarães Sá Correia
Axons from ganglion cells that lie
at the nasal region of each retina cross
from one side to another at the optic
chiasm (Fig. 6), while other axons
continue in the same side. Fibers leave
the chiasm to form the optic tract that
contains information from both eyes
and extends towards several targets in the diencephalon and the midbrain. [13]
Thalamus receives information from all the sensatory systems with the exception of the
olfactory. Processing of the visual information in this structure is conveyed to the Lateral
Geniculate Nucleus (LGN). It is the major destination of the optic tract, a multilayered structure
located on the dorsolateral side of the thalamus, where retinal axons terminate (axons from
each eye terminate in different layers). [30, 37]
Two layer types can be distinguished based on cell size: magnocellular layers and
parvocellular layers which receive input from different sets of ganglion cells (Fig. 7):
Two magnocellular layers (layers 1 and 2), located at the ventral side of the LGN, are
composed by large cells that receive input from M ganglion cells. This ganglion cells have
large dendritic trees, fast conduction speed, large diameter of their axons and aren’t able to
transmit chromatic information. These properties are important to understand the role of
the magnocellular organization at the visual perception. The magnocellular system is
therefore accountable for processes that require high temporal resolution such as
perceiving fast moving objects. [13, 20, 27, 30, 37]
Fig. 6 - Projections of retinal ganglion cells towards the Lateral Geniculate Nucleus: ganglion cells that lie at the nasal region terminate at the LGN of the opposite side.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
13
Located at the dorsal side of the LGN we can find four parvocellular layers (layers 3, 4, 5 and
6) composed by smaller cells that receive input from P ganglion cells. This neurons have
smaller cell bodies and center-surround receptive fields, slower conduction speeds and are
present in larger number than M ganglion cells. Because their receptive field centers and
surrounds are selective and distinct in the class of cones they receive input from (i.e. while
the center receptive field may receive input only from red cones, its surround receptive
field receive input from a distinct but specific set of cones like blue or green), they are able
to perceive variances in wavelengths, thus conveying color information. Like as in the
magnocellular organization, P neurons properties allow us to infer on the role of the
parvocellular system: it is important in processes that require high spatial resolution such as
the identification of size, shape and color of objects. [13, 20, 27, 30]
Besides these two distinct pathways, a third one, the koniocellular pathway, has been
identified in the inter-laminar space of the LGN (Fig. 7). Its contribution to visual perception is
not still completely understood.
Fibers leaving the LGN are called optic radiations and terminate at the primary visual
cortex that also sends feedback to this structure.
Fig. 7 - M and P pathways: a) Photomicrograph of LGN showing magnocellular and parvocellular layers - R and L for right or left eye. b) and c) “Tracings of M and P
ganglion cells as seen in flat mounts of the retina after staining by the Golgi method.”
a) b) c)
14 Pedro Guimarães Sá Correia
The Cerebral Cortex
The cerebral cortex is a convoluted sheet of tissue that surrounds the rest of the brain
with an average thickness of 3 mm. Deviations can occur due to physiological or pathological
reasons. Physiological variation depends mainly on measurement spot and age. Normal non-
pathological range stands between 2 and 6 millimeters. Anatomical observations suggest that it
is divided vertically into layers, generally 6, and horizontally into columns. [1, 19, 42]
Primary Visual Cortex
The primary visual cortex, also named Brodmann area 17, striate cortex or simply V1,
can be found at the posterior limit of the brain, in the occipital lobe, surrounding the calcarine
sulcus. It is the first visual cortical area, the most important structure in the visual system and
the most systematically studied one.
Axons from LGN neurons terminate at layer 4 (subdivided into different strata) of the
striate cortex, while axons from layer 6 send feedback to this structure. LGN division into
magnocellular and parvocellular organization (M and P pathways) continues into this structure
(Fig. 8): while the magnocellular system reaches layer 4Cα that extends to layer 4B and from
there to the extrastriate cortex, axons from the parvocellular system reach to a distinct, deeper
layer 4Cβ, which in time project to the extrastriate cortex. [3, 31, 39]
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
15
The koniocellular pathway and also projections of the M and P pathways from layer 4C,
send axons to layers 2 and 3 of V1 (Fig. 8), a pathway of conversion where it has been
suggested to handle the analysis of object color. [3]
Extrastriate Cortex
Placed next to the striate cortex, the extrastriate cortex is associated with an
intermediate level of visual processing. Four distinct areas are distinguishable in this region: V2,
V3, V4 and middle temporal (MT).
Areas V2 and V3 constitute two cortical strips that surround V1. Each of these two
regions presents a discontinuous hemifield of the visual space divided along the horizontal
meridian (i.e. two quarterfields).
Fig. 8 – Visual information pathway at the striate cortex.
a) b)
16 Pedro Guimarães Sá Correia
Two independent visual information streams (Fig. 9) of the extrastriate cortex visual
processing were identified in the macaque monkey brain:
The dorsal stream starts at V1 and connects V2, V3 and MT
visual areas. Its neurons are similar to those in the
magnocellular portion of V1, and are thought to be involved in
motor coordination and on the analysis of motion. [3, 39]
On the other hand, the ventral stream is associated with V1, V2 and V3 towards the
temporal cortex. It is associated with perception of the visual space and object
identification. [3, 39]
Magnetic Resonance Imaging
In 1976 at the University of Nottingham, Sir Peter Mansfield and his team presented for
the first time images of human anatomy using magnetic resonance technology (MRI) [18]. Since
then MRI has become an essential tool in almost all fields of medicine.
When an object is placed under a strong magnetic field, the nuclei of certain atoms will
align with it, spinning at a frequency dependent of that field’s strength. A radiofrequency wave
adjusted to that frequency is then emitted, so that, when those atoms receive it, they are
removed from their alignment, and consequently release oscillatory energy by slowly realigning
with that field. [30, 32]
The emitted energy is then detected by the scanner. Different molecules emit energy at
different frequencies and the strength of the captured signal is dependent on the nuclei
number involved in the process. [30, 32]
Fig. 9 – Dorsal and Ventral streams representation.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
17
Cortical thickness
Cortical thickness is a powerful tool in longitudinal studies and diagnosis of a wide range
of neuropathologies since it may provide incisive knowledge about neurodegenerative and
neurodevelopmental patterns. In vivo cortical thickness measurements can be achieved by
using magnetic resonance images and appropriate data processing software. [1, 4, 41]
What is cortical thickness? An anatomical perspective may lie in its communal
organization: it is the length of the functional cortical columns. [25, 34]
Two surfaces are defined by finding the boundaries between gray matter (GM), white
matter (WM) and cerebrospinal fluid.
The convoluted disposition of the human cortex poses as a difficult challenge to
overcome. How to retrieve thickness values from a curved structure? The shortest path or
simply perpendicular lines to the surface are useless as shown in Figure 10. To any given point
defined at any of the surfaces, a single cortical thickness values should correspond. [1, 19, 25,
43]
One solution that is widely accepted by the scientific community was first presented by
Jones et al. [19] based on the Laplace’s equation:
⁄ ⁄
⁄ (Eq. 1)
The suggested principle is that one surface is set to the null potential and the other as a
potential of one thousand Volts, while Laplace’s equation is solved to the potentials in
between, similar to the electrostatic field. Streamlines are the lines that connect both surfaces
and are defined as orthogonal to equipotential surfaces as defined by the ruling equation. The
final outcome is represented in figure 10. Streamlines are independent from initial potential
18 Pedro Guimarães Sá Correia
values and result in smooth transitions between surfaces. Cortical thickness is therefore
defined by the length of these streamlines. [19, 43]
Functional MRI
Functional magnetic resonance imaging (fMRI) is a variant of MRI and its major source
of contrast is the blood oxygenation level-dependent (BOLD) signal.
When a brain area is activated, the consumption of oxygen in this area increases leading
to a rise in blood flow. Local blood oxygenation level at the venous portion of the circulatory
system rises, since oxygen delivered exceeds its consumption. Subsequently, the levels of
deoxygenated hemoglobin drop. [32]
Fig. 10 – Diagram illustrating cortical thickness estimation by: a) surface normals (green) and minimum distance (red);
b) Laplace’s equation with equipotencial field lines.
a)
b)
’
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
19
Unlike oxygenated, deoxygenated hemoglobin (deoxyhemoglobin) has strong
paramagnetic properties, and it locally distorts the magnetic field, i.e. it distorts the magnetic
resonance properties of hydrogen nuclei in its neighborhood. The decrease in the
concentration of deoxyhemoglobin leads to the local smoothing of the uniform magnetic field.
Magnetic resonance signal is stronger in this region. [30, 32]
As conditions modify during a functional paradigm, different regions will activate or
deactivate.
Retinotopic Mapping
In retinotopic mapping, stimuli are provided to subjects in order to assign the visual
space to a pattern of temporal activation, i.e. to link every position in the visual space to a
single delay in stimulation. This is achieved by measuring the stimulus position that causes the
largest response at each cortical coordinate. [40]
Studies in primates have led to the discovery of several areas that handle the processing
of visual information. These areas can be found at the occipital, parietal, and temporal lobes
and each comprises a map of the visual space. [30, 40]
In humans, a strong correlation was observed between visual deficit and lesions located
in the primary visual cortex (V1). Later, using positron emission tomography (PET), V1 maps
were characterized in human living brains. Only with developments in functional imaging and
advanced data-analysis methods have allowed the identification of additional visual structures
in human brains. [30, 40]
The first fMRI research application involved retinotopic mapping, as the early stages of
cortical visual processing were the main target of the involved researchers. The very first
human fMRI research study tried to demonstrate that the primary visual cortex located at the
calcarine fissure was essential to the early visual processing. [32]
20 Pedro Guimarães Sá Correia
Currently, retinotopic analysis with specific
stimuli has made possible to locate on 16 separate
maps of the visual space (Fig. 11). The segmentation of
these regions it’s of extreme help since it may provide
useful information about visual system pathologies, a
new insight into the organization of the visual cortex
and even enlighten different visual responses in
cognitive experiences. [40]
Fig. 11 – Visual field maps in the human cortex.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
21
Methods
As aforementioned, each visual area contains a representation of the visual space and
therefore a representation of each retinal point. In order to achieve the intended correlation
we need to map the retina in each visual area, thus visual area segmentation is mandatory.
Through retinotopic mapping we can link the visual space to the visual cortex. The visual
space will serve as a reference that links both ends of the correlation.
Retinal Thickness Measurements
Optical coherence tomography allows imaging the ocular fundus in vivo and is herewith
applied as the technique of choice for measuring the retinal thickness. We resort to the high-
definition spectral domain system from Zeiss (Cirrus HD-OCT, Carl Zeiss Meditec, Dublin, CA,
USA) to compute maps of retinal thickness covering the central 20 degrees of the human
macula.
The 200x200x1024 Macular Cube protocol was used for OCT data acquisition, while
retinal thickness was obtained using proprietary software.
22 Pedro Guimarães Sá Correia
MRI Data Acquisition
The acquisition of structural data was performed using a Siemens Trio 3T (Siemens AG,
Healthcare Sector, Erlangen, Germany) MRI scanner. Data was acquired using two MPRAGE
(magnetization prepared rapid acquisition gradient echo) T1-weighted sequences with voxel
size of 1 x 1 x 1 mm3.
The two nine minutes-long sessions of optimized MPRAGE sequences were averaged to
obtain a single high-resolution anatomical image therefore improving signal-to-noise ratio.
We resort to Brain Voyager QX® (Brain Innovation B.V., Maastricht, The Netherlands)
software to derive cortical thickness from the human visual cortex. Cortical thickness
measurements are computed based on the Laplace method following white and grey matter
(WM and GM) segmentation.
In order to segment WM and GM, Brain Voyager uses a region growing approach based
on voxel intensity values and therefore intensities across voxels of the same tissue should be
uniform. White matter voxels at one position might have similar intensities as grey matter
voxels at other locations, which poses a problem in this type of segmentation. Performing
inhomogeneity correction is thus mandatory.
Some of Brain Voyager automatic segmentation tools exploit anatomical knowledge in
their processes. Transforming anatomical data to Talairach coordinate system is therefore
required. Talairach coordinate system of the human brain is a unified framework that allows
localizing of brain structures in an independent manner from overall brain size and proportions:
structures of the brain that cannot be identified only by the visual interpretation of anatomical
images are defined relatively to other anatomic cerebral structures which we can be
pinpointed.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
23
Surface Model
The construction of a surface model allows for a better visualization of functional data
mainly inside the sulci. Two main types of models can be used: the inflation and the flattened
models. In both cases the process introduces distortion to the final model. We chose to work
with the flattened one because it is two dimensional thus reducing processing time in
subsequent stages.
Flattening is accomplished with Brain Voyager software following guidelines provided by
the manufacturer. Due to its strong curvature, the surface requires to be sectioned. Despite the
automatic tools provided by the software for this requirement, automatic sectioning at the
calcarine fissure is not accurate enough, thus being required to identify 5 reference points
along it. An automatic distortion correction tool provided by the software is applied in order to
minimize this problem.
Functional MRI data
Functional data was also obtained using Siemens Trio 3T scanner. Technical features
were:
TR (repetition-time) = 2000 ms
TE (echo-time) = 39ms
Interslice time = 76 ms
Voxel size = 2 x 2 x 2 mm3
FOV (field-of-view)= 256 mm x 256 mm
Slice thickness = 2 mm
Number of slices = 26
Imaging matrix = 128 x 128
24 Pedro Guimarães Sá Correia
Brain Voyager QX software was used to gather functional maps from the gathered raw
data.
Head movements in the scanner at acquisition time, diminishes fMRI data quality,
leading to hampered activation patterns.
Motion correction through Brain Voyager software allows rectification of small
movements. Three translation parameters as well as three rotation parameters are able to
describe head motion. These are iteratively assessed by evaluating in what way the volume has
to be translated or rotated so that a better alignment with a reference volume is achieved.
Another important task in preprocessing functional data is to correct slice scan time.
The problem is that the entire volume is normally not covered at once. Instead, a series of two
dimensional scans are sequentially acquired and assembled to form 3-D data. In the current
case, with a TR of 2000 ms, the last scan is acquired virtually 2 seconds after the first one.
One solution to this problem is to perform the statistical analysis at each of the whole 2-
D scans and then shift the time point to the reference scan, thereby considering that the
volume was acquired simultaneously in time, i.e. at the beginning of TR. This method requires
temporal interpolation of each scan.
In order to display the final functional data, anatomical and functional volumes need to
be aligned with each other. Brain Voyager software initially performs a scaling transformation
so that anatomical and functional data sets have the same resolution. Finally, rotation and
translation transformations allow the required alignment.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
25
Retinotopic Mapping
Visual stimuli are provided to subjects in real time by computerized imaging through a
mirror, which was used to reflect the image from a screen while the subject is undergoing the
fMRI imaging. In this process, subjects should focus on a fixation target (a yellow dot) in the
center of the image. Magnetic resonance imaging sequences are obtained simultaneously.
Stimuli
One of the most applied techniques is the
travelling-wave method or phase-encoded retinotopic
mapping. In this technique two functional paradigms
must be presented to subjects (Fig. 12):
Polar angle paradigm: consists on a
checkerboard wedge that rotates around the
fixation target;
Eccentricity paradigm: consists on a checkerboard ring that expands from the fixation
target;
Together, these two measurements specify visual field positioning in polar coordinates,
providing angle and eccentricity values. Three polar angle data sets are combined into a single
volume for improved data quality.
The checkerboard panel presented on the stimuli intents to ensure good neural
response.
a) b)
Fig. 12: Polar Angle (a) and Eccentricity (b) stimuli.
26 Pedro Guimarães Sá Correia
Segmentation of Visual Areas
Manual segmentation of the visual cortex is time consuming, labor intensive and
presents undesired variability. Automated software that maps the visual field is thus desirable.
An algorithm (Fig. 18) was developed and implemented in Matlab® (The MathWorks Inc.,
Natick, Massachusetts, United States).
Segmentation Algorithm
Data ara imported into Matlab using the BVQXtools v0.8, a Matlab
toolbox available at the BrainVoyager site. It allows reliably data exporting to
Matlab, for most of the major BrainVoyager QX's file formats.
Brain Voyager analysis produces a large amount of files, most of them
not required at the present stage, and each of these files still contains
unnecessary information. We retrieve vertex coordinates using the surface file
and polar angle and eccentricity stimulus position for each vertex from ‘smp’
files.
Additionally using BVQXtools v0.8, a threshold map of brain activation
is created. Retinotopic data is then filtered using this map so that only
relevant vertices are displayed, which significantly improves visualization and
facilitates subsequent algorithm processing phases.
Data Importing
Data Restricting
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
27
The representation is casted into an image for future analysis so that
more efficient methods could be applied, hence reducing processing time and
enhancing results. The limitations imposed by the triangular mesh are thus
avoided.
Because of its local computation gradients are sensitive to noise
leading to the need low pass filtering. A low-pass filter of [30x30] kernel was
used.
The Kernel size was decided on a try-and-error basis. In figure 13 it is
possible to visualize filter size effect in the image and in the corresponding
gradient. As kernel size increases, the boundaries become more visible.
Casting into Image
Data Filtering
b)
Kernel size: [30x30] Kernel size: [15x15]
a)
Kernel size: [5x5]
Fig. 13 – Filtered data (a) and respective directional gradient (b) with different Kernel sizes.
28 Pedro Guimarães Sá Correia
A previous study combined Polar Angle and Eccentricity information to
compute a Vector Field Sign map (VFS) [42]. In this way, the gradient of the
polar angle (∇θ) and eccentricity (∇ρ) maps are required to be computed on
the cortical surface:
)), (Eq. 2)
where ψ is the function that maps each point on the cortical surface position
in the visual field. Therefore:
∇ ∇ )), (Eq. 3)
where is normal to the surface.
This method was implemented using Matlab in a first attempt.
However, it didn’t prove itself as robust as we expected, i.e. the method was
too much sensitive to data artifacts. The achieved results presented artifacts
that turn into a challenge for the delineation of the visual areas.
Several attempts to improve achieved results were made. By using a
flattened surface model we removed the requirement of including (Eq. 3),
therefore:
∇ ∇ )). (Eq. 4)
Higher order filters were also used in order to get smoother data.
However, it didn´t seem the results were enhanced to a sufficient degree
which ultimately led us to look for alternatives.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
29
Longer acquisition time and/or using the same combination method
for eccentricity paradigms as the use of polar angle paradigms might have
improved results (three polar angle data sets where combined).
Segmentation of the visual areas V1, V2 and V3, using retinotopic data,
is only possible since the representation of the visual space between
contiguous areas changes its orientation, i.e. adjacent areas have mirror or
non-mirror visual representations. Therefore, using a flattened surface model,
polar angle is enough to determine boundaries between retinotopic visual
areas.
Attending to this information we propose that the reversals on the
orientation of polar angle representations can be easily perceived in a
directional gradient map. In this way:
), (Eq. 5)
where Fθ represents polar angle function.
A similar process is applied to eccentricity data:
), (Eq. 6)
where Fρ represents eccentricity function. The inversion in its gradient
direction allows us to identify the boundary between dorsal and ventral
regions of the visual cortex.
Two binary images are created, one containing pixels with positive
orientation and the other with negative orientation. We perform
morphological opening, erosion followed by dilation, in order to break weak
links between clusters of pixels with the same orientation. The resulting
clusters become candidate visual areas (Fig. 15).
Gradient Gathering
Dorsal and Ventral Regions Segmentation
Clusters Gathering
30 Pedro Guimarães Sá Correia
Information from each of the candidate visual areas, such as gradient
orientation (mirror or non-mirror visual representation), centroid, area,
neighbor regions and pixel position in the image, is calculated/retrieved and
stored for future application.
Neighbor regions are defined as the regions that directly contact with
the region of reference but only in the direction of the visual pathway
(posterior to anterior), i.e. V2 would be neighbor of V1, but V1 would not be
neighbor of V2.
The candidate regions are labeled and a mask is created (Fig. 14) using
a structuring element such as:
[ ] .
The mask is then applied to the labeled image as the neighbor regions are the
ones labeled with the values of the resulting non-zero pixels. This process
turned out to be helpful since, as we will see, the classification is performed
sequentially: it limits the number of regions which will go under
consideration, eliminating lateral and posterior regions.
Neighbor Information
Fig. 14 – a) Labelled image; b) Mask.
a) b)
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
31
As shown in figure 15 the
pixel clusters obtained don´t
possess a clear boundary between
them. Instead we clearly distinguish
the regions and an interregional
space. The use of the boundaries of
each singular region would go
against anatomy, since it would
mean the presence of two different boundaries between two visual areas. To
obtain the final boundaries we achieve the inter-regional space skeleton.
The resulting skeleton is shown in figure 16. It presents the wanted
boundaries in addition to loose ends needing to be pruned. The boundaries
are continuous, with no endpoints (Fig. 17). Pruning is performed by
sequentially calculating and removing endpoints until the boundaries become
clean and no endpoints can be found.
Fig. 15 – Candidate Visual Areas. Skeleton
Gathering
Pruning
Fig. 17 – The central pixel is:
a) not an endpoint; b) an endpoint.
a) b)
Loose ends Fig. 16 – Inter-regional space skeleton and loose ends
removal.
32 Pedro Guimarães Sá Correia
The next step is the classification of the candidate visual areas, and it
starts by dividing regions into dorsal or ventral. We then find ventral V1 since
it is the more robust one. First we need to know that visual areas have typical
orientation and relative positions, so candidate regions are matched against
for these features.
As it has been said before, V1 is the most posterior visual area, and by
using this anatomical feature we are able to locate this region. When tracing a
line between dorsal and ventral V1 it would not be a vertical line (Fig. 15). This
natural orientation makes ventral V1 classification less affected by lateral
artifact resulting regions. Dorsal V1 is found by a similar “neighbor finding”
process but instead of using a horizontal structuring element, a vertical one is
used.
Sequentially the algorithm finds V2 that must be a neighbor of V1 and
V3 that must a neighbor of V2. Please note that this process is divided in
dorsal and ventral streams. Some discrepancies might be found in this
process, such as small clusters that are the result from fMRI artifacts, but are
overcome by the ranking system that is able to exclude them.
A Brain Voyager ‘poi’ (patch-of-interest) file is then created. This file
contains information about each visual area such as name or vertices that
belong to that area.
Visual Areas Classification
Data Exporting
Fig. 18 Algorithm schematics
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
33
Algorithm Validation
The segmentation algorithm was validated against manual segmentation using a total of
6 data sets from healthy subjects. Inter and intra-subject boundaries variation of manual
segmentation was compared to variation values obtained between manual and automatic
segmentation.
Manual segmentation was performed by two researchers in order to gather inter-
subject variation. One of the researchers repeated the segmentation process two weeks after
to achieve intra-subject variation.
For each boundary point we calculated relative distance (r) and angle (θ) to a fixed
center of each region, therefore computing their polar coordinates (Fig. 19 a). In order to obtain
a continuous line between the points, we performed linear interpolation of r between -π<θ<=π.
The first and last points are repeated at the end and at the beginning respectively, to achieve
an accurate interpolation in the full length of the interval (Fig. 19 b).
Fig. 19 – a) Region Boundary and schematics of r and θ; b) Boundary points (blue) and interpolation (black). In red we can see the repeated points.
a) b)
θ r
r θ
θ
r
34 Pedro Guimarães Sá Correia
The average ( ) and the standard deviation ( ) of the variation of the two
segmentations was calculated using:
∑
√
∑ )
(Eq. 7 and 8)
where is the number of elements in the sample and the data.
Visual Areas Cortical Thickness
A Matlab function was developed to retrieve cortical thickness values and other statistic
measurements, such as average, standard deviation, median or standard error, from visual
areas.
It receives as input: ‘poi’ Brain Voyager files containing the visual areas, and ‘smp’ files
containing cortical thickness surface map. The output is an excel file as well as a scatter plot
graphic.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
35
Correlating the Retina to the Visual Cortex
Retinotopic mapping links each vertex in the
visual cortex to a polar angle and eccentricity stimulus
position. A total of 24x12 patches are created
mapping the visual space, therefore it is a poor
resolution technique at least when compared to data
obtained by OCT (Fig. 21).
In order to correlate the retina to the brain it is
mandatory to find a continuous transformation
function that links each retinal point to a cortical point
and vice-versa. The problem of attributing a value for
new points within a given set of scattered points is called interpolation [28].
Interpolation
Using Delaunay triangulation, interpolation of a data set which is defined by its location
) produces a surface:
) (Eq. 9)
therefore the value of any given location ) within the smallest convex set comprising
the original set of points (the convex hull) can be obtained by:
) (Eq. 10)
Fig. 20 – Polar angle and eccentricity mapping.
36 Pedro Guimarães Sá Correia
As the word suggests, a triangulation of a given set
of points is the organization of those points in triangles. In
order to achieve a ‘good’ triangulation two models can be
distinguished. One that minimizes the maximal angle, or
alternatively, that maximizes the minimal angle of all the
angles of the triangles in the triangulation. [16]
Delaunay triangulation is defined at the convex
hull of the given set of points and uses the MaxMin angle
criterion. It is built so that no point of the set is contained
in any triangle circumcircle (Fig. 22). [16]
Retrieving a scattered data set from the retinotopic data implicates finding and crossing
the representative lines obtained from each of the polar angle and eccentricity regions. They
are obtained by performing object reduction, i.e. sequential boundary pixels removal while
maintaining the Euler number. The result is a line equidistant to the region boundaries.
Fig. 21 - Delaunay triangulation with its circumcircles represented.
Fig. 22 – Polar Angle (a) and Eccentricity (b) before and after filtering in V2 region of the visual cortex.
a) b)
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
37
Data is filtered through a low pass filter (Fig. 23). By obtaining a smoother set of
representative lines we reduce pruning and therefore processing time. The improvement
achieved can be easily perceived in figure 24.
Pruning
In the process of retrieving the representative lines, redundant branches are often
created that may corrupt the final results (Fig. 24). Crossing these lines often creates duplicated
points for the same polar angle and eccentricity values.
Fig. 24 – The pixel is: a) a branchpoint; b) not a branchpoint.
Fig. 23 – Polar angle and eccentricity representative lines before (a and c) and after filtering (b and d).
a)
d) c)
38 Pedro Guimarães Sá Correia
Trying to overcome this limitation a pruning method was developed. The first step
involves labeling each branch. Starting at the line endpoints, each branch is tracked until it
reaches a branchpoint (Fig. 25). Tracking is processed by scanning the current point
neighborhood (Fig. 26) for the following one.
Representative lines that don’t possess any
branchpoints are considered final and don’t take part in this
process.
Having labeled each branch, we scan the
branchpoints finding conflicts where we are able to apply
our algorithm, i.e. the branchpoint has at least two branches
connected to it that possess an endpoint (end-branches). If
at a certain branchpoint this is not the case, it means that
other branchpoints need to be solved first.
In each conflict, identifying a reference branch is mandatory:
If one of the branches is connected to another branchpoint it is considered as the
reference;
If not, the longest branch is chosen.
In order to solve each conflict three criteria are used:
If a branch is shorter than five pixels it is automatically deleted;
If one of the branches is much smaller than the other, the longer branch is chosen;
Finally the branch is chosen based on a continuity criterion, i.e. the branch which vector
forms the smallest angle with the reference branch vector (Fig. 27).
Fig. 25 – Neighborhood.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
39
End-branches vectors are created between its branchpoint and endpoint. In the
reference branch the vector is created between the branchpoint and the point located at the
same distance as the one between endpoint and branchpoint of the smallest end-branch.
Instead of calculating the angle and since:
) (Eq. 11)
where and are two unit vectors, we retrieve the branch whose unit vector generates the
maximum dot product with the reference branch.
Finally if there still are branchpoints present, the process is repeated.
Fig. 26 – Continuity criterion.
40 Pedro Guimarães Sá Correia
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
41
Results
In this study, we were able to export Brain Voyager data, develop and implement
automated software that maps the visual field, retrieve and cross representative lines, to
employ an effective pruning method and to interpolate polar angle and eccentricity data.
In this section, we show MRI resulting data and demonstrate our approach for
segmenting the visual areas V1, V2 and V3. We also present the results of the comparison
between our automated method and the manual segmentation one.
In addition the reader may visualize the processing stages in order to achieve the
interpolation of retinotopic data and the final results.
42 Pedro Guimarães Sá Correia
Magnetic Resonance Imaging
With magnetic resonance imaging we are able to obtain cortical thickness maps (Fig.
28). Cortical thickness measurements allow us to evaluate cortical integrity.
Two functional paradigms: polar angle and eccentricity, allow us to map the visual
space. Functional data in figure 29 is presented in the flattened surface model and was
obtained on the right hemisphere of two healthy subjects.
Fig. 27 – Cortical thickness map on a flattened surface model. Blue color represents thinner regions while green represents thicker ones (range: 0.5 – 5 mm).
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
43
Data Importing
The first step in order to achieve the intended correlation was to make data fully
accessible. Cortical thickness and retinotopic data was successfully exported from Brain
Voyager QX® into Matlab.
By removing vertices whose activation is not related to the provided stimulus and
casting the coveyed triangular mesh into an image, we enhance subsequent algorithm
processes. Figure 30 shows unprocessed polar angle and eccentricity maps within Matlab.
a) b)
c) d)
Eccentricity Polar Angle
Fig. 28 – Polar angle and eccentricity fMRI results for two different subjects; Subject one: a) and b); Subject two: c) and d)
44 Pedro Guimarães Sá Correia
Segmentation
Our approach was based on the computation of directional gradient maps which allows
identifying inversions of polar angle representations. Our algorithm is capable of the
segmentation and classification of dorsal and ventral V1, V2 and V3 visual areas. The
segmentation results presented in this section were obtained on the data presented above and
followed the procedure already described.
) )
) )
Eccentricity Polar Angle
Fig. 29 – The same data presented in Fig. 28 after being imported and preprocessed in Matlab.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
45
Data Filtering
As mentioned, in order to get an accurate gradient representation, it is essential to have
smooth data transitions. Figure 30 shows polar angle data after filtering (as an example arrows
represent gradient direction).
Gradient Gathering and Visual Areas Delineation
Figure 31 shows the obtained directional gradient maps. These maps allow the direct
delineation of the visual areas: positive and negative directional gradients indicate opposite
directions in which the visual space is represented in the visual cortex. The reversal lines in the
gradient map of the polar angle correspond to visual areas boundaries (Fig. 32).
a) b)
Fig. 30 - Polar Angle representations of the two subjects after filtering. Arrows in white indicate local gradient direction.
46 Pedro Guimarães Sá Correia
Fig. 31 - Polar angle directional gradient maps.
Fig. 32 - Polar Angle representations of the two subjects. In white visual areas boundaries obtained
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
47
Visual Areas Classification and Final Results
Figure 33 shows segmentation result after being imported back into Brain Voyager
software.
Fig. 33 – Final segmentation results for both subjects.
48 Pedro Guimarães Sá Correia
Algorithm Validation
To test the validity of the herewith proposed algorithm, we performed automated
segmentation on retinotopic data from six healthy subjects. The same data was also manually
segmented by two researchers. One of the researchers repeated the analysis two weeks after.
Reference boundaries for inter- and intra-subject validation were obtained by averaging the
two manual boundaries.
Pre-processing stages for all the subjects were performed following the same method.
Error was computed at each boundary point and averaged to the entire boundary. Two
types of error measures were used:
Each boundary point variation value was computed and normalized with the standard
deviation (sd) of the manual segmentations (standard score). Since N=2 (two graders) the
boundaries may cross, therefore it is possible that sd=0. To avoid this normalization was
performed with (sd+1):
(Eq. 12)
where z is the ‘standard score’ and the average of the two manual segmentations.
Standard deviation of the computed boundary points.
Data was excluded when the graders or the algorithm were not able to produce valid
boundaries.
Due to the similarity and to the large amount of data produced by this process, only
particular cases will be further discussed.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
49
Inter-Subject Validation
For each individual region and for each hemisphere, table 1 presents inter-subject
variation values normalized with (sd+1), while table 2 presents standard deviation values in
mm. Two researchers provided manual segmentations (N=2).
Tab. 1 - Normalized Inter-subject Variation (N=2).
Subject Hemisphere Dorsal Ventral
Average V1 V2 V3 V1 V2 V3
S1 Left 0.08 0.37 0.38 0.29 0.35 0.37 0.31
Right 0.62 0.52 0.11 0.25 0.46 0.22 0.36
S2 Left 0.32 0.19 0.47 0.09 0.19 0.22 0.25
Right - - - - - - - *1
S3 Left 0.16 1.68 0.37 0.53 0.04 0.38 0.53
Right 0.70 0.29 1.13 0.13 0.25 0.79 0.55
S4
Left 0.14 0.63 0.57 0.93 0.36 0.13 0.44 *2
Right 0.59 0.35 0.28 0.36 0.60 0.22 0.40
S5 Left 0.57 0.68 0.68 0.10 0.38 0.21 0.42
Right - - - - - - - *3
S6 Left 0.04 0.35 0.53 - 0.57 - 0.37 *4
Right 0.57 0.68 0.68 0.10 0.38 0.21 0.42
50 Pedro Guimarães Sá Correia
Tab. 2 - Inter-subject Standard Deviation values (mm).
Subject Hemisphere Dorsal Ventral
Average V1 V2 V3 V1 V2 V3
S1 Left 1.47 1.17 1.10 0.74 0.64 0.55 0.95
Right 1.38 1.24 1.19 0.86 0.95 0.75 1.06
S2 Left 0.67 1.21 1.33 1.07 0.85 1.10 1.04
Right - - - - - - - *1
S3 Left 1.14 3.17 2.11 0.92 0.53 0.84 1.45
Right 1.75 1.00 1.81 0.85 0.77 1.60 1.30
S4
Left 0.83 1.16 1.51 2.69 0.79 0.92 1.32 *2
Right 1.29 0.82 0.86 0.65 0.98 1.06 0.94
S5 Left 1.14 1.28 1.30 0.65 0.93 0.67 0.99
Right - - - - - - - *3
S6 Left 1.32 0.84 0.87 - 1.64 - 1.17 *4
Right 1.14 1.28 1.30 0.65 0.93 0.67 0.99
* 1. The algorithm was incapable of producing valid data;
* 2. Despite Brain Voyager being able to read the file containing data from the first
segmentation, when imported into Matlab, it appeared to be corrupted. Data from the
second segmentation was used instead;
* 3. One of the researchers was not able to provide a valid segmentation, due to poor
data quality;
* 4. The manual segmentations contained completely different boundaries for
ventral regions. This case will be further explained in a subsequent section.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
51
Intra-Subject Validation
For each individual region and for each hemisphere, Table 1 presents intra-subject
variation values normalized with (sd+1), while Table 2 presents standard deviation values in
mm. Manual segmentations where provided by the same researcher two weeks apart (N=2).
Tab. 3 - Intra-subject Variation (N=2) normalized with (sd+1).
Subject Hemiphere Dorsal Ventral
Average V1 V2 V3 V1 V2 V3
S1 Left 0.16 0.31 0.87 0.41 0.73 0.52 0.50
Right 0.64 0.42 0.12 0.10 0.39 0.45 0.35
S2
Left - 0.27 0.24 0.39 0.22 0.32 0.29 *1
Right - - - - - - - *2
S3 Left 0.71 1.95 0.54 0.41 0.42 0.26 0.71
Right 1.12 0.42 1.19 0.37 0.32 1.26 0.78
S4
Left - - - - - - - *3
Right 0.83 0.48 0.40 0.25 0.30 0.35 0.43
S5 Left 0.58 0.30 0.53 0.39 0.17 1.26 0.54
Right - - - - - - - *4
S6 Left 0.13 0.52 0.76 0.75 0.40 0.43 0.50
Right 0.62 0.86 0.58 0.11 0.49 0.22 0.48
52 Pedro Guimarães Sá Correia
Tab. 4 - Intra-subject Standard Deviation values in mm.
Subject Hemiphere Dorsal Ventral
Average V1 V2 V3 V1 V2 V3
S1 Left 1.42 1.23 1.61 0.92 1.10 0.91 1.20
Right 1.39 1.28 1.09 0.89 0.92 0.87 1.07
S2
Left - 1.36 1.47 1.11 1.09 1.42 1.29 *1
Right - - - - - - - *2
S3 Left 1.09 3.38 2.10 0.86 0.75 0.67 1.47
Right 1.56 1.01 1.86 0.98 0.84 1.75 1.33
S4
Left - - - - - - - *3
Right 1.34 1.02 0.84 0.80 0.91 1.01 0.99
S5 Left 1.01 0.77 1.06 0.81 0.74 2.70 1.18
Right - - - - - - - *4
S6 Left 1.31 0.77 1.05 1.14 1.02 0.93 1.04
Right 1.17 1.27 1.49 0.70 1.05 0.65 1.06
* 1. When imported into Matlab, dorsal V1 from the second segmentation is not
correctly read;
* 2. The algorithm was incapable of producing valid boundaries;
* 3. This analysis was not possible, since when imported into Matlab data from the
first segmentation appeared to be corrupted;
* 4. One of the researchers was not able to provide a valid segmentation, due to poor
data quality.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
53
Validation Analysis
In this section we will analyze overall results and present segmentation results for the
best and worst performances achieved, i.e. the highest and lowest standard deviation. Table 5
summarizes the validation data.
Tab. 5 – Validation Data Summary.
Hemisphere
Dorsal Ventral By
V1 V2 V3 V1 V2 V3 Hemisphere
Inte
r-su
bje
ct a
nal
ysis
z (s
d+1
)
Left 0.22 0.65 0.50 0.38 0.33 0.26 0.39
Right 0.62 0.46 0.55 0.21 0.42 0.36 0.44
By Region 0.42 0.55 0.52 0.29 0.38 0.31 Total Average
0.41
sd (
mm
)
Left 1.10 1.47 1.37 1.22 0.89 0.82 1.15
Right 1.39 1.08 1.29 0.75 0.91 1.02 1.07
By Region 1.24 1.28 1.33 0.98 0.90 0.92 Total Average
1.11 mm
Intr
a-su
bje
ct a
nal
ysis
z (s
d+1
)
Left 0.40 0.67 0.59 0.47 0.39 0.56 0.51
Right 0.83 0.52 0.57 0.21 0.38 0.57 0.51
By Region 0.61 0.60 0.58 0.34 0.38 0.56 Total Average
0.51
sd (
mm
)
Left 1.21 1.50 1.46 0.97 0.94 1.33 1.24
Right 1.37 1.15 1.32 0.84 0.93 1.07 1.11
By Region 1.29 1.32 1.39 0.90 0.94 1.20 Total Average
1.17 mm
Differences between the algorithm and manual segmentations are consistently higher in
dorsal regions, which supports the theory that ventral V1 is less affected by artifacts. No major
differences are present between left and right hemisphere error measures.
Both error measures are higher in intra- than inter-subject validation. However, the
difference in ‘z’ values (≈ 19.3%) are higher when compared to sd (≈ 5.4%).
54 Pedro Guimarães Sá Correia
One of the reasons for the higher difference verified in ‘z’ values, may lay in the fact that
sd values of the manual segmentations for the intra-subject validation are consistently lower
than the ones from inter-subject validation (Tab. 6).
Tab. 6 – Variation values for the Manual Segmentations (mm).
Subject Hemisphere Inter-Subject Intra-Subject
S1 Left 1.63 0.85
Right 1.60 0.71
S2 Left 2.45 1.24
Right - -
S3 Left 1.53 1.52
Right 1.97 1.38
S4 Left - -
Right 1.88 0.76
S5 Left 2.10 1.92
Right - -
S6 Left 1.55 0.91
Right 1.39 1.02
Subject S4; Right Hemisphere
The right hemisphere of subject S4 provided the
lowest average standard deviation values for the inter-
and intra-subject validation. Figure 34 shows reference
(green) and obtained (blue) boundary for ventral V1
(inter-subject validation), which is the region with the
lowest standard deviation. Red and black curves
represent the standard deviation of the obtained
boundary and from the manual segmentations
respectively.
Fig. 34 – Validation data for ventral V1. Reference (green) and obtained (blue)
boundary representation. In red and black, sd values for the obtained and manual
segmentation, respectively.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
55
Notice that standard deviation values for our algorithm are lower than the ones for the
graders.
The major differences are located in the superior and inferior boundaries. As we will
see, this is constant throughout the results, even when comparing manual segmentations of
different researchers.
Figure 35 presents inter- and intra-subject validation results for dorsal region V2.
Despite the reference boundaries being similar for both analyses, manual boundaries standard
deviation values are lower for the intra-subject validation, which may explain the higher
normalized variation values.
This hemisphere is demonstrative that the developed algorithm usually performs worse
on dorsal regions than on ventral regions.
Fig. 35 – Inter- (a) and intra-subject (b) validation data for dorsal region V2.
a) b)
56 Pedro Guimarães Sá Correia
Subject S3; Left Hemisphere
The worst performance from inter- and intra-subject validation was found on the left
hemisphere of subject S3, mainly due to dorsal regions V2 and V3.
Figure 36 shows inter- and intra-subject validation results for dorsal region V2, which is
the region with the highest standard deviation values in both analyses.
In contrast, the algorithm performs considerably better in ventral regions. Figure 37
shows validation results for ventral region V2. In both analysis, this region presents low
standard deviation and normalized variation values.
Fig. 36 – Inter- (a) and intra-subject (b) validation data for dorsal region V2.
a) b)
Fig. 37 – Inter- (a) and intra-subject (b) validation data for ventral region V2.
a) b)
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
57
Subject S6; Left Hemisphere
In figure 38, IV (black) and OA (red) boundaries represent two different manual
segmentations. Notice that in the ventral region:
V1 and V2 from OA segmentation are completely inside V1 from IV;
V2 from IV corresponds to V3 provided by OA;
Finally IV segmentation of V3 has no correspondence at all.
To avoid losing all the data from this hemisphere, and since retinotopic data seems to
support OA segmentation, we assumed V2 provided by IV as V3 and excluded other ventral
regions. The algorithm resulting regions are similar to those from the OA segmentation.
Correlating the Retina to the Visual Cortex
To correlate the retina to the visual cortex we need to compute a continuous
transformation function that maps each retinal point in the cortex and vice-versa.
Fig. 38 – Manual segmentation of V1, V2 and V3 performed by two researchers. IV – Segmentation in red; OA – Segmentation in black.
58 Pedro Guimarães Sá Correia
By retrieving and crossing representative lines from polar angle and eccentricity data we are
able to create a scattered set of points with polar angle and eccentricity values well defined. In
order to get a continuous representation, data in between that set of points, needs to be
interpolated through the Delaunay triangulation approach.
Before being intersected, representative lines need to be pruned. In this section it will be
presented pruning and crossing results.
Pruning
Representative lines are attained by object reduction, i.e. sequential boundary pixels
removal while maintaining the Euler number. As a result we get line equidistant to the region
boundaries.
Since smoother transitions reduce pruning necessity, data is filtered with a low pass filter.
Figure 39 shows visual areas dorsal and ventral V2 representative lines before and after
pruning.
Fig. 39 – Polar angle (a) and eccentricity (b) representative lines before and after pruning.
a) b)
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
59
Pruning is the process in which redundant branches are removed. Our method was
based on the three aforementioned criteria. The proposed method was employed with success,
and a consistent small increase in points with singular representations was verified (Fig. 40).
Interpolation
After intersecting the representative lines, the outcome is a collection of scattered data
points that can be visualized in a polar angle or eccentricity scale (Fig. 41).
Fig. 41 – Eccentricity (a) and Polar angle scale (b) on resulting scattered points.
Fig. 40 – Resulting scattered points. Represented in red are points removed by pruning.
a) b)
60 Pedro Guimarães Sá Correia
Figure 42 and 43 shows initial data and interpolation results for two subjects. The
number of points that we find by intersecting the lines is the major limitation to this process.
The higher the number of points, the more accurate is the interpolation. In these particular
cases almost 40% of the points weren’t located.
Notice that the interpolation is only achieved in the visual areas V1, V2 and V3. Data in
figure 42 and 43 b) represents an augmented region of a).
Fig. 42 – Initial maps and interpolation results. Interpolation data in b) represents an augmented region of a).
a)
b)
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
61
Fig. 43 – Initial maps and interpolation results. Interpolation data in b) represents an augmented region of a).
a)
b)
62 Pedro Guimarães Sá Correia
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
63
Discussion and Conclusion
In order to reduce consumed time, labor necessity and variability, we implemented an
automatic method that segments visual areas V1, V2 and V3.
The number of experienced graders available for the validation process is limited, thus
the small number of cases presented (N=6).
Inter- and intra-subject validation presented a total average standard deviation of 1.11
and 1.17 mm respectively. One of the limitations for this technique is the imaging method itself.
On the validation set, the average distance to the closest point in the visual cortex for the
surface model, ranges between 0.75 to 0.85 mm. As a curiosity, we computed standard
deviation for our segmentation before and after being casted to the original vertices, the
average was of 0.27 mm.
The algorithm performed worst with the left hemisphere of subject S3. Figure 44 shows
its polar angle map. Notice that white and blue lines represent automatic and manual
boundaries respectively.
Fig. 44 – Polar angle map of subject S3 (left hemisphere) and the manual segmentations provided by different researchers (a). In b) an augmented region is displayed. White – Automatic Segmentation;
Blue - Manual Segmentation
a) b)
64 Pedro Guimarães Sá Correia
As shown in figure 45, our algorithm is able of interpolating ‘holes’, i.e. vertices whose
activation values were lower than the threshold due to acquisition artifacts. However, it does
not overcome overall image boundaries. In contrast, manual segmentations may disregard
these boundaries, which can contribute to higher standard deviation values.
Between dorsal V1 and V2, a depression in polar angle values (Fig. 44 b) creates
distortions in gradient directions, which directly influence dorsal V1 and V2 boundaries and
subsequently V3. Manual segmentations may perceive it as an artifact and overcome it.
However, it will always be a region of divergence that depends on the applied criteria. Figure 45
shows for the same subject the two researchers’ manual segmentations.
Fig. 21 – Final results for both subjects.
) )
Fig. 45 – Polar angle map of subject S3 (left hemisphere) and the manual segmentations provided by different researchers.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
65
Figure 46 shows the manual segmentations provided by different researchers. As
mentioned, the major differences are located in the superior and inferior boundaries. Lateral
boundaries have been the main target of study in the literature, therefore superior and inferior
boundaries are not so well defined and the criteria might vary.
Interpolation quality depends on the number of located scattered points. Due to data
quality not all points are found. In several cases, retinotopic data is not distributed in the full
scale or, only a very small region becomes activated to a specific stimulus position (Fig. 47).
Fig. 46 – Polar angle map of subject S1 (right hemisphere) and the manual segmentations provided by different researchers.
Fig. 47 Polar angle map.
66 Pedro Guimarães Sá Correia
In conclusion, the algorithm is viable. However, due to the large variability in the data,
supervision from the user may be required. The inclusion of a semi-automatic mode that can be
applied to problematic data may be beneficial (inclusion of changeable parameters).
Increasing acquisition time of the functional data may improve interpolation but also
segmentation results.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
67
References
1 Acosta, O., Bourgeat, P., Zuluaga, M.A., Fripp, J., Salvado, O., Ourselin, S. (2009).
Automated voxel-based 3D cortical thickness measurement in a combined
Lagrangian–Eulerian PDE approach using partial volume maps. Medical Image
Analysis. 13(5):730-743.
2 Baroni, M., Fortunato, P., La Torre, A. (2006). Towards quantitative analysis of
retinal features in optical coherence tomography. Medical Engineering & Physics.
29(4):432-441
3 Bear, M.F., Connors, B.W., Paradiso, M.A. Neuroscience Exploring the Brain. (2007).
3rd edition, Lippincots Williams & Wilkins.
4 Bernal-Rusiel, J.L., Atienza, M., Cantero, J.L. (2010). Determining the optimal level
of smoothing in cortical thickness analysis: A hierarchical approach based on
sequential statistical thresholding. NeuroImage. 52(1):158-171.
5 Bernardos, R.L., Barthel, L.K., Meyers, J.R., Raymond, P.A. (2007). Late-Stage
Neuronal Progenitors in the Retina Are Radial Müller Glia That Function as Retinal
Stem Cells. The Journal of Neuroscience. 27(26):7028-7040.
6 Boucard, C.C., Hernowo, A.T., Maguire, R.P., Jansonius N.M., Roerdink, J.B.T.M.,
Hooymans J.M.M., Cornelissen, F.W. (2009). Changes in cortical grey matter density
associated with long-standing retinal visual field defects. Brain. 132(7):1898-1906.
7 Brezinski, M.E. Optical Coherence Tomography: Principles and Applications. (2006).
Academic Press.
68 Pedro Guimarães Sá Correia
8 Dowling, J.E. The Retina: an Approachable Part of the Brain. (1987). Belknap Press
of Harvard University Press.
9 Drexle, W., & Fujimoto, J.G. Optical Coherence Tomography: Technology and
Applications.(2008). Springer.
10 Fabritius, T., Makita, S., Miura, M., Myllylä, R., Yasuno, Y. (2009). Automated
segmentation of the macula by optical coherence tomography. Optical Society of
America. 17(18):15659-15669.
11 Fernández, D.C., Salinas, H.M., Puliafito, C.A. (2005). Automated detection of retinal
layer structures on optical coherence tomography images. Optical Society of
America. 13(25):10200-10216.
12 Garvin, M.K. Automated 3-D Segmentation and Analysis of Retinal Optical
Coherence Tomography Images. (2008). The University of Iowa, Iowa City, Iowa.
13 Gazzaniga, M.S. The New Cognitive Neurosciences.(2000). 2nd edition, The MIT
Press.
14 Gupta, N., Greenberg, G., Noël de Tilly, L., Gray, B., Polemidiotis, M., Yücel, Y.H.
(2009). Atrophy of the lateral geniculate nucleus in human glaucoma detected by
magnetic resonance imaging. British Journal of Ophthalmology. 93(1):56-60.
15 Han, I.C., Jaffe, G.J. (2008). Comparison of Spectral- and Time-Domain Optical
Coherence Tomography for Retinal Thickness Measurements in Healthy and
Diseased Eyes. American Journal of Ophthalmology. 147(5):847-858.
16 Hjelle, Ø. & Dæhlen, M. Triangulations and Applications (Mathematics and
Visualization). (2006). Springer
17 Ishikawa, H., Stein, D.M., Wollstein, G., Beaton, S., Fujimoto, J.G., Schuman, J.S.
(2005). Macular Segmentation with Optical Coherence Tomography. Investigative
Ophthalmology & Visual Science. 46(6):2012-2017.
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
69
18 Johnson, M.C., Policeni, B., Lee, A.G., Smoker, W.R.K. Neuroimaging in
Ophthalmology. (2011). 2nd edition, Oxford University Press in cooperation with the
American Academy of Ophthalmology.
19 Jones, S.E., Buchbinder, B.R., Aharon, I. (2000). Three-Dimensional Mapping of
Cortical Thickness Using Laplace’s Equation. Human Brain Mapping. 11(1):12-32.
20 Kandel, E.R., Schwartz, J.H., Jessel, T.M. Principles of Neural Science. (2000). 4th
edition, McGraw-Hill Medical.
21 Kiernan, D.F., Hariprasad, S.M., Chin, E.K., Kiernan, C.L., Rago, J., Mieler, W.F.
(2008). Prospective Comparison of Cirrus and Stratus Optical Coherence
Tomography for Quantifying Retinal Thickness. American Journal of
Ophthalmology. 147(2):267-275.e2.
22 Krieglstein, G.K. & Weinreb, R.N., . Essentials in Ophthalmology: Glaucoma. (2009).
Springer.
23 Krieglstein, G.K. & Weinreb, R.N., . Essentials in Ophthalmology: Medical retina.
(2010). Springer.
24 Kyung, M.L. Segmentations Of The Intraretinal Surfaces, Optic Disc And Retinal
Blood Vessels In 3D-OCT Scans.(2009). The University of Iowa, Iowa City, Iowa.
25 Lerch, J. Measuring Cortical Thickness. (2011). McGill University, Montreal, Canada.
26 Liew, S.H., Gilbert, C.E., Spector, T.D., Mellerio, J., Van Kuijk, F.J., Beatty, S., Fitzke,
F., Marshall, J., Hammond, C.J. (2006). Central retinal thickness is positively
correlated with macular pigment optical density. Experimental Eye Research.
82(5):915-920.
27 Livingstone, M., Hubel, D. (1988). Segregation of form, color, movement, and depth:
anatomy, physiology, and perception. Science. 240(4853):740-749.
28 Mastroianni, G. & Milovanovic, G.V. Interpolation Processes - Basic Theory and
Applications. (2008). Springer
70 Pedro Guimarães Sá Correia
29 Morrison, J.C. & Pollack, I.P. Glaucoma: Science and Practice. (2002). 1st edition,
Thieme.
30 Purves, D., Augustine, G.J., Fitzpatrick, D., Hall, W.C., LaMantia, A.S., McNamara,
J.O., Williams, S.M. Neuroscience. (2004). 3rd edition, Sinauer Associates, Inc.
31 Remington, L.A. Clinical Anatomy of the Visual System. (2005). 2nd edition, Elsevier
Inc.
32 Savoy, R.L. Functional Magnetic Resonance Imaging (fMRI). (2007). Encyclopedia of
the Brain, Functional Magnetic Resonance Imaging.
33 Schmidt-Erfurth, U., Leitgeb, R.A., Michels, S., Považay, B., Sacu, S., Hermann, B.,
Ahlers, C., Sattmann, H., Scholda, C., Fercher, A.F., Drexler, W. (2005). Three-
Dimensional Ultrahigh-Resolution Optical Coherence Tomography of Macular
Diseases. Investigative Ophthalmology & Visual Science. 46(9):3393-3402.
34 Scott, M.L.J., & Thacker, N.A. Cerebral Cortical Thickness Measurements.(2005).
Medical Image Analysis.
35 Sehi, M., Grewal, D.S., Sheets, C.W., Greenfield, D.S. (2009). Diagnostic Ability of
Fourier-Domain vs Time-Domain Optical Coherence Tomography for Glaucoma
Detection. American Journal of Ophthalmology. 148(4):597-605.
36 Shahidi, M., Wang, Z., Zelkha, R. (2005). Quantitative Thickness Measurement of
Retinal Layers Imaged by Optical Coherence Tomography. American Journal of
Ophthalmology. 139(6):1056-1061.
37 Squire, L., Berg, D., Bloom, F.E., du Lac, S., Ghosh, A., Spitzer, N.C. Fundamental
Neuroscience.(2008). 3rd edition, Elsevier Inc.
38 Tan, O., Li, G., Lu, A.T., Varma, R., Huang, D. (2008). Mapping of Macular
Substructures with Optical Coherence Tomography for Glaucoma Diagnosis.
American Academy of Ophthalmology. 115(6):949-956.
39 Tovée, M.J. An Introduction to the Visual System. (2008), 2nd edition, Cambridge
Segmentation Processes and Pattern Recognition in Retina and Brain Imaging
71
University Press.
40 Wandell, B.A., Dumoulin, S.O., Brewer, A.A. (2007). Visual Field Maps in Human
Cortex. Neuron. 56(2):366-383.
41 Wang, X., Bauer, W., Chiaia, N., Dennis, M., Gerken, M., Hummel, J., Kane,
J., Kenmuir, C., Khuder, S., Lane, R., Mooney, R., Bazeley, P., Apkarian, V., Wall, J.
(2008). Longitudinal MRI evaluations of human global cortical thickness over
minutes to weeks. Neuroscience Letters. 441(2):145-148.
42 Wotawa, N., Thirion, B. Castet, E., Anton, J.L., Faugeras, O. (2005). Human
Retinotopic Mapping Using fMRI. Rapport de Recherche. Nr 5472.
43 Yezzi, A.J. & Prince, J.L (2003). An Eulerian PDE Approach for Computing Tissue
Thickness. Medical Imaging, IEEE Transactions on. 22(10):1332.1339.
Recommended