Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Voice recognition in the LabTablet electronic laboratory notebook
Susana Ventura Ricardo Amorim João Rocha da Silva Cristina Ribeiro
[email protected] [email protected] [email protected] [email protected]
Faculdade de Engenharia Universidade do Porto
Faculdade de Engenharia Universidade do Porto
Faculdade de Engenharia Universidade do Porto / INESC-TEC
Faculdade de Engenharia Universidade do Porto / INESC-TEC
Contents
1. Context and motivation
2. Managing research data with LabTablet and Dendro
a. LabTablet, an electronic laboratory notebook
b. Dendro, a collaborative data management platform
3. Speech recognition in mobile environments
4. Speech in LabTablet
5. Conclusions
2
Context and Motivation
3
Researchers know a lot about their data, so we should make it easier for researchers to describe
it adequately.
This should motivate them to do it sooner, encouraging later sharing and reuse.
4
Managing research data with LabTablet and Dendro
5
Gather
Process
Describe
Publish
Researchers Curators
InstitutionsDevelopers
Research Managers
Funders
Data Providers
http://dendro.fe.up.pt 6
LabTabletan electronic laboratory notebook
7
8
LabTablet
● An Electronic Laboratory Notebook (ELN)● Runs on Android devices● Allows researchers to record metadata during field runs
and others● Uses device’s onboard sensors to record metadata (GPS
location, Luminosity, Temperature, Camera…)
9
LabTablet
● Metadata are represented as descriptor values● Descriptors can be generic of domain-specific
○ e.g. “Author”, “[temporal / geographical] Coverage”, “Temperature”, “Depiction”
○ They can depend on the research domain○ Researchers can be assisted in choosing which
descriptors to fill in
LabTablet interacts with Dendro
● Dendro recommends descriptor sets○ Researchers fill in the descriptors during data
production○ Metadata records are pushed back to Dendro ○ Researchers then upload the data to a Dendro folder
● Experiment Metadata + Data combined
10
Why speech recognition?
1. Researchers are free to use their hands while they dictate to the tablet
2. Reduces the amount of interaction with the tablet to produce metadata
11
Dendroan ontology-based RDM platform
12
Screenshot taken from http://dendro-prd.fe.up.pt:3007/project/dendrorecommendation/data/Base%20Data 13
14
File explorer
Metadata Editor
15
Descriptor selection area
16
Speech recognition in mobile environments
17
Speech recognition solutions
● Speech-based apps are becoming a part of daily life○ Google Now (Android)○ Siri (iOS)○ Cortana (Windows)
● Challenges○ Noisy environments○ Large amount of vocabulary
18
Evaluating speech recognition solutions
● Field work means that network access may be limited○ We needed offline speech recognition○ Selected library had to be open-source
● Online solutions, however, are very effective○ Faster translation speed ○ Better recognition overall
● We considered both scenarios
19
Online vs. Offline
● Online → Google Speech Recognition API○ Recognizes full sentences for note-taking○ Always-on speech recognition is taxing on the mobile device
■ Only active during note-taking○ LabTablet allows Portuguese and English keywords when in online
mode
20
Online vs. Offline (cont’d)
● Offline → CMUSphinx○ Training a speech recognition model hinders rapid prototyping○ Limited to basic word recognition
■ Keywords: “Battery” for battery temperature sensor, “luminosity” for light sensor values
○ Dictionary-based recognition■ Some very specific words are not recognized (e.g.
“descriptor”)
21
Speech in LabTablet
22
Application’s field mode
23
Descriptors gathering
“Descriptor”
Which descriptor?
“Description”
Save...
24
Online speech recognition configuration
Customization of:
● Language (ENG or PT)● Voice speed rate● Keywords
25
Conclusions
26
Conclusions
● LabTablet + Dendro
○ Tools to help researchers manage and describe data from creation to deposit
● LabTablet
○ Android-based Electronic Laboratory Notebook for researchers to use on field work or the lab
○ Uses readings from onboard sensors to fill in metadata descriptors
● Dendro
○ A web-based collaborative data management platform
○ Captures data and metadata within the research group
27
Conclusions (cont’d)
● Voice recognition in LabTablet
○ Voice commands for various operations
■ Record audio, take temperature/luminosity reading, record a note…
○ Uses online and offline voice recognition (Google + CMUSphinx)
● Tablet as an unobtrusive companion
○ Hands-free interaction
○ Less touch-based interactions mean less time spent handling the tablet
28
This work is financed by the ERDF – European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation - COMPETE 2020 Programme, and by National Funds through the FCT – Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) within project POCI-01-0145-FEDER-006961.
Acknowledgements
30