About: Exploring Automatic Short Answer Grading as a Tool to Assist in Human Rating

Facets (new session)
Description
Metadata
Settings
- owl:sameAs
- Inference Rule:

About: Exploring Automatic Short Answer Grading as a Tool to Assist in Human Rating Goto Sponge NotDistinct Permalink

An Entity of Type : schema:ScholarlyArticle, within Data Space : wasabi.inria.fr associated with source document(s)

Attributes	Values
type	Academic Article research paper schema:ScholarlyArticle
isDefinedBy	Covid-on-the-Web dataset
has title	Exploring Automatic Short Answer Grading as a Tool to Assist in Human Rating
Creator	Condor, Aubrey
Source	PMC
abstract	This project proposes using BERT (Bidirectional Encoder Representations from Transformers) as a tool to assist educators with automated short answer grading (ASAG) as opposed to replacing human judgement in high-stakes scenarios. Many educators are hesitant to give authority to an automated system, especially in assessment tasks such as grading constructed response items. However, evaluating free-response text can be time and labor costly for one rater, let alone multiple raters. In addition, some degree of inconsistency exists within and between raters for assessing a given task. Recent advances in Natural Language Processing have resulted in subsequent improvements for technologies that rely on artificial intelligence and human language. New, state-of-the-art models such as BERT, an open source, pre-trained language model, have decreased the amount of training data needed for specific tasks and in turn, have reduced the amount of human annotation necessary for producing a high-quality classification model. After training BERT on expert ratings of constructed responses, we use subsequent automated grading to calculate Cohen’s Kappa as a measure of inter-rater reliability between the automated system and the human rater. For practical application, when the inter-rater reliability metric is unsatisfactory, we suggest that the human rater(s) use the automated model to call attention to ratings where a second opinion might be needed to confirm the rater’s correctness and consistency of judgement.
has issue date	2020-06-10(xsd:dateTime)
bibo:doi	10.1007/978-3-030-52240-7_14
has license	no-cc
sha1sum (hex)	b9fcec933d83c12ed091294bce2b5025c9eaf63d
schema:url	https://doi.org/10.1007/978-3-030-52240-7_14
resource representing a document's title	Exploring Automatic Short Answer Grading as a Tool to Assist in Human Rating
has PubMed Central identifier	PMC7334737
schema:publication	Artificial Intelligence in Education
resource representing a document's body	covid:b9fcec933d83c12ed091294bce2b5025c9eaf63d#body_text
is schema:about of	named entity 'judgement' named entity 'expert' named entity 'needed' named entity 'Cohen' named entity 'grading' named entity 'assessing' named entity 'evaluating' named entity 'automated' named entity 'multiple' named entity 'HUMAN' named entity 'NECESSARY' covid:arg/b9fcec933d83c12ed091294bce2b5025c9eaf63d named entity 'For' named entity 'automated' named entity 'human' named entity 'automated' named entity 'subsequent' named entity 'authority' named entity 'metric' named entity 'needed' named entity 'However' named entity 'educators' named entity 'Short' named entity 'human judgement' named entity 'inter-rater reliability' named entity 'human language' named entity 'ASAG' named entity 'pre-trained' named entity 'training data' named entity 'Cohen's Kappa' named entity 'Short Answer' named entity 'rely' named entity 'ASAG' named entity 'BERT' named entity 'ASAG' named entity 'test set' named entity 'ASAG' named entity 'transfer learning' named entity 'Kappa statistic' named entity 'text classification' named entity 'multiple choice questions' named entity 'Short Answer' named entity 'probability' named entity 'CNN' named entity 'random chance' named entity 'random chance' named entity 'ASAG' named entity 'training data' named entity 'Cohen's Kappa' named entity 'NLU' named entity 'Natural Language Understanding' named entity '85%' named entity 'human judgement' named entity 'conceptual physics' named entity 'validation set' named entity 'ASAG' named entity 'probability' named entity 'Cohen's Kappa' named entity 'Latent Semantic Analysis' named entity 'inter-rater reliability' named entity 'Intelligent Systems' named entity 'language model' named entity 'data set' named entity 'training data' named entity 'decision-making' named entity 'LSA' named entity 'deep neural network' named entity 'pre-trained' named entity 'inter-rater reliability' named entity 'Newtonian Physics' named entity 'tokenizer' named entity 'Cohen's Kappa' named entity 'Textual Entailment' named entity 'data collection' named entity 'training data' named entity 'decision making' named entity 'response variable' named entity 'University of Memphis' named entity 'language models' named entity 'training set' named entity 'learning rate' named entity 'open source' named entity 'language model'

Faceted Search & Find service v1.13.91 as of Mar 24 2020

Alternative Linked Data Documents: Sponger | ODE Content Formats:

RDF

ODATA

Microdata

About

OpenLink Virtuoso version 07.20.3229 as of Jul 10 2020, on Linux (x86_64-pc-linux-gnu), Single-Server Edition (94 GB total memory)
Data on this page belongs to its respective rights holders.
Virtuoso Faceted Browser Copyright © 2009-2024 OpenLink Software