Document Type
Conference Paper
Rights
Available under a Creative Commons Attribution Non-Commercial Share Alike 4.0 International Licence
Disciplines
Computer Sciences, Information Science, Linguistics
Abstract
Sentence embeddings encode information relating to the usage of idioms in a sentence. This paper reports a set of experiments that combine a probing methodology with input masking to analyse where in a sentence this idiomatic information is taken from, and what form it takes. Our results indicate that BERT’s idiomatic key is primarily found within an idiomatic expression, but also draws on information from the surrounding context. Also, BERT can distinguish between the disruption in a sentence caused by words missing and the incongruity caused by idiomatic usage.
DOI
http://dx.doi.org/10.18653/v1/2021.mwe-1.7
Recommended Citation
Nedumpozhimana, V. & Kelleher, J.D. (2021) Finding BERT’s Idiomatic Key, in the Proceedings of the 17th Workshop on Multiword Expressions, pages 57–62, Bangkok, Thailand (online), August 6, 2021. 2021 Association for Computational Linguistics. DOI:10.18653/v1/2021.mwe-1.7
Funder
ADAPT Centre
Publication Details
This paper published in the Proceedings of the 17th Workshop on Multiword Expressions, pages 57–62, Bangkok, Thailand (online), August 6, 2021. 2021 Association for Computational Linguistics