Human performance on a deductive reasoning task in the description logic ALE

SND-ID: 2023-292. Version: 1. DOI: https://doi.org/10.5878/5739-da47

Citation

Creator/Principal investigator(s)

Tjeerd Fokkens - University of Gothenburg, Department of Philosophy, Linguistics, Theory of Science orcid

Fredrik Engström - University of Gothenburg, Department of Philosophy, Linguistics, Theory of Science orcid

Research principal

University of Gothenburg - Logic group, department of philosophy, linguistics and theory of science rorId

Description

The data was collected through the (online-distributed, currently unavailable) survey to test the accuracy of the cognitive model SHARP (for more info about this model, see: https://hdl.handle.net/2077/74797 ).
SHARP was designed to simulate human performance on certain deduction tasks related to the description logic ALE.
More specifically, SHARP models the task of checking whether a given ABox (in the logic ALE) is inconsistent.
In the survey, participants were presented (among other things) ABoxes after which they were supposed to decide their inconsistency.
The data consists of:
- responses to questions about previous logic experience
- responses to demographic questions
- responses for each presented ABox (`Consistent', `Inconsistent' or `I don't know')
- responses for questions where the participant was asked to rate the difficulty of the previous ABox on a five-point Likert scale
- response times for all questions, in seconds

... Show more..
The data was collected through the (online-distributed, currently unavailable) survey to test the accuracy of the cognitive model SHARP (for more info about this model, see: https://hdl.handle.net/2077/74797 ).
SHARP was designed to simulate human performance on certain deduction tasks related to the description logic ALE.
More specifically, SHARP models the task of checking whether a given ABox (in the logic ALE) is inconsistent.
In the survey, participants were presented (among other things) ABoxes after which they were supposed to decide their inconsistency.
The data consists of:
- responses to questions about previous logic experience
- responses to demographic questions
- responses for each presented ABox (`Consistent', `Inconsistent' or `I don't know')
- responses for questions where the participant was asked to rate the difficulty of the previous ABox on a five-point Likert scale
- response times for all questions, in seconds
- a number indicating the order in which the ABoxes were presented (this order was randomised and different for each participant)
See list of variable codes and their explanation below, please note that the mathematical notation used there is only properly shown when using MathJax. The documentation file Data_Description-2.pdf has been rendered with the corresponding notation and may be used for reference.

DATA-SPECIFIC INFORMATION FOR: [results-survey539252-16.csv]

Number of variables: 108

Number of cases/rows: 84

Variable List:

G01Q37: 'Do you know what a contradiction is?'
G01Q36: 'Do you have experience with description logic?''
G01Q39: 'Have you seen this notation before? \( \\{ a \{\,:\,\} (A \sqcap B) \\} \) (If the expression does not show properly, please consider using a different browser)'
G02Q38: 'What is your gender?'
G02Q39: 'What is your age?'
G02Q40: 'What is your nationality?'
G01Q00: '\( \\{ a \{\,:\,\} A,\ b \{\,:\,\} \neg A,\ a \{\,:\,\} \neg B \\} \)'
G01Q01: '\( \\{ a \{\,:\,\} \forall s.(\neg A \sqcap B),\ (b,a) \{\,:\,\} r,\ c \{\,:\,\} (\neg B \sqcap C),\ b \{\,:\,\} \forall r. \exists s.(A \sqcap B),\ a \{\,:\,\} (B \sqcap \neg C),\ (b,c) \{\,:\,\} s \\} \)'
G02Q27: 'How difficult was it to decide whether the last ABox was inconsistent? 1 is very easy. 5 is very difficult.'
G02Q18: '\( \\{ a \{\,:\,\} (A \sqcap (B \sqcap (C \sqcap (D \sqcap \neg A)))) \\} \)'
G04Q56: the order of the ABox presented in question G02Q18
G02Q12: '\( \\{ a \{\,:\,\} (B \sqcap (C \sqcap (\neg A \sqcap A))) \\} \)'
G04Q57: the order of the ABox presented in question G02Q12
G02Q19: '\( \\{ a \{\,:\,\} (A \sqcap B),\ a \{\,:\,\} (B \sqcap C),\ a \{\,:\,\} (C \sqcap D),\ a\{\,:\,\} (D \sqcap \neg A) \\} \)'
G04Q58: the order of the ABox presented in question G02Q19
G02Q17: '\( \\{ a \{\,:\,\} ((\exists r. A \sqcap \exists r. B) \sqcap \forall r. ((\exists r. A \sqcap \exists r. B) \sqcap \forall r. (\exists r. A \sqcap \exists r. B))) \\} \)'
G04Q59: the order of the ABox presented in question G02Q17
G02Q22: '\( \\{ a \{\,:\,\} \exists r. \exists s. \neg A,\ a \{\,:\,\} \forall r. \forall s. A,\ b \{\,:\,\} \exists s. \neg B \\} \)'
G04Q60: the order of the ABox presented in question G02Q22
G02Q23: 'How difficult was it to decide whether the last ABox was inconsistent? 1 is very easy. 5 is very difficult.'
G02Q07: '\( \\{ a \{\,:\,\} (A \sqcap B),\ a \{\,:\,\} \neg B \\} \)'
G04Q61: the order of the ABox presented in question G02Q07
G02Q14: '\( \\{ a \{\,:\,\} (\exists r. (\exists r. A \sqcap \exists r. B) \sqcap \exists r. B) \\} \)'
G04Q62: the order of the ABox presented in question G02Q14
G02Q03: '\( \\{ a \{\,:\,\} A,\ a \{\,:\,\} B,\ a \{\,:\,\} \neg A \\} \)'
G04Q63: the order of the ABox presented in question G02Q03
G02Q11: '\( \\{ a \{\,:\,\} (A \sqcap (\neg A \sqcap (B \sqcap C))) \\} \)'
G04Q64: the order of the ABox presented in question G02Q11
G02Q13: '\( \\{ a \{\,:\,\} (\exists r. A \sqcap \exists r. B) \\} \)'
G04Q65: the order of the ABox presented in question G02Q13
G02Q25: 'How difficult was it to decide whether the last ABox was inconsistent? 1 is very easy. 5 is very difficult.'
G02Q10: '\( \\{ a \{\,:\,\} \neg A,\ a \{\,:\,\} \neg B \\} \)'
G04Q66: the order of the ABox presented in question G02Q10
G02Q21: '\( \\{ a \{\,:\,\} A,\ a \{\,:\,\} \neg A,\ b \{\,:\,\} \forall r. A \\} \)'
G04Q67: the order of the ABox presented in question G02Q21
G02Q15: '\( \\{ a \{\,:\,\} ( (\exists r. A \sqcap \exists r. B) \sqcap \forall r. (\exists r. A \sqcap \exists r. B)) \\} \)'
G04Q68: the order of the ABox presented in question G02Q15
G02Q09: '\( \\{ a \{\,:\,\} A,\ a \{\,:\,\} B \\} \)'
G04Q69: the order of the ABox presented in question G02Q09
G02Q05: '\( \\{ a \{\,:\,\} \exists r.A,\ a \{\,:\,\} \forall s. \neg A,\ a \{\,:\,\} \exists r. B \\} \)'
G04Q69Copy: the order of the ABox presented in question G02Q05
G02Q26: 'How difficult was it to decide whether the last ABox was inconsistent? 1 is very easy. 5 is very difficult.'
G02Q16: \( \\{ a \{\,:\,\} ( (\exists r. A \sqcap \exists r. B) \sqcap \forall r. (\exists r. (\exists r. A \sqcap \exists r. B) \sqcap \exists r. B)) \\} \)
G04Q70: the order of the ABox presented in question G02Q16
G02Q08: '\( \\{ a \{\,:\,\} \neg B,\ a \{\,:\,\} (A \sqcap B) \\} \)'
G04Q71: the order of the ABox presented in question G02Q08
G02Q06: '\( \\{ a \{\,:\,\} \exists r. A,\ a \{\,:\,\} \forall s. \neg A,\ a \{\,:\,\} \exists s. B \\} \)'
G04Q72: the order of the ABox presented in question G02Q06
G02Q04: '\( \\{ a \{\,:\,\} A,\ b \{\,:\,\} B,\ a \{\,:\,\} \neg A \\} \)'
G04Q73: the order of the ABox presented in question G02Q04
G02Q20: '\( \\{ a \{\,:\,\} (\neg A \sqcap (B \sqcap \neg C)),\ b \{\,:\,\} (\neg B \sqcap (C \sqcap A)),\ b \{\,:\,\} (A \sqcap C),\ a \{\,:\,\} B \\} \)'
G04Q74: the order of the ABox presented in question G02Q20
G02Q26Copy: 'How difficult was it to decide whether the last ABox was inconsistent? 1 is very easy. 5 is very difficult.'
interviewtime: the total response time
G01Q37Time: the reponse time of question G01Q37
G01Q36Time: the reponse time of question G01Q36
G01Q39Time: the reponse time of question G01Q39
G02Q38Time: the reponse time of question G02Q38
G02Q39Time: the reponse time of question G02Q39
G02Q40Time: the reponse time of question G02Q40
G01Q00Time: the reponse time of question G01Q00
G01Q01Time: the reponse time of question G01Q01
G02Q27Time: the reponse time of question G02Q27
G02Q28Time: the reponse time of question G02Q28 remove
G04Q41CopyTime: the waiting time before question G02Q18
G02Q18Time: the reponse time of question G02Q18
G04Q41Time: the waiting time before question G02Q12
G02Q12Time: the reponse time of question G02Q12
G04Q42Time: the waiting time before question G02Q19
G02Q19Time: the reponse time of question G02Q19
G04Q45Time: the waiting time before question G02Q17
G02Q17Time: the reponse time of question G02Q17
G04Q45CopyTime: the waiting time before question G02Q22
G02Q22Time: the reponse time of question G02Q22
G02Q23Time: the reponse time of question G02Q23
G04Q43Time: the waiting time before question G02Q07
G02Q07Time: the reponse time of question G02Q07
G04Q44Time: the waiting time before question G02Q14
G02Q14Time: the reponse time of question G02Q14
G04Q46Time: the waiting time before question G02Q03
G02Q03Time: the reponse time of question G02Q03
G04Q88Time: the waiting time before question G02Q11
G02Q11Time: the reponse time of question G02Q11
G04Q88CopyTime: the waiting time before question G02Q13
G02Q13Time: the reponse time of question G02Q13
G02Q25Time: the response time of question G02Q25
G04Q48Time: the waiting time before question G02Q10
G02Q10Time: the reponse time of question G02Q10
G04Q49Time: the waiting time before question G02Q21
G02Q21Time: the reponse time of question G02Q21
G04Q50Time: the waiting time before question G02Q15
G02Q15Time: the reponse time of question G02Q15
G04Q51Time: the waiting time before question G02Q09
G02Q09Time: the reponse time of question G02Q09
G04Q89Time: the waiting time before question G02Q05
G02Q05Time: the reponse time of question G02Q05
G02Q26Time: the reponse time of question G02Q26
G04Q53Time: the waiting time before question G02Q16
G02Q16Time: the reponse time of question G02Q16
G04Q54Time: the waiting time before question G02Q08
G02Q08Time: the reponse time of question G02Q08
G04Q54CopyTime: the waiting time before question G02Q06
G02Q06Time: the reponse time of question G02Q06
G04Q52CopyTime: the waiting time before question G02Q04
G02Q04Time: the reponse time of question G02Q04
G04Q92Time: the waiting time before question G02Q20
G02Q20Time: the reponse time of question G02Q20
G02Q26CopyTime: the reponse time of question G02Q26Copy Show less..

Data contains personal data

No

Language

Method and outcome

Unit of analysis

Population

People affiliated to study programs in logic and/or computer science in Sweden, Germany and the Netherlands who have certain experience with logic and deductive reasoning.

Time Method

Sampling procedure

Non-probability: Respondent-assisted
Non-probability: Availability
University teachers of logic courses in Sweden, Germany and the Netherlands were contacted and asked to participate and share the survey link. Sampling was stopped after 71 usable (based on the criterion of 75% correct responses) responses were collected.

Time period(s) investigated

2023-06-01 – 2023-11-30

Variables

108

Number of individuals/objects

84

Response rate/participation rate

Impossible to state the response rate, as the size of the total population was not known.

Data format / data structure

Data collection
  • Mode of collection: Self-administered questionnaire: web based
  • Description of the mode of collection: Online questionnaire using the LimeSurvey platform. Both responses and responses times were recorded.
  • Time period(s) for data collection: 2023-06-01 – 2023-11-30
  • Data collector: University of Gothenburg
  • Instrument: LimeSurvey (Structured questionnaire) - Web based questionnaire tool.
  • Source of the data: Population group
Geographic coverage

Geographic spread

Geographic location: Sweden, Netherlands, Germany

Geographic description: People from Swedish, German and Dutch universities were contacted for this study.

Administrative information

Responsible department/unit

Logic group, department of philosophy, linguistics and theory of science

Funding

  • Funding agency: Karl Langenskiölds Memorial Foundation
  • Funding agency's reference number: KL2023-0010
Topic and keywords

Research area

Algebra and logic (Standard för svensk indelning av forskningsämnen 2011)

Applied psychology (Standard för svensk indelning av forskningsämnen 2011)

Psychology (CESSDA Topic Classification)

Publications

Sort by name | Sort by year

Engström, F., & Fokkens, J. T. (2023). Cognitively adequate complexity of reasoning in a description logic. In CEUR Workshop Proceedings, 3548. CAKR’23: The 2nd International Workshop on Cognitive Aspects of Knowledge Representation, September 4, 2023, Rhodes, Greece. https://ceur-ws.org/Vol-3548/paper6.pdf
URN: urn:nbn:de:0074-3548-5
SwePub: oai:gup.ub.gu.se/331156

Fokkens, J. T. (2023). Modelling the logical mind - Using the cognitive architecture ACT-R to model human symbolic reasoning in the description logic 𝒜ℒℰ [Licentiate thesis]. University of Gothenburg.
Handle: https://hdl.handle.net/2077/74797

If you have published anything based on these data, please notify us with a reference to your publication(s). If you are responsible for the catalogue entry, you can update the metadata/data description in DORIS.

License

CC0 1.0

Versions

Version 1. 2024-01-15

Version 1: 2024-01-15

DOI: https://doi.org/10.5878/5739-da47

Contacts for questions about the data

Fredrik Engström

fredrik.engstrom@gu.se

Jelle Tjeerd Fokkens

tjeerd.fokkens@gu.se

This resource has the following relations

Published: 2024-01-15