Tutorial 03: Binary outcomes

The shallow data

In this tutorial, you will analyse data from Song et al. 2020. Second language users exhibit shallow morphological processing. DOI: 10.1017/S0272263120000170.

shallow <- read_csv("data/shallow.csv")

Rows: 6500 Columns: 11
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (8): Group, ID, List, Target, Critical_Filler, Word_Nonword, Relation_ty...
dbl (3): ACC, RT, logRT

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

shallow

ABCDEFGHIJ0123456789

Group <chr>	ID <chr>	List <chr>	Target <chr>	ACC <dbl>	RT <dbl>	logRT <dbl>	Critical_Filler <chr>	Word_Nonword <chr>	Relation_type <chr>
L1	L1_01	A	banoshment	1	423	6.0474	Filler	Nonword	Phonological
L1	L1_01	A	unawareness	1	603	6.4019	Critical	Word	Unrelated
L1	L1_01	A	unholiness	1	739	6.6053	Critical	Word	Constituent
L1	L1_01	A	bictimize	1	510	6.2344	Filler	Nonword	Phonological
L1	L1_01	A	unhappiness	1	370	5.9135	Critical	Word	Unrelated
L1	L1_01	A	entertainer	1	689	6.5352	Filler	Word	Phonological
L1	L1_01	A	unsharpness	1	821	6.7105	Critical	Word	Constituent
L1	L1_01	A	fersistent	1	677	6.5177	Filler	Nonword	Phonological
L1	L1_01	A	specificity	0	798	6.6821	Filler	Word	Semantic
L1	L1_01	A	termination	1	610	6.4135	Filler	Word	Semantic

Next

123456

...

650

Previous

1-10 of 6,500 rows | 1-10 of 11 columns

The study consisted of a lexical decision task in which participants where first shown a prime, followed by a target word for which they had to indicate whether it was a real word or a nonce word.

The prime word belonged to one of three possible groups (Relation_type in the data) each of which refers to the morphological relation of the prime and the target word:

• Unrelated: for example, prolong (assuming unkindness as target, [[un-kind]-ness]).

• Constituent: unkind.

• NonConstituent: kindness.

The expectation is that lexical decision for native English participants should be facilitated in the Constituent condition, but not in the Unrelated and NonConstituent conditions (if you are curious as to why that is the expectation, I refer you to the paper).

Let’s interpret that as:

The Constituent condition should elicit better accuracy than the other two conditions.

The study compared results of English L1 vs L2 participants and of left- vs right-branching words, but for this tutorial we will only be looking at the L1 and left-branching data. The data file also contains data from the filler items, which we filter out.

We also mutate the ACC column.

shallow_filt <- shallow %>%
  filter(
    Group == "L1",
    Branching == "Left",
    Critical_Filler == "Critical",
  ) %>%
  mutate(
    ACC = ifelse(ACC == 0, "incorrect", "correct")
  )
shallow_filt

ABCDEFGHIJ0123456789

Group <chr>	ID <chr>	List <chr>	Target <chr>	ACC <chr>	RT <dbl>	logRT <dbl>	Critical_Filler <chr>	Word_Nonword <chr>	Relation_type <chr>
L1	L1_01	A	unawareness	correct	603	6.4019	Critical	Word	Unrelated
L1	L1_01	A	unholiness	correct	739	6.6053	Critical	Word	Constituent
L1	L1_01	A	unhappiness	correct	370	5.9135	Critical	Word	Unrelated
L1	L1_01	A	unsharpness	correct	821	6.7105	Critical	Word	Constituent
L1	L1_01	A	unripeness	incorrect	1035	6.9422	Critical	Word	Unrelated
L1	L1_01	A	unkindness	correct	498	6.2106	Critical	Word	NonConstituent
L1	L1_01	A	unwariness	correct	1133	7.0326	Critical	Word	NonConstituent
L1	L1_01	A	unclearness	correct	513	6.2403	Critical	Word	Constituent
L1	L1_01	A	reobtainable	incorrect	964	6.8711	Critical	Word	NonConstituent
L1	L1_01	A	resealable	correct	709	6.5639	Critical	Word	Unrelated

Next

123456

...

53

Previous

1-10 of 522 rows | 1-10 of 11 columns

Let’s have a look at a plot that shows accuracy based on relation type.

shallow_filt %>%
  ggplot(aes(Relation_type, fill = ACC)) +
  geom_bar(position = "fill")

Modeling binary variables

Accuracy (ACC) is a binary variable and to model the probability of a binary variable we need to use the Bernoulli family. Moreover, since probabilities are bounded between 0 and 1 and linear models expect unbounded variables, the estimates of the model will be in log-odds.

This is how log-odds are related to probabilities.

Now, fit a model with brm(). Here’s some tips:

• In the formula, you want to include ACC as the outcome and Relation_type as the predictor.

• This time, you have to specify family = bernoulli, to use the Bernoulli family (Gaussian is the default).

• Remember to use shallow_filt as the data.

Then check the model summary with summary(). What does it tell you?

Compare what you understand from the model summary with the model of the plots (use conditional_effects() to get the plot).

Challenge

If you feel like and you have the time, try to run the model using both L1 and L2 data. You will have to include Group as a predictor and make sure you also include an interaction Group:Relation_type.