RULESTR_FACTOR

Define a rule strength factor to manipulate the fraction of a test item's bigrams, etc., that were present in training items.

Contents

Basic rule strength factor

RULESTR_FACTOR(aglss, 'fname') defines target values for low and high rule strength test items, composed 75% or 100% of bigrams and trigrams that appear in training items. The selection of training items will avoid some trigrams that would otherwise be allowed by the grammar.

Rule strength is the fraction of a test item's chunks from bigrams up to a specified maximum size (trigrams, by default) that are NOT novel. In other words, a test item's rule strength is the fraction of its chunks that are consistent with (the chunks appearing in) training items.

The example below generates training items based on the XX_GRAMMAR, and generates test items that are either low or high rule strength relative to trigrams in the training items.

The first output variable returned by RULESTR_FACTOR is an AGLSS object, updated to reflect the rule strength factor. The second output variable is a cell array of strings naming the different levels of rule strength defined. Actual rule strength target values are returned as the third output variable.

s_xx = aglss(xx_grammar, [3 10]);

[s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr');

levnames

tgts

s = factorial_testsets(s, {'RuleStr', levnames{:}});
s = choose_items(s, 6, 2);

disp('Training items:');
disp(format_train_items(s));

disp('Test items:');
disp(format_test_items(s));
 Potential items:
	Grammar involves 2 symbols (xy)
	2040 possible strings of length 3-10
		369 grammatical strings (18.09%)
		1671 ungrammatical strings (81.91%)
	Using all 369 grammatical strings
	Using all 1671 ungrammatical strings
 
	1 of 4 chunks of length 2 appear in no grammatical items
	3 of 8 chunks of length 3 appear in no grammatical items

levnames = 

    'LowRuleStr'
    'HighRuleStr'


tgts =

    0.7500    1.0000

 Choosing training item 1....
 Choosing training item 2....
 Choosing training item 3....
 Choosing training item 4....
 Updating potential items....
 Choosing test item 1 for each set... 1 2.
 Choosing training item 5....
 Updating potential items....
 Choosing test item 2 for each set... 1 2.
 Choosing training item 6....

Training items:
  Itm_num    Itm_name
       01         xxy
       02      xxxxyx
       03         xyx
       04       xxxyx
       05        xxyx
       06        xxxy

Test items:
 Tset_num   RuleStr_cat  Itm_num    Itm_name    RuleStr RuleStr_tgt
       01    LowRuleStr       01   xxxxxyyxy      0.733       0.750
       01    LowRuleStr       02   xxyxxxyyx      0.733       0.750
       02   HighRuleStr       01   xxxxxxxxy      1.000       1.000
       02   HighRuleStr       02  xxxxxxxxyx      1.000       1.000

Rule strength categories

RULESTR_FACTOR(aglss, 'fname', [r1 r2 ... rN]), specifies rule strength values for N categories. Test items in category I will have rule strength approximately R(I).

The example below builds on the AGLSS object S_XX created for the previous example. Three levels of rule strength are defined.

[s, levnames, tgts] = rulestr_factor(s_xx, 'MyRuleStr', [.5 .8 1]);

levnames

tgts

s = factorial_testsets(s, {'MyRuleStr', levnames{:}});
s = choose_items(s, 6, 2);

disp('Training items:');
disp(format_train_items(s));

disp('Test items:');
disp(format_test_items(s));
	1 of 4 chunks of length 2 appear in no grammatical items
	3 of 8 chunks of length 3 appear in no grammatical items

levnames = 

    'MyRuleStr1'
    'MyRuleStr2'
    'MyRuleStr3'


tgts =

    0.5000    0.8000    1.0000

 Choosing training item 1....
 Choosing training item 2....
 Choosing training item 3....
 Choosing training item 4....
 Updating potential items....
 Choosing test item 1 for each set... 3 2 1.
 Choosing training item 5....
 Updating potential items....
 Choosing test item 2 for each set... 2 3 1.
 Choosing training item 6....

Training items:
  Itm_num    Itm_name
       01         xxy
       02      xxxxyx
       03         xyx
       04       xxxyx
       05        xxyx
       06        xxxy

Test items:
 Tset_num MyRuleStr_cat  Itm_num    Itm_name  MyRuleStr MyRuleStr_tgt
       01    MyRuleStr1       01  xxxyyxyyyx      0.471         0.500
       01    MyRuleStr1       02  xxyyxyxyyy      0.471         0.500
       02    MyRuleStr2       01   xxyxxxxyy      0.800         0.800
       02    MyRuleStr2       02   yxyxyxxyx      0.800         0.800
       03    MyRuleStr3       01   xxxxxxxxy      1.000         1.000
       03    MyRuleStr3       02  xxxxxxxxyx      1.000         1.000

Naming rule strength categories

RULESTR_FACTOR(aglss, 'fname', T, {'name1', 'name2', ...}) specifies names for the different categories of rule strength, as an alternative to the default names otherwise assigned.

The example below builds on the AGLSS object S_XX created for a previous example. The default levels of rule strength are named 'L' and 'H' (for Low and High). These names appear in the display of test items.

[s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr', [], {'L', 'H'});

levnames

tgts

s = factorial_testsets(s, {'RuleStr', levnames{:}});
s = choose_items(s, 6, 2);

disp('Training items:');
disp(format_train_items(s));

disp('Test items:');
disp(format_test_items(s));
	1 of 4 chunks of length 2 appear in no grammatical items
	3 of 8 chunks of length 3 appear in no grammatical items

levnames = 

    'L'    'H'


tgts =

    0.7500    1.0000

 Choosing training item 1....
 Choosing training item 2....
 Choosing training item 3....
 Choosing training item 4....
 Updating potential items....
 Choosing test item 1 for each set... 2 1.
 Choosing training item 5....
 Updating potential items....
 Choosing test item 2 for each set... 1 2.
 Choosing training item 6....

Training items:
  Itm_num    Itm_name
       01         xxy
       02      xxxxyx
       03         xyx
       04       xxxyx
       05        xxyx
       06        xxxy

Test items:
 Tset_num   RuleStr_cat  Itm_num    Itm_name    RuleStr RuleStr_tgt
       01             L       01   xxxxxyyxy      0.733       0.750
       01             L       02   xxyxxxyyx      0.733       0.750
       02             H       01   xxxxxxxxy      1.000       1.000
       02             H       02  xxxxxxxxyx      1.000       1.000

Reserving novel grammatical chunks

RULESTR_FACTOR(aglss, 'fname', T, NAMES, M) specifies the minimum number of grammatical trigrams to exclude from training items, over and above any ungrammatical trigrams. M determines the number of different trigrams (or chunks of a different length, if specified) that will be available to appear in test items as novel trigrams which did not appear in any training items. By default, M is 2.

The example below builds on the AGLSS object S_XX created for a previous example. Here, a single grammatical trigram is excluded from training items. Thus, only one novel trigram will be available to grammatical test items. The rule strength factor is combined factorially with grammaticality to generate four sets of test items.

Note that one set of test items here fails to achieve its rule strength target. Trigrams within an item overlap with each other, and it is not always possible to manipulate chunk novelty and rule strength precisely while simultaneously satisfying other constraints on test items (like grammaticality). The limitation here is particularly severe because the grammar involves only two letters. Grammars with a greater number of different letters provide a greater number of trigrams to choose from.

[s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr', [], [], 1);

levnames

tgts

[s, glevs] = gram_factor(s, 'gram');
s = factorial_testsets(s, {'RuleStr', levnames{:}}, {'gram', glevs{:}});
s = choose_items(s, 6, 2);

disp('Training items:');
disp(format_train_items(s));

disp('Test items:');
disp(format_test_items(s));
	1 of 4 chunks of length 2 appear in no grammatical items
	3 of 8 chunks of length 3 appear in no grammatical items

levnames = 

    'LowRuleStr'
    'HighRuleStr'


tgts =

    0.7500    1.0000

 Choosing training item 1....
 Choosing training item 2....
 Choosing training item 3....
 Choosing training item 4....
 Updating potential items....
 Choosing test item 1 for each set... 3 4 1 2.
 Choosing training item 5....
 Updating potential items....
 Choosing test item 2 for each set... 2 4 3 1.
 Choosing training item 6....

Training items:
  Itm_num    Itm_name
       01         xyx
       02        yxxy
       03     xyxxxyx
       04      xyxxxy
       05      yxxxyx
       06       yxxxy

Test items:
 Tset_num   RuleStr_cat      gram_cat  Itm_num    Itm_name    RuleStr RuleStr_tgt
       01    LowRuleStr             G       01     yxyxyxy      0.727       0.750
       01    LowRuleStr             G       02   yxyxyxyxy      0.733       0.750
       02   HighRuleStr             G       01   xxyxxyxxy      1.000       1.000
       02   HighRuleStr             G       02   xxxxxxxxy      1.000       1.000
       03    LowRuleStr            NG       01   yyyxxyxxx      0.733       0.750
       03    LowRuleStr            NG       02   yxyxyxxyy      0.733       0.750
       04   HighRuleStr            NG       01  yyxxxxyxxx      0.882       1.000
       04   HighRuleStr            NG       02  yyxxxxxxyx      0.882       1.000

Reserving novel chunks regardless of grammaticality

RULESTR_FACTOR(aglss, 'fname', T, NAMES, [N M]) specifies the minimum number of chunks (N) to exclude from training items whether those chunks are grammatical or not, in addition to the minimum number of grammatical chunks (M) to exclude. In particular, M can be set to 0 if grammaticality is not going to be used as a stimulus factor.

The example below builds on the AGLSS object S_XX created for a previous example. Here, a single trigram is specifically excluded from training items, guaranteeing that at least one novel trigram will be available to test items. However, the XX_GRAMMAR uses only five trigrams, leaving three non-grammatical trigrams. Since training items are always grammatical, in this case all three of the non-grammatical trigrams will be available to test items.

[s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr', [], [], [1 0]);

levnames

tgts

s = factorial_testsets(s, {'RuleStr', levnames{:}});
s = choose_items(s, 6, 2);

disp('Training items:');
disp(format_train_items(s));

disp('Test items:');
disp(format_test_items(s));
	1 of 4 chunks of length 2 appear in no grammatical items
	3 of 8 chunks of length 3 appear in no grammatical items

levnames = 

    'LowRuleStr'
    'HighRuleStr'


tgts =

    0.7500    1.0000

 Choosing training item 1....
 Choosing training item 2....
 Choosing training item 3....
 Choosing training item 4....
 Updating potential items....
 Choosing test item 1 for each set... 1 2.
 Choosing training item 5....
 Updating potential items....
 Choosing test item 2 for each set... 2 1.
 Choosing training item 6....

Training items:
  Itm_num    Itm_name
       01        yxxy
       02      xyxxxy
       03  yxxxxyxyxy
       04     yxxxyxy
       05     yxyxxxy
       06  yxyxyxxxxy

Test items:
 Tset_num   RuleStr_cat  Itm_num    Itm_name    RuleStr RuleStr_tgt
       01    LowRuleStr       01   xxxxyxyyy      0.733       0.750
       01    LowRuleStr       02   yyyxxyxxx      0.733       0.750
       02   HighRuleStr       01   xxyxxyxxy      1.000       1.000
       02   HighRuleStr       02  xxyxxxxyxy      1.000       1.000

Chunk size used to compute rule strength

RULESTR_FACTOR(aglss, 'fname', T, NAMES, NM, CHSIZE) specifies the length of chunks on which to compute rule strength.

The example below builds on the AGLSS object S_XX created for a previous example. Here, rule strength is defined on bigrams rather than trigrams. A minimum of two bigrams are excluded from training items, with a minimum of zero grammatical bigrams excluded.

[s, levnames, tgts] = rulestr_factor(s_xx, 'BigramRuleStr', [], [], [2 0], 2);

levnames

tgts

s = factorial_testsets(s, {'BigramRuleStr', levnames{:}});
s = choose_items(s, 6, 2);

disp('Training items:');
disp(format_train_items(s));

disp('Test items:');
disp(format_test_items(s));
	1 of 4 chunks of length 2 appear in no grammatical items

levnames = 

    'LowBigramRuleStr'
    'HighBigramRuleStr'


tgts =

    0.7500    1.0000

 Choosing training item 1....
 Choosing training item 2....
 Choosing training item 3....
 Choosing training item 4....
 Updating potential items....
 Choosing test item 1 for each set... 1 2.
 Choosing training item 5....
 Updating potential items....
 Choosing test item 2 for each set... 2 1.
 Choosing training item 6....

Training items:
  Itm_num    Itm_name
       01         xxy
       02       xxxxy
       03    xxxxxxxy
       04        xxxy
       05      xxxxxy
       06     xxxxxxy

Test items:
 Tset_num BigramRuleStr_cat  Itm_num    Itm_name BigramRuleStr BigramRuleStr_tgt
       01  LowBigramRuleStr       01   xxyxxyxxy         0.750             0.750
       01  LowBigramRuleStr       02   xxxxxyyxy         0.750             0.750
       02 HighBigramRuleStr       01   xxxxxxxxy         1.000             1.000
       02 HighBigramRuleStr       02       xxxxx         1.000             1.000