RULESTR_FACTOR
Define a rule strength factor to manipulate the fraction of a test item's bigrams, etc., that were present in training items.
Contents
Basic rule strength factor
RULESTR_FACTOR(aglss, 'fname') defines target values for low and high rule strength test items, composed 75% or 100% of bigrams and trigrams that appear in training items. The selection of training items will avoid some trigrams that would otherwise be allowed by the grammar.
Rule strength is the fraction of a test item's chunks from bigrams up to a specified maximum size (trigrams, by default) that are NOT novel. In other words, a test item's rule strength is the fraction of its chunks that are consistent with (the chunks appearing in) training items.
The example below generates training items based on the XX_GRAMMAR, and generates test items that are either low or high rule strength relative to trigrams in the training items.
The first output variable returned by RULESTR_FACTOR is an AGLSS object, updated to reflect the rule strength factor. The second output variable is a cell array of strings naming the different levels of rule strength defined. Actual rule strength target values are returned as the third output variable.
s_xx = aglss(xx_grammar, [3 10]); [s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr'); levnames tgts s = factorial_testsets(s, {'RuleStr', levnames{:}}); s = choose_items(s, 6, 2); disp('Training items:'); disp(format_train_items(s)); disp('Test items:'); disp(format_test_items(s));
Potential items: Grammar involves 2 symbols (xy) 2040 possible strings of length 3-10 369 grammatical strings (18.09%) 1671 ungrammatical strings (81.91%) Using all 369 grammatical strings Using all 1671 ungrammatical strings 1 of 4 chunks of length 2 appear in no grammatical items 3 of 8 chunks of length 3 appear in no grammatical items levnames = 'LowRuleStr' 'HighRuleStr' tgts = 0.7500 1.0000 Choosing training item 1.... Choosing training item 2.... Choosing training item 3.... Choosing training item 4.... Updating potential items.... Choosing test item 1 for each set... 1 2. Choosing training item 5.... Updating potential items.... Choosing test item 2 for each set... 1 2. Choosing training item 6.... Training items: Itm_num Itm_name 01 xxy 02 xxxxyx 03 xyx 04 xxxyx 05 xxyx 06 xxxy Test items: Tset_num RuleStr_cat Itm_num Itm_name RuleStr RuleStr_tgt 01 LowRuleStr 01 xxxxxyyxy 0.733 0.750 01 LowRuleStr 02 xxyxxxyyx 0.733 0.750 02 HighRuleStr 01 xxxxxxxxy 1.000 1.000 02 HighRuleStr 02 xxxxxxxxyx 1.000 1.000
Rule strength categories
RULESTR_FACTOR(aglss, 'fname', [r1 r2 ... rN]), specifies rule strength values for N categories. Test items in category I will have rule strength approximately R(I).
The example below builds on the AGLSS object S_XX created for the previous example. Three levels of rule strength are defined.
[s, levnames, tgts] = rulestr_factor(s_xx, 'MyRuleStr', [.5 .8 1]); levnames tgts s = factorial_testsets(s, {'MyRuleStr', levnames{:}}); s = choose_items(s, 6, 2); disp('Training items:'); disp(format_train_items(s)); disp('Test items:'); disp(format_test_items(s));
1 of 4 chunks of length 2 appear in no grammatical items 3 of 8 chunks of length 3 appear in no grammatical items levnames = 'MyRuleStr1' 'MyRuleStr2' 'MyRuleStr3' tgts = 0.5000 0.8000 1.0000 Choosing training item 1.... Choosing training item 2.... Choosing training item 3.... Choosing training item 4.... Updating potential items.... Choosing test item 1 for each set... 3 2 1. Choosing training item 5.... Updating potential items.... Choosing test item 2 for each set... 2 3 1. Choosing training item 6.... Training items: Itm_num Itm_name 01 xxy 02 xxxxyx 03 xyx 04 xxxyx 05 xxyx 06 xxxy Test items: Tset_num MyRuleStr_cat Itm_num Itm_name MyRuleStr MyRuleStr_tgt 01 MyRuleStr1 01 xxxyyxyyyx 0.471 0.500 01 MyRuleStr1 02 xxyyxyxyyy 0.471 0.500 02 MyRuleStr2 01 xxyxxxxyy 0.800 0.800 02 MyRuleStr2 02 yxyxyxxyx 0.800 0.800 03 MyRuleStr3 01 xxxxxxxxy 1.000 1.000 03 MyRuleStr3 02 xxxxxxxxyx 1.000 1.000
Naming rule strength categories
RULESTR_FACTOR(aglss, 'fname', T, {'name1', 'name2', ...}) specifies names for the different categories of rule strength, as an alternative to the default names otherwise assigned.
The example below builds on the AGLSS object S_XX created for a previous example. The default levels of rule strength are named 'L' and 'H' (for Low and High). These names appear in the display of test items.
[s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr', [], {'L', 'H'}); levnames tgts s = factorial_testsets(s, {'RuleStr', levnames{:}}); s = choose_items(s, 6, 2); disp('Training items:'); disp(format_train_items(s)); disp('Test items:'); disp(format_test_items(s));
1 of 4 chunks of length 2 appear in no grammatical items 3 of 8 chunks of length 3 appear in no grammatical items levnames = 'L' 'H' tgts = 0.7500 1.0000 Choosing training item 1.... Choosing training item 2.... Choosing training item 3.... Choosing training item 4.... Updating potential items.... Choosing test item 1 for each set... 2 1. Choosing training item 5.... Updating potential items.... Choosing test item 2 for each set... 1 2. Choosing training item 6.... Training items: Itm_num Itm_name 01 xxy 02 xxxxyx 03 xyx 04 xxxyx 05 xxyx 06 xxxy Test items: Tset_num RuleStr_cat Itm_num Itm_name RuleStr RuleStr_tgt 01 L 01 xxxxxyyxy 0.733 0.750 01 L 02 xxyxxxyyx 0.733 0.750 02 H 01 xxxxxxxxy 1.000 1.000 02 H 02 xxxxxxxxyx 1.000 1.000
Reserving novel grammatical chunks
RULESTR_FACTOR(aglss, 'fname', T, NAMES, M) specifies the minimum number of grammatical trigrams to exclude from training items, over and above any ungrammatical trigrams. M determines the number of different trigrams (or chunks of a different length, if specified) that will be available to appear in test items as novel trigrams which did not appear in any training items. By default, M is 2.
The example below builds on the AGLSS object S_XX created for a previous example. Here, a single grammatical trigram is excluded from training items. Thus, only one novel trigram will be available to grammatical test items. The rule strength factor is combined factorially with grammaticality to generate four sets of test items.
Note that one set of test items here fails to achieve its rule strength target. Trigrams within an item overlap with each other, and it is not always possible to manipulate chunk novelty and rule strength precisely while simultaneously satisfying other constraints on test items (like grammaticality). The limitation here is particularly severe because the grammar involves only two letters. Grammars with a greater number of different letters provide a greater number of trigrams to choose from.
[s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr', [], [], 1); levnames tgts [s, glevs] = gram_factor(s, 'gram'); s = factorial_testsets(s, {'RuleStr', levnames{:}}, {'gram', glevs{:}}); s = choose_items(s, 6, 2); disp('Training items:'); disp(format_train_items(s)); disp('Test items:'); disp(format_test_items(s));
1 of 4 chunks of length 2 appear in no grammatical items 3 of 8 chunks of length 3 appear in no grammatical items levnames = 'LowRuleStr' 'HighRuleStr' tgts = 0.7500 1.0000 Choosing training item 1.... Choosing training item 2.... Choosing training item 3.... Choosing training item 4.... Updating potential items.... Choosing test item 1 for each set... 3 4 1 2. Choosing training item 5.... Updating potential items.... Choosing test item 2 for each set... 2 4 3 1. Choosing training item 6.... Training items: Itm_num Itm_name 01 xyx 02 yxxy 03 xyxxxyx 04 xyxxxy 05 yxxxyx 06 yxxxy Test items: Tset_num RuleStr_cat gram_cat Itm_num Itm_name RuleStr RuleStr_tgt 01 LowRuleStr G 01 yxyxyxy 0.727 0.750 01 LowRuleStr G 02 yxyxyxyxy 0.733 0.750 02 HighRuleStr G 01 xxyxxyxxy 1.000 1.000 02 HighRuleStr G 02 xxxxxxxxy 1.000 1.000 03 LowRuleStr NG 01 yyyxxyxxx 0.733 0.750 03 LowRuleStr NG 02 yxyxyxxyy 0.733 0.750 04 HighRuleStr NG 01 yyxxxxyxxx 0.882 1.000 04 HighRuleStr NG 02 yyxxxxxxyx 0.882 1.000
Reserving novel chunks regardless of grammaticality
RULESTR_FACTOR(aglss, 'fname', T, NAMES, [N M]) specifies the minimum number of chunks (N) to exclude from training items whether those chunks are grammatical or not, in addition to the minimum number of grammatical chunks (M) to exclude. In particular, M can be set to 0 if grammaticality is not going to be used as a stimulus factor.
The example below builds on the AGLSS object S_XX created for a previous example. Here, a single trigram is specifically excluded from training items, guaranteeing that at least one novel trigram will be available to test items. However, the XX_GRAMMAR uses only five trigrams, leaving three non-grammatical trigrams. Since training items are always grammatical, in this case all three of the non-grammatical trigrams will be available to test items.
[s, levnames, tgts] = rulestr_factor(s_xx, 'RuleStr', [], [], [1 0]); levnames tgts s = factorial_testsets(s, {'RuleStr', levnames{:}}); s = choose_items(s, 6, 2); disp('Training items:'); disp(format_train_items(s)); disp('Test items:'); disp(format_test_items(s));
1 of 4 chunks of length 2 appear in no grammatical items 3 of 8 chunks of length 3 appear in no grammatical items levnames = 'LowRuleStr' 'HighRuleStr' tgts = 0.7500 1.0000 Choosing training item 1.... Choosing training item 2.... Choosing training item 3.... Choosing training item 4.... Updating potential items.... Choosing test item 1 for each set... 1 2. Choosing training item 5.... Updating potential items.... Choosing test item 2 for each set... 2 1. Choosing training item 6.... Training items: Itm_num Itm_name 01 yxxy 02 xyxxxy 03 yxxxxyxyxy 04 yxxxyxy 05 yxyxxxy 06 yxyxyxxxxy Test items: Tset_num RuleStr_cat Itm_num Itm_name RuleStr RuleStr_tgt 01 LowRuleStr 01 xxxxyxyyy 0.733 0.750 01 LowRuleStr 02 yyyxxyxxx 0.733 0.750 02 HighRuleStr 01 xxyxxyxxy 1.000 1.000 02 HighRuleStr 02 xxyxxxxyxy 1.000 1.000
Chunk size used to compute rule strength
RULESTR_FACTOR(aglss, 'fname', T, NAMES, NM, CHSIZE) specifies the length of chunks on which to compute rule strength.
The example below builds on the AGLSS object S_XX created for a previous example. Here, rule strength is defined on bigrams rather than trigrams. A minimum of two bigrams are excluded from training items, with a minimum of zero grammatical bigrams excluded.
[s, levnames, tgts] = rulestr_factor(s_xx, 'BigramRuleStr', [], [], [2 0], 2); levnames tgts s = factorial_testsets(s, {'BigramRuleStr', levnames{:}}); s = choose_items(s, 6, 2); disp('Training items:'); disp(format_train_items(s)); disp('Test items:'); disp(format_test_items(s));
1 of 4 chunks of length 2 appear in no grammatical items levnames = 'LowBigramRuleStr' 'HighBigramRuleStr' tgts = 0.7500 1.0000 Choosing training item 1.... Choosing training item 2.... Choosing training item 3.... Choosing training item 4.... Updating potential items.... Choosing test item 1 for each set... 1 2. Choosing training item 5.... Updating potential items.... Choosing test item 2 for each set... 2 1. Choosing training item 6.... Training items: Itm_num Itm_name 01 xxy 02 xxxxy 03 xxxxxxxy 04 xxxy 05 xxxxxy 06 xxxxxxy Test items: Tset_num BigramRuleStr_cat Itm_num Itm_name BigramRuleStr BigramRuleStr_tgt 01 LowBigramRuleStr 01 xxyxxyxxy 0.750 0.750 01 LowBigramRuleStr 02 xxxxxyyxy 0.750 0.750 02 HighBigramRuleStr 01 xxxxxxxxy 1.000 1.000 02 HighBigramRuleStr 02 xxxxx 1.000 1.000