fpath <- system.file("extdata", "ex2.SHW",package="WrightMap")WrightMap Tutorial - Part 4
Using Conquest Output and Making Thresholds
Using Conquest Output and Making Thresholds
Updated March 18, 2016 for changes in version 1.2.
Intro
In this part of the tutorial, we’ll show how to load ConQuest output to make a CQmodel object and then WrightMaps. We’ll also show how to turn deltas into thresholds. All the example files here are available in the /inst/extdata folder of our GitHub site. If you download the latest version of the package, they should be in a folder called /extdata wherever your R packages are stored. You can set this folder as your working directory with setwd() or use the system.file() command—as in the next set of examples—to run them.
Making the model
Let’s load a model. The first parameter should be the name of the person estimates file, while the second should be the name of the show file. Both are necessary for creating Wright maps (although the CQmodel function will run fine with only one or the other, provided that they are properly passed).
We start by loading the WrightMap example files.
And we load the example output.
model1 <- CQmodel(p.est = (system.file("extdata", "ex2.eap",package="WrightMap"))
, show = (system.file("extdata", "ex2.SHW",package="WrightMap")))This (model1) is a CQmodel object. Enter the name of the object to see the names of all the tables & information stored within this object.
model1
ConQuest Output Summary:
========================
Partial Credit Analysis
The item model: item+item*step
1 dimension
582 participants
Deviance: 9272.597 (21 parameters)
Additional information available:
Summary of estimation: $SOE
Response model parameter estimates: $RMP
Regression coefficients: $reg.coef
Variances: $variances
Reliabilities: $rel.coef
GIN tables (thresholds): $GIN
EAP table: $p.est
Additional details: $run.detailsType the name of any of these tables to see the information stored there.
model1$SOE
Summary of estimation
Estimation method: Gauss-Hermite Quadrature with 15 nodes
Assumed population distribution: Gaussian
Constraint: DEFAULT
Termination criteria:
1000 iterations
0.0001 change in parameters
0.0001 change in deviance
100 iterations without a deviance improvement
10 Newton steps in M-step
Estimation terminated after 27 iterations because the deviance convergence criteria was reached.
Random number generation seed: 1
2000 nodes used for drawing 5 plausible values
200 nodes used when computing fit
Value for obtaining finite MLEs for zero/perfects: 0.3 model1$equation[1] "item+item*step"model1$reg.coef CONSTANT
Main dimension 0.972
S. errors 0.062model1$rel.coef MLE Person separation RELIABILITY
Main dimension NA
WLE Person separation RELIABILITY EAP/PV RELIABILITY
Main dimension NA 0.813 model1$variances errors
[1,] 2.162 NAThe most relevant for our purposes are the RMP, GIN, and p.est tables. The RMP tables contain the Response Model Parameters. These are item parameters. Typing model1$RMP would display them, but they’re a little long, so I’m just going to ask for the names and then show the first few rows of each table.
names(model1$RMP)[1] "item" "item*step"For this model, the RMPs have item and item*step parameters. We could add these to get the deltas. Let’s see what the tables look like.
head(model1$RMP$item) n_item item est error U.fit U.Low U.High U.T W.fit W.Low W.High W.T
1 1 1 0.753 0.055 1.11 0.88 1.12 1.8 1.10 0.89 1.11 1.8
2 2 2 1.068 0.053 1.41 0.88 1.12 6.0 1.37 0.89 1.11 6.0
3 3 3 -0.524 0.058 0.82 0.88 1.12 -3.2 0.87 0.88 1.12 -2.3
4 4 4 -1.174 0.060 0.76 0.88 1.12 -4.3 0.85 0.88 1.12 -2.7
5 5 5 -0.389 0.057 0.95 0.88 1.12 -0.9 0.95 0.89 1.11 -0.9
6 6 6 0.067 0.055 1.03 0.88 1.12 0.6 1.02 0.89 1.11 0.3head(model1$RMP$"item*step") n_item item step est error U.fit U.Low U.High U.T W.fit W.Low W.High W.T
1 1 1 0 NA NA 2.03 0.88 1.12 13.3 1.18 0.89 1.11 3.0
2 1 1 1 -1.129 0.090 0.99 0.88 1.12 -0.1 1.00 0.95 1.05 0.0
3 1 1 2 1.129 NA 0.80 0.88 1.12 -3.5 0.95 0.89 1.11 -0.9
4 2 2 0 NA NA 2.25 0.88 1.12 15.4 1.40 0.90 1.10 7.1
5 2 2 1 -0.626 0.093 1.04 0.88 1.12 0.7 1.04 0.94 1.06 1.3
6 2 2 2 0.626 NA 1.08 0.88 1.12 1.2 1.08 0.89 1.11 1.4Let’s look at a more complicated example.
model2 <- CQmodel(file.path(fpath, "ex4a.mle"), file.path(fpath, "ex4a.shw"))model2$equation[1] "rater+topic+criteria+rater*topic+rater*criteria+topic*criteria+rater*topic*criteria*step"names(model2$RMP)[1] "rater" "topic"
[3] "criteria" "rater*topic"
[5] "rater*criteria" "topic*criteria"
[7] "rater*topic*criteria*step"head(model2$RMP$"rater*topic*criteria*step") n_rater rater n_topic topic n_criteria criteria step est error U.fit
1 1 Amy 1 Sport 1 spelling 1 NA NA 0.43
2 1 Amy 1 Sport 1 spelling 2 0.299 0.398 1.34
3 1 Amy 1 Sport 1 spelling 3 -0.299 NA 1.28
4 2 Beverely 1 Sport 1 spelling 0 NA NA 0.41
5 2 Beverely 1 Sport 1 spelling 1 -0.184 0.491 3.23
6 2 Beverely 1 Sport 1 spelling 2 0.051 0.461 0.87
U.Low U.High U.T W.fit W.Low W.High W.T
1 0.70 1.30 -4.7 0.99 0.00 2.00 0.1
2 0.70 1.30 2.1 1.05 0.42 1.58 0.3
3 0.70 1.30 1.7 1.05 0.51 1.49 0.3
4 0.74 1.26 -5.8 1.47 0.00 2.09 0.9
5 0.74 1.26 10.9 0.95 0.30 1.70 0.0
6 0.74 1.26 -1.0 1.30 0.62 1.38 1.5The GIN tables show the threshold parameters.
model1$GIN [,1] [,2]
Item_1 -0.469 1.977
Item_2 0.234 1.906
Item_3 -1.789 0.742
Item_4 -2.688 0.336
Item_5 -1.656 0.883
Item_6 -1.063 1.195
Item_7 -1.969 1.047
Item_8 -1.617 1.289
Item_9 -0.957 1.508
Item_10 -0.992 2.094model2$GIN$Amy
$Amy$Sport
[,1] [,2] [,3]
spelling -31.996 -1.976 -1.250
coherence -1.447 -1.446 -1.209
structure -2.247 -0.911 -0.172
grammar -0.885 -0.773 -0.107
content -0.486 0.104 0.627
$Amy$Family
[,1] [,2] [,3]
spelling -31.996 -2.516 -0.912
coherence -1.401 -1.280 -1.103
structure -1.966 -1.260 -0.294
grammar -1.069 -0.380 -0.106
content -0.728 -0.012 0.950
$Amy$Work
[,1] [,2] [,3]
spelling -2.055 -2.051 -1.128
coherence -1.515 -1.320 -0.862
structure -1.402 -1.158 -0.631
grammar -0.816 -0.550 0.122
content -0.430 0.212 0.762
$Amy$School
[,1] [,2] [,3]
spelling -31.996 -2.059 -0.997
coherence -1.403 -1.402 -0.999
structure -1.629 -1.148 -0.462
grammar -0.967 -0.421 0.070
content -0.782 -0.027 1.121
$Beverely
$Beverely$Sport
[,1] [,2] [,3]
spelling -2.054 -1.339 -0.663
coherence -1.751 -1.129 -0.674
structure -1.042 -0.437 0.013
grammar -0.502 -0.082 0.529
content -0.253 0.613 1.184
$Beverely$Family
[,1] [,2] [,3]
spelling -31.996 -2.264 -0.718
coherence -1.524 -1.357 -0.684
structure -1.326 -0.577 0.164
grammar -0.796 0.118 0.599
content -0.469 0.690 1.230
$Beverely$Work
[,1] [,2] [,3]
spelling -2.366 -1.465 -0.672
coherence -1.388 -1.088 -0.925
structure -1.115 -0.621 0.197
grammar -0.345 0.045 0.495
content -0.212 0.482 1.282
$Beverely$School
[,1] [,2] [,3]
spelling -1.826 -1.611 -0.873
coherence -1.632 -1.222 -0.794
structure -1.270 -0.865 0.321
grammar -0.491 -0.037 0.413
content -0.361 0.449 1.137
$Colin
$Colin$Sport
[,1] [,2] [,3]
spelling -1.660 -0.685 0.564
coherence -0.612 -0.168 0.362
structure -0.485 0.519 1.512
grammar 0.611 1.275 1.698
content 1.037 1.853 2.343
$Colin$Family
[,1] [,2] [,3]
spelling -1.477 -0.677 -0.022
coherence -0.441 -0.277 0.332
structure -0.318 0.265 1.299
grammar 0.361 1.252 1.839
content 1.009 1.683 2.374
$Colin$Work
[,1] [,2] [,3]
spelling -1.697 -1.002 0.089
coherence -0.654 -0.105 0.192
structure -0.502 0.502 1.205
grammar 0.662 1.218 1.573
content 0.766 1.806 2.357
$Colin$School
[,1] [,2] [,3]
spelling -1.595 -0.788 0.095
coherence -0.629 -0.389 0.123
structure -0.470 0.122 1.237
grammar 0.385 1.010 1.679
content 0.698 1.520 2.310
$David
$David$Sport
[,1] [,2] [,3]
spelling -1.405 -0.482 0.412
coherence -0.357 0.136 0.581
structure 0.023 0.724 1.811
grammar 0.714 1.454 1.959
content 1.256 2.031 2.912
$David$Family
[,1] [,2] [,3]
spelling -1.271 -0.404 0.741
coherence 0.028 0.415 0.977
structure 0.474 1.069 1.756
grammar 1.177 1.733 2.085
content 1.284 2.169 3.596
$David$Work
[,1] [,2] [,3]
spelling -1.378 -0.587 0.498
coherence -0.119 0.260 0.795
structure 0.173 1.003 1.885
grammar 1.199 1.592 2.008
content 1.437 2.174 3.117
$David$School
[,1] [,2] [,3]
spelling -0.815 -0.330 0.424
coherence 0.062 0.293 0.805
structure 0.295 1.012 1.955
grammar 1.035 1.642 2.260
content 1.312 2.107 3.407Finally, the p.est table shows person parameters.
head(model1$p.est) ##EAPs casenum est (d1) error (d1) pop (d1)
1 1 -0.08240 0.50495 0.88205
2 2 1.75925 0.55966 0.85510
3 3 0.16483 0.49122 0.88838
4 4 3.57343 0.82692 0.68367
5 5 -0.62303 0.52908 0.87051
6 6 0.16483 0.49122 0.88838head(model2$p.est) ##MLEs casenum sscore (d1) max (d1) est (d1) error (d1)
1 1 23 60 -0.49687 0.25349
2 2 36 60 0.69311 0.26051
3 3 24 60 -0.26371 0.26378
4 4 52 60 1.85869 0.37825
5 5 47 60 1.91466 0.28843
6 6 47 60 0.53122 0.28348CQmodel, meet wrightMap
Ok, we have person parameters and item parameters: Let’s make a Wright Map.
wrightMap(model1)Using GIN table for threshold parameters
The above uses the GIN table as thresholds. But you may want to use RMP tables. For example, if you have an item table and an itemstep table, you might want to combine them to make deltas. You could do this yourself, but you could also let the make.deltas function do it for you. This function reshapes the itemstep parameters, checks the item numbers to see if there are any dichotomous items, and then adds the steps and items. This can be especially useful if you didn’t get a GIN table from ConQuest (see below).
model3 <- CQmodel(file.path(fpath, "ex2a.eap"), file.path(fpath, "ex2a.shw"))model3$GINNULLmodel3$equation[1] "item+item*step"make.deltas(model3) 1 2 3
Earth shape -0.961 -0.493 NA
Earth pictu.. -0.650 0.256 2.704
Falling off -1.416 1.969 1.265
What is Sun -0.959 1.343 NA
Moonshine 0.157 -0.482 -0.128
Moon and ni.. -0.635 0.861 NA
Night and d.. 0.157 -0.075 -0.739
Breathe on .. 0.657 1.152 -3.558When sent a model with no GIN table, wrightMap will automatically send it to make.deltas without the user having to ask.
wrightMap(model3, label.items.row = 2)
The make.deltas function can also handle rating scale models.
model4 <- CQmodel(file.path(fpath, "ex2b.eap"), file.path(fpath, "ex2b-2.shw"))model4$GINNULLmodel4$equation[1] "item+step"make.deltas(model4) 1 2
Curriculum .. -0.468 1.900
Not Until E.. -0.123 2.245
Financial R.. -1.743 0.625
Staff Commi.. -2.230 0.138
Commitment .. -1.609 0.759
Run for som.. -1.193 1.175
Achievable .. -1.570 0.798
Principals .. -1.317 1.051
Parents sup.. -0.952 1.416
Student mot.. -0.636 1.732Or let wrightMap make them automatically.
wrightMap(model4, label.items.row = 2)
Specifying the tables
In the above examples, we let wrightMap decide what parameters to graph. wrightMap starts by looking for a GIN table. If it finds that, it assumes they are thresholds and graphs them accordingly. If there is no GIN table, it then sends the function to make.deltas, which will examine the model equation to see if it knows how to handle it. Make.deltas can handle equations of the form:
A (e.g. `item`)
A + B (e.g. `item + step` [RSM])
A + A * B (e.g. `item + item * step` [PCM])
A + A * B + B (e.g `item + item * gender + gender`)But sometimes we may want something other than the default. Let’s look at model2 again.
model2$equation[1] "rater+topic+criteria+rater*topic+rater*criteria+topic*criteria+rater*topic*criteria*step"Here’s the default Wright Map, using the GIN table:
wrightMap(model2, min.logit.pad = -29)Using GIN table for threshold parameters
This doesn’t look great. Instead of showing all these estimates, we can specify a specific RMP table to use using the item.table parameter.
wrightMap(model2, item.table = "rater")
That shows just the rater parameters. Here’s just the topics.
wrightMap(model2, item.table = "topic")
What I really want, though, is to show the rater*topic estimates. For this, we can use the interactions and step.table parameters.
wrightMap(model2, item.table = "rater", interactions = "rater*topic",
step.table = "topic")
Switch the item and step names to graph it the other way:
wrightMap(model2, item.table = "topic", interactions = "rater*topic",
step.table = "rater")
You can leave out the interactions to have more of a rating scale-type model.
wrightMap(model2, item.table = "rater", step.table = "topic")
Or leave out the step table:
wrightMap(model2, item.table = "rater", interactions = "rater*topic")
Again, make.deltas is reading the model equation to decide whether to add or subtract. If, for some reason, you want to specify a different sign for one of the tables, you can use item.sign, step.sign, and inter.sign for that.
wrightMap(model2, item.table = "rater", interactions = "rater*topic",
step.table = "topic", step.sign = -1)
Making thresholds
So far, we’ve seen how to use the GIN table to graph thresholds, or the RMP tables to graph deltas. We have one use case left: Making thresholds out of those RMP-generated deltas. The make.thresholds function can handle this. The example below uses the model3 deltas, but you can send it any matrix with items as rows and steps as columns.
deltas <- make.deltas(model3)
deltas 1 2 3
Earth shape -0.961 -0.493 NA
Earth pictu.. -0.650 0.256 2.704
Falling off -1.416 1.969 1.265
What is Sun -0.959 1.343 NA
Moonshine 0.157 -0.482 -0.128
Moon and ni.. -0.635 0.861 NA
Night and d.. 0.157 -0.075 -0.739
Breathe on .. 0.657 1.152 -3.558make.thresholds(deltas)Assuming partial credit model [,1] [,2] [,3]
Earth shape -1.3229164 -0.1310804 NA
Earth pictu.. -0.9241595 0.4451567 2.7832333
Falling off -1.4503041 1.3141486 1.9728871
What is Sun -1.0466830 1.4306938 NA
Moonshine -0.6759150 -0.2252513 0.4156190
Moon and ni.. -0.8076978 1.0336795 NA
Night and d.. -0.6343026 -0.1937096 0.1852925
Breathe on .. -0.7007363 -0.5078997 -0.4741583Alternately, we can just send the model object directly:
make.thresholds(model3)Assuming partial credit model [,1] [,2] [,3]
Earth shape -1.3229164 -0.1310804 NA
Earth pictu.. -0.9241595 0.4451567 2.7832333
Falling off -1.4503041 1.3141486 1.9728871
What is Sun -1.0466830 1.4306938 NA
Moonshine -0.6759150 -0.2252513 0.4156190
Moon and ni.. -0.8076978 1.0336795 NA
Night and d.. -0.6343026 -0.1937096 0.1852925
Breathe on .. -0.7007363 -0.5078997 -0.4741583You don’t have to do any of this to make a Wright Map. You can just send the model to wrightMap, and use the type parameter to ask it to calculate the thresholds for you.
wrightMap(model3, type = "thresholds", label.items.row = 2)
Finally: If all you want is the Wright Maps, you can skip CQmodel entirely and just send your files to wrightMap.
wrightMap(file.path(fpath,"ex2a.eap"), file.path(fpath,"ex2.shw"), label.items.row = 3)wrightMap(
file.path("https://raw.githubusercontent.com/david-ti/wrightmap/refs/heads/master/inst/extdata/ex2a.eap"),
file.path("https://raw.githubusercontent.com/david-ti/wrightmap/refs/heads/master/inst/extdata/ex2a.shw")
, label.items.row = 3)