Skip to content

Instantly share code, notes, and snippets.

@seananderson
Created November 27, 2013 20:57
Show Gist options
  • Save seananderson/7683078 to your computer and use it in GitHub Desktop.
Save seananderson/7683078 to your computer and use it in GitHub Desktop.
ss3sim-ms.md changes in last day or so
diff --git a/inst/ms/ss3sim-ms.md b/inst/ms/ss3sim-ms.md
index 69cf7fc..9c4ece0 100644
--- a/inst/ms/ss3sim-ms.md
+++ b/inst/ms/ss3sim-ms.md
@@ -5,13 +5,13 @@ to evaluating fishery stock assessment methods.
In the last decade, the fisheries stock assessment modeling framework
Stock Synthesis (SS3) has become widely-used around the world.
However, there lacks a generalized
-and scriptable framework for running SS3 simulations.
+and scriptable framework for SS3 simulation testing.
Here, we introduce `ss3sim`, an `R` package that facilitates
large-scale, rapid, and reproducible simulation testing with SS3.
`ss3sim` requires an existing SS3 model configuration
along with a set of plain-text control files describing
-alternative states of nature, sampling scenarios,
-and assessment scenarios.
+alternative population dynamics, fishery properties,
+sampling scenarios, and assessment approaches.
`ss3sim` then generates an underlying truth,
samples from that truth,
modifies and runs an estimation model,
@@ -23,7 +23,7 @@ free to be modified under an open-source MIT license.
between the underlying truth and assumptions of an estimation model,
or between multiple estimation model configurations.
For example, `ss3sim` can be used to answer questions about
-time-varying model misspecification,
+model misspecification,
retrospective patterns,
and the relative importance of various types of fisheries data.
We demonstrate the software with a simple example,
@@ -37,12 +37,10 @@ and outline specific research questions that `ss3sim` could address.
Fisheries stock assessment models are crucial
to providing scientific advice and to evaluating the impact
of alternative management actions on fishery resources [@gulland1983; @hilborn1992].
-Although a variety of stock assessment methods and models
-are currently available,
-choosing among competing models
-is often not straightforward, and the choice
-often leads to different modeling outcomes
-and associated scientific advice to management.
+Although a variety of stock assessment approaches are available,
+it is often not straightforward to choose among competing approaches that
+may lead to different modeling outcomes and associated scientific advice to
+management.
Simulation testing is a critical component
to testing fishery stock assessment methods,
@@ -50,9 +48,9 @@ particularly given the potential for model misspecification
[@hilborn1987; @hilborn1992; @rosenberg1994; @peterman2004; @deroba2013a].
With simulation testing
we can evaluate the precision and bias
-of alternative complex assessment methods
+of alternative assessment methods
in a controlled environment
-where we know the true state of nature.
+where we know the true dynamics of fisheries resources under exploitation.
Recent simulation studies
have been key to improving strategies for dealing with, for example,
time-varying natural mortality ($M$) [@lee2011; @jiao2012; @deroba2013; @johnson2013]
@@ -68,7 +66,7 @@ By using this generalized framework,
individuals conducting fisheries stock assessments and peer reviewers
can focus on the underlying science, instead of the model code [@methot2013].
Owing in part to these advantages, SS3
-is one of the world's most commonly-used stock assessment models,
+is one of the world's most commonly-used stock assessment modelling frameworks,
particularly in the United States and Australia,
where it has been used in 35 and 12 stock assessments as of 2012,
respectively [@methot2013].
@@ -124,12 +122,13 @@ in `R` or other statistical software.
Finally, the `ss3sim` code is written
under an open-source MIT license and can be freely modified.
*Rapid*: `ss3sim` relies on SS3,
-which uses AD Model Builder as a back-end optimization platform ---
-the most rapid and robust non-linear optimization software [@fournier2012; @bolker2013].
+which uses AD Model Builder as a back-end optimization platform @fournier2012] ---
+the most rapid and robust non-linear optimization software [@bolker2013].
`ss3sim` also facilitates the deployment of simulations
across multiple computers or computer cores (i.e. parallelization), thereby accelerating computation.
-By using a vetted model like SS3 with the tested `ss3sim` framework,
-the time to develop, test, and verify a large-scale simulation
+By using the vetted SS3 framework [@methot2013]
+with the tested `ss3sim` package [@johnson2013; @ono2013],
+the time to develop a large-scale simulation
can be reduced substantially,
shifting focus to the research questions themselves.
@@ -144,11 +143,6 @@ of the high-level function `run_ss3sim`;
however, the low-level functions
can be used on their own
as part of a customized simulation.
-<!--TODO COLE This isn't really discussed in the vignette I don’t think. We
- might want to add a section about how to use the functions outside of
- ss3sim. Unless you meant the base function, which really isn’t a
- “customized simulation”. I’m thinking more: hey I want to use FS but I can
- use change_tv() on my models..-->
An `ss3sim` simulation requires three types of input:
(1) a base SS3 model configuration describing
@@ -204,8 +198,8 @@ These model configurations are based on
North Sea cod (*Gadus morhua*; R. Methot, pers.\ comm.),
yellowtail flounder (*Limanda ferruginea*; R. Methot, pers.\ comm.),
and Pacific sardine (*Sardinops sagax caeruleus*) [@hill2012] (Text S1).
-We recommend modifying these built-in models to match a desired scenario,
-although it is possible to modify an existing SS3 model to work with `ss3sim` (Text S1).
+We recommend modifying these built-in model configurations to match a desired scenario,
+although it is possible to modify an existing SS3 model configurations to work with `ss3sim` (Text S1).
We will base our example around the built-in cod-like model setup.
## Setting up the case files
@@ -280,10 +274,10 @@ and the folder with the plain-text case files:
## Running the simulations
-It is important to validate a simulation testing framework [@rykiel1996].
-One approach is to test the ability of the model to consistently recover
-precise and unbiased parameter estimates under ideal conditions
-with minimal process and observation error [@hilborn1992].
+It is important to validate a simulation testing framework
+with minimal or no process and observation error
+to ensure unbiased and consistent recovery of parameters
+under ideal conditions [@hilborn1992; @rykiel1996].
`ss3sim` makes this form of model validation simple by allowing users
to specify process error (i.e. recruitment deviations)
and sampling error (Text S1).
@@ -307,7 +301,6 @@ in our current directory with one function call:
\noindent
This command creates two files in our working directory:
-<!---TODO Not true, it creates 3 files, should we get rid of the residual file?-->
`ss3sim_scalars.csv` and `ss3sim_ts.csv`,
which contain scalar output estimates
(e.g. steepness and maximum sustainable yield)
@@ -346,7 +339,7 @@ or $M$ (Figure 2, lower panels).
The general purpose of `ss3sim`
is to explore model behaviour and performance
across combinations of EM configurations
-and alternative states of nature
+and alternative dynamics of fisheries resources under exploitation
specified by the OM.
In particular, `ss3sim` provides a suite of functions
for dynamically creating structural differences in both OMs and EMs.
@@ -408,7 +401,7 @@ and is therefore only compatible with the Windows operating system.
# Research opportunities with ss3sim
The `ss3sim` package has been used so far
-to evaluate alternative approaches
+to evaluate alternative assessment approaches
when $M$ is thought to vary across time [@johnson2013],
the importance of length- and age-composition data [@ono2013],
and the causes of retrospective patterns in stock assessment models.
@@ -481,8 +474,8 @@ which can be difficult to detect
based on residual patterns alone [@maunder2013].
Therefore, it is important to investigate
the consequences of model misspecification.
-Simulation testing allows for the formal evaluation
-of the ability of assessment models
+One important role of simulation testing is formally evaluating
+the ability of assessment models
to accurately and precisely
estimate parameters of interest
under different conditions and levels of misspecification
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment