Model

class model.modelTemplate.Model(number_actions=2, number_cues=1, number_critics=None, action_codes=None, non_action=u'None', prior=None, stimulus_shaper=None, stimulus_shaper_name=None, stimulus_shaper_properties=None, reward_shaper=None, reward_shaper_name=None, reward_shaper_properties=None, decision_function=None, decision_function_name=None, decision_function_properties=None, **kwargs)

Bases: object

The model class is a general template for a model. It also contains universal methods used by all models.

Name

The name of the class used when recording what has been used.

Type:string

currAction

The current action chosen by the model. Used to pass participant action to model when fitting

Type:int

Parameters:

• number_actions (integer, optional) – The maximum number of valid actions the model can expect to receive. Default 2.
• number_cues (integer, optional) –

  The initial maximum number of stimuli the model can expect to receive.

  Default 1.

• number_critics (integer, optional) – The number of different reaction learning sets. Default number_actions*number_cues
• action_codes (dict with string or int as keys and int values, optional) – A dictionary used to convert between the action references used by the task or dataset and references used in the models to describe the order in which the action information is stored.
• prior (array of floats in [0,1], optional) – The prior probability of of the states being the correct one. Default ones((self.number_actions, self.number_cues)) / self.number_critics)
• stimulus_shaper_name (string, optional) – The name of the function that transforms the stimulus into a form the model can understand and a string to identify it later. stimulus_shaper takes priority
• reward_shaper_name (string, optional) – The name of the function that transforms the reward into a form the model can understand. rewards_shaper takes priority
• decision_function_name (string, optional) – The name of the function that takes the internal values of the model and turns them in to a decision. decision function takes priority
• stimulus_shaper (Stimulus class, optional) – The class that transforms the stimulus into a form the model can understand and a string to identify it later. Default is Stimulus
• reward_shaper (Rewards class, optional) – The class that transforms the reward into a form the model can understand. Default is Rewards
• decision_function (function, optional) – The function that takes the internal values of the model and turns them in to a decision. Default is weightProb(range(number_actions))
• stimulus_shaper_properties (list, optional) – The valid parameters of the function. Used to filter the unlisted keyword arguments Default is None
• reward_shaper_properties (list, optional) – The valid parameters of the function. Used to filter the unlisted keyword arguments Default is None
• decision_function_properties (list, optional) – The valid parameters of the function. Used to filter the unlisted keyword arguments Default is None

Methods Summary

actStimMerge(actStimuliParam[, stimFilter])	Takes the parameter to be merged by stimuli and filters it by the stimuli values
action()	Returns the action of the model
actorStimulusProbs()	Calculates in the model-appropriate way the probability of each action.
calcProbabilities(actionValues)	Calculate the probabilities associated with the action
choiceReflection()	Allows the model to update its state once an action has been chosen.
chooseAction(probabilities, lastAction, …)	Chooses the next action and returns the associated probabilities
delta(reward, expectation, action, stimuli)	Calculates the comparison between the reward and the expectation
feedback(response)	Receives the reaction to the action and processes it
get_name()
lastChoiceReinforcement()	Allows the model to update the reward expectation for the previous trialstep given the choice made in this trialstep
observe(state)	Receives the latest observation and decides what to do with it
overrideActionChoice(action)	Provides a method for overriding the model action choice.
params()	Returns the parameters of the model
processEvent([action, response])	Integrates the information from a stimulus, action, response set, regardless of which of the three elements are present.
returnTaskState()	Returns all the relevant data for this model
rewardExpectation(stimuli)	Calculate the expected reward for each action based on the stimuli
setsimID(simID)	param simID:
standardResultOutput()	Returns the relevant data expected from a model as well as the parameters for the current model
storeStandardResults()	Updates the store of standard results found across models
storeState()	Stores the state of all the important variables so that they can be accessed later
updateModel(delta, action, stimuli, …)	param delta:The difference between the reward and the expected reward

Methods Documentation

actStimMerge(actStimuliParam, stimFilter=1)

Takes the parameter to be merged by stimuli and filters it by the stimuli values

Parameters:

• actStimuliParam (list of floats) –

  The list of values representing each action stimuli pair, where the stimuli will have their filtered

  values merged together.

• stimFilter (array of floats or a float, optional) – The list of active stimuli with their weightings or one weight for all. Default 1

Returns:

actionParams – The parameter values associated with each action

Return type:

list of floats

action()

Returns the action of the model

Returns:action Return type:integer or None

actorStimulusProbs()

Calculates in the model-appropriate way the probability of each action.

Returns:probabilities – The probabilities associated with the action choices Return type:1D ndArray of floats

calcProbabilities(actionValues)

Calculate the probabilities associated with the action

Parameters:actionValues (1D ndArray of floats) – Returns:probArray – The probabilities associated with the actionValues Return type:1D ndArray of floats

choiceReflection()

Allows the model to update its state once an action has been chosen.

chooseAction(probabilities, lastAction, events, validActions)

Chooses the next action and returns the associated probabilities

Parameters:

• probabilities (list of floats) – The probabilities associated with each combinations
• lastAction (int) – The last chosen action
• events (list of floats) – The stimuli. If probActions is True then this will be unused as the probabilities will already be
• validActions (1D list or array) – The actions permitted during this trialstep

Returns:

• newAction (int) – The chosen action
• decProbabilities (list of floats) – The weights for the different actions

delta(reward, expectation, action, stimuli)

Calculates the comparison between the reward and the expectation

Parameters:

• reward (float) – The reward value
• expectation (float) – The expected reward value
• action (int) – The chosen action
• stimuli ({int | float | tuple | None}) – The stimuli received

Returns:

Return type:

delta

feedback(response)

Receives the reaction to the action and processes it

Parameters:response (float) – The response from the task after an action. Returns without doing anything if the value of response is None.

classmethod get_name()

lastChoiceReinforcement()

Allows the model to update the reward expectation for the previous trialstep given the choice made in this trialstep

observe(state)

Receives the latest observation and decides what to do with it

There are five possible states: Observation Observation Action Observation Action Feedback Action Feedback Observation Feedback

Parameters:state (tuple of ({int | float | tuple},{tuple of int | None})) – The stimulus from the task followed by the tuple of valid actions. Passes the values onto a processing function, self._updateObservation``.

overrideActionChoice(action)

Provides a method for overriding the model action choice. This is used when fitting models to participant actions.

Parameters:action (int) – Action chosen by external source to same situation

params()

Returns the parameters of the model

Returns:parameters Return type:dictionary

processEvent(action=None, response=None)

Integrates the information from a stimulus, action, response set, regardless of which of the three elements are present.

Parameters:

• stimuli ({int | float | tuple | None}) – The stimuli received
• action (int, optional) – The chosen action of the model. Default None
• response (float, optional) – The response from the task after an action. Default None

returnTaskState()

Returns all the relevant data for this model

Returns:results Return type:dictionary

rewardExpectation(stimuli)

Calculate the expected reward for each action based on the stimuli

This contains parts that are task dependent

Parameters:stimuli ({int | float | tuple}) – The set of stimuli Returns:

• expectedRewards (float) – The expected reward for each action
• stimuli (list of floats) – The processed observations
• activeStimuli (list of [0, 1] mapping to [False, True]) – A list of the stimuli that were or were not present

setsimID(simID)

Parameters:simID (float) –

standardResultOutput()

Returns the relevant data expected from a model as well as the parameters for the current model

Returns:results – A dictionary of details about the Return type:dictionary

storeStandardResults()

Updates the store of standard results found across models

storeState()

Stores the state of all the important variables so that they can be accessed later

updateModel(delta, action, stimuli, stimuliFilter)

Parameters:

• delta (float) – The difference between the reward and the expected reward
• action (int) – The action chosen by the model in this trialstep
• stimuli (list of float) – The weights of the different stimuli in this trialstep
• stimuliFilter (list of bool) – A list describing if a stimulus cue is present in this trialstep