Preprocessing

Load human data

And define helper functions

## `summarise()` has grouped output by 'relation', 'drop'. You can override using the `.groups` argument.
## `summarise()` has grouped output by 'relation', 'drop'. You can override using the `.groups` argument.

Fit models to all trials (MSE objective)

Find best-fitting betas per feature for children and adults, contain and support trials together.

Fit models to all trials (MSE - r objective)

Fit separately to contain and support trials (MSE)

Could also fit separately to contain and support trials with MSE - r objective…

Feature correlations

Some of these features are perfectly correlated and even have identical values: e.g. support_probability and support_combo. All of the support_response_sharpness features with C<=.01 (C=.1 and above behave differently). All of the support_response_sharpness_accuracy features with C<=.01 (C=.1 and above behave differently). We remove some of these from the below tables.

We have 52 features from 250 simulated drops per combination to optimize softmax betas for. The tables below show fits and betas for the best-fitting 10 features (sorted by adult fit).

Fits to all trials

feature beta_adults beta_children mse_adults mse_children
support_probability 1.178 0.500 0.034 0.015
support_std 1.046 0.500 0.053 0.020
support_response_sharpness_C=0.01 1.944 0.810 0.054 0.016
support_response_linearity_r 0.975 0.500 0.054 0.020
support_response_sharpness_C=100000.0 1.237 0.596 0.059 0.017
support_response_linearity_pv 0.912 0.500 0.060 0.021
support_response_sharpness_C=1 1.090 0.574 0.061 0.017
support_response_sharpness_C=0.1 1.269 0.611 0.061 0.018
support_response_sharpness_C=10.0 1.104 0.582 0.062 0.017
support_response_sharpness_C=100.0 1.110 0.586 0.062 0.017

Fits to all trials (MSE - r)

feature beta_adults beta_children mse_adults mse_children
support_response_sharpness_C=0.01 1.944 0.810 -0.393 -0.385
normed_velocity_std_after_first_collision_objects=target 0.500 0.500 -0.240 0.133
obj_final_position_invstd_objects=target 10.000 10.000 -0.087 -0.039
support_response_sharpness_C=0.1 1.269 0.611 -0.069 -0.188
support_response_sharpness_C=100000.0 1.237 0.596 -0.057 -0.186
support_response_sharpness_C=1 1.090 0.574 0.085 -0.166
support_response_sharpness_C=10.0 1.222 0.600 0.097 -0.161
avg_final_radius_objects=target 0.500 0.500 0.099 -0.103
support_response_sharpness_C=100.0 1.299 0.616 0.106 -0.157
support_response_sharpness_GridSearchCV 1.300 0.616 0.106 -0.157

Fits to support trials (MSE)

feature beta_adults beta_children mse_adults mse_children
support_response_linearity_r 0.521 0.5 0.019 0.018
support_response_linearity_pv 0.514 0.5 0.020 0.019
support_probability 0.500 0.5 0.020 0.018
support_std 0.500 0.5 0.021 0.019
support_response_sharpness_accuracy_C=10.0 0.500 0.5 0.022 0.020
support_response_sharpness_accuracy_GridSearchCV 0.500 0.5 0.022 0.020
support_response_sharpness_accuracy_C=100.0 0.500 0.5 0.022 0.020
support_response_sharpness_accuracy_C=1 0.500 0.5 0.022 0.020
support_response_sharpness_accuracy_C=10000.0 0.500 0.5 0.022 0.020
support_response_sharpness_accuracy_C=0.1 0.500 0.5 0.022 0.020

Fits to containment trials (MSE)

feature beta_adults beta_children mse_adults mse_children
support_probability 2.480 0.839 0.010 0.008
support_std 10.000 0.829 0.028 0.018
support_response_linearity_r 1.898 0.662 0.069 0.022
support_response_sharpness_C=100000.0 1.944 0.810 0.075 0.018
support_response_sharpness_C=0.01 1.944 0.810 0.075 0.018
support_response_sharpness_C=0.1 1.944 0.810 0.075 0.018
support_response_sharpness_C=1 1.944 0.810 0.075 0.018
support_response_sharpness_C=10.0 2.030 0.842 0.075 0.018
support_response_sharpness_C=100.0 2.112 0.861 0.075 0.018
support_response_sharpness_GridSearchCV 2.113 0.861 0.075 0.018

What do the predicted choice proportions from the best features for each age group look like? Let’s look at the best feature for each each group from each fit method.

Best feature from fitting all trials (MSE)

When fit with MSE, the best-fitting feature for both children and adults is support probability, shown below with betas optimized separately for adults’ and children’s data. Note that although the MSE is quite low, the correlation between model and human responding for each trial is negative.

The next-best feature for adults was support_std, which has r=0.72 with the support_probability feature. The next-best feature for children was support_response_sharpness_C=0.01 (or C=.001, .0001, or 1e-05), which has r=0.91 with the support_probability feature. These second-best features are shown below.

Best feature from fitting all trials (MSE - r)

When fit with a combined MSE and r objective, the best-fitting feature for both children and adults is support_response_sharpness_C=0.01, shown below with betas optimized separately for adults’ and children’s data.

The next-best feature for adults was normed_velocity_std_after_first_collision_objects=target, which has r=-0.14 with the support_response_sharpness_C=0.01 feature. The next-best feature for children was support_response_sharpness_C=0.1, which has r=0.95 with the support_response_sharpness_C=0.01 feature. These second-best features are shown below.

Best features from fitting containment trials (MSE)

The best-fitting feature from fitting only containment trials was support_probability, for both children and adults:

Age feature beta mse
children support_probability 0.839 0.008
adults support_probability 2.480 0.010

Best features from fitting support trials (MSE)

The best-fitting feature for support trials for children was normed_velocity_std_after_first_collision_objects, and for adults was support_response_linearity_r (although support_probability was not far behind for adults):

Age feature beta mse
children normed_velocity_std_after_first_collision_objects 0.911 0.012
adults support_response_linearity_r 0.521 0.019

Other promising features: avg_final_radius_objects=target

avg_final_radius_objects=target shows positive correlations with children’s and adults’ preferences, but for many of the trials (both contain and support) the model chooses the target with small probability.

Other promising features: support_response_sharpness_C=0.01

support_response_sharpness_C=0.01 has a fairly good MSE in particular for children, and also shows a positive correlation with children’s preferences on containment trials.