\documentclass[jou,fignum]{apa}
\usepackage{Sweave}
\usepackage{fancyheadings}

%opening
\title{Waiting for Grapes: Expectancy and Delayed Gratification in Bonobos}
\threeauthors{Jeffrey R. Stevens}{Alexandra G. Rosati and Sarah R. Heilbronner}{Nelly Schm\"{u}cking}
\threeaffiliations{Max Planck Institute for Human Development}{Duke University}{Max Planck Institute for Human Development}

\shorttitle{Delayed Gratification in Bonobos}
\rightheader{Delayed Gratification in Bonobos}
\leftheader{Stevens, Rosati, Heilbronner, and Schm\"{u}cking}

\abstract{Responses to delayed rewards vary widely across individuals and have important implications for  personality and temperament. Animals may avoid delayed rewards because the future is uncertain. Therefore, expectations about receiving a future reward should influence the response to delayed payoffs. Here, we offered bonobos (\emph{Pan paniscus}) a delayed gratification task in which food accumulated over time. Once subjects chose to consume the reward, food stopped accumulating. We tested their willingness to wait with a reliable and an unreliable experimenter to vary the subjects' expectations that they would receive the food. Subjects waited less often with the unreliable experimenter but showed individual differences in the degree to which reliability generalized across experimental tasks. These data suggest that the expectations generated about the likelihood of receiving future rewards influence how individuals balance current and future needs.}

\acknowledgements{Jeffrey R. Stevens and Nelly Schm\"{u}cking, Center for Adaptive Behavior  and Cognition, Max Planck Institute for Human Development, Berlin, Germany; Alexandra G. Rosati, Department of Evolutionary Anthropology, Duke University, Durham, North Carolina, USA; Sarah R. Heilbronner, Department of Neurobiology, Duke University Medical Center, Durham, North Carolina, USA.

We are indebted to our test experimenters: Juliet Conlin and Verena K\"{o}stner.   We would like to thank the staff of Zoo Berlin, Bernhard Blaszkiewitz, Peter Rahn, and Andr\'{e} Sch\"{u}le for granting permission for the project and especially Ruben Gralki for invaluable help in testing the bonobos.  This work is in accordance with the American Psychological Association's Guidelines for the Ethical Conduct in the Care and Use of Animals, the Animal Behavior Society's Guidelines for the Treatment of Animals in Behavioral Research and Teaching, and the animal welfare laws of Germany.

Correspondence concerning this article should be addressed to Jeffrey R. Stevens, Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany, E-mail: jstevens@mpib-berlin.mpg.de }

\ifapamodeman{
\note{\begin{flushleft}
    Contact:\\
    Dr.\ Jeffrey R. Stevens\\
    Center for Adaptive Behavior and Cognition\\
    Max Planck Institute for Human Development\\
    Lentzeallee 94\\
    14195 Berlin, Germany\\
    E-mail: jstevens@mpib-berlin.mpg.de\\
   fax: +49 30 82406 347
   \end{flushleft}}
}

\begin{document}

\maketitle 
\thispagestyle{fancy} \lfoot[\small Stevens, J.R., Rosati, A.G., Heilbronner, S.R., \& M\"{u}hlhoff, N. (2011). Waiting for grapes: Expectancy and delayed gratification in bonobos. \emph{International Journal for Comparative Psychology}, 24, 99-111.]{\small Stevens, J.R., Rosati, A.G., Heilbronner, S.R., \& M\"{u}hlhoff, N. (2011). Waiting for grapes: Expectancy and delayed gratification in bonobos. \emph{International Journal for Comparative Psychology}, 24, 99-111.} \cfoot[]{} \chead[]{} \rhead[]{} \lhead[]{}
All organisms face tradeoffs between immediate and future rewards. Accept the available mate or seek out a more attractive one? Raise offspring in the current environment or wait for more prosperous conditions? Stop to eat this piece of food or keep searching for a richer patch? The way in which individuals choose to balance rewards over time, or make intertemporal choices \cite{read_intertemporal_2004,stevens_intertemporal_inpress}, can therefore influence many biologically important functions, ranging from reproductive success to foraging efficiency. One of the central paradoxes in the study of intertemporal choice is why most humans and animals seem to prefer immediacy to more delayed outcomes with higher value \cite{stevens_adaptive_2009}. This may occur because the future is uncertain: delayed outcomes may not come to fruition. Indeed, uncertainty relates to intertemporal choice both in terms of the psychology underlying preferences  \cite{prelec_decision-making_1991,rachlin_subjective_1991,green_discounting_2004} and evolutionary models of how decision makers should choose \cite{kacelnik_evolution_2003,stevens_adaptive_2009}. However, few studies have clearly  demonstrated a causal relationship between the probability of receiving rewards and intertemporal choice. Consequently, understanding the role of uncertainty in temporal preferences---as well as why individuals differ in their response to uncertainty---will advance our understanding of intertemporal choice at both the proximate and ultimate level.

In one of the first studies to relate uncertainty and intertemporal choice,  \citeA{mahrer_role_1956} examined the role of expectancy on delayed reinforcement in children.  Here, expectancy refers to building a likelihood of receiving a reward.  To generate expectancy, Mahrer had children interact with one of three experimenters that varied in his/her reliability, or probability of delivering a reward.  In the `high-reliability' condition, over four sessions, the experimenter fulfilled the promise to bring a toy to the child on the following day. In the \mbox{`low-'} and `medium-reliability' conditions, the experimenter fulfilled the promise in zero or two of the four sessions.  Following this expectancy-generating phase, the experimenter then offered each child the choice between an immediate reward and a more preferred but delayed reward.  The children generalized the information about the reliability of the experimenter to this choice phase.  As reliability decreased, the children chose the delayed reward in 76\%, 42\%, and 25\% of the trials. Thus, the children learned about the reliability of the experimenters and used this information about uncertainty when making choices regarding the future.

Although Mahrer's \citeyear{mahrer_role_1956} work suggests a role for uncertainty in intertemporal choice, whether these effects extend beyond human verbal interactions remains unclear. Here we tested whether an expectation about uncertainty influences intertemporal choice in bonobos (\emph{Pan paniscus}), a species whose temporal preferences \cite{rosati_evolutionary_2007} and responses to uncertainty in payoffs \cite{heilbronner_fruit_2008} have both been studied.  Subjects participated in a standard delayed gratification task (also called a delay maintenance or food accumulation task---\cite{toner_age_1977,pele_long-tailed_2010}) previously used by Beran and colleagues \cite{beran_delay_1999,beran_maintenance_2002,beran_maintenance_2006}.  In their paradigm, an experimenter typically placed a bowl in front of subjects and then placed single rewards in the bowl one-by-one until either the subject reached into the bowl or all rewards had been placed into the bowl.  At that point, the subject could access the bowl of rewards.  Once the subject reached into the bowl, however, the experimenter stopped distributing rewards.  Therefore, with increased wait times, subjects could obtain larger quantities of reward, so the amount of time subjects waited represented a measure of their willingness to delay gratification.  By introducing into this paradigm a potential risk of not receiving the rewards, we examine how uncertainty plays a role in intertemporal choice.

To manipulate expectancy, we varied the reliability of experimenters in an introductory period before offering the bonobos any intertemporal choices.  The low-reliability experimenter began a session by offering the subject a highly valued food item but then pulling it away from the subject, thereby preventing the subject from consuming the food.  After this pre-feeding period, the low-reliability experimenter conducted a standard experimental delayed gratification session with the bonobo.  In contrast, the high-reliability experimenter began a session by offering the subject food but followed through in providing the food before conducting a standard experimental session.  With this manipulation, we can assess whether the bonobos generalize reliability from the initial phase to the subsequent delayed gratification phase, preferring more immediate options in the presence of unreliable experimenters.

\section{Methods}
\subsection{Subjects}
Four bonobos from Zoo Berlin participated in this experiment: Simon (28 year old male), Santi (26 year old male), Vifijo (13 year old male) and Opala (9 year old female). These four bonobos were housed together with the alpha female and her infant in a series of adjacent enclosures.  Simon was hand reared by zookeepers, whereas the other subjects were reared by bonobos.  None had experienced similar experiments before, but they all experienced a ``clicker'' training regime conducted by a zookeeper in which they received food for behavioral responses needed for veterinary purposes.  The bonobos received fresh fruits and vegetables after the experimental task and had constant access to water.

\subsection{Materials}
We tested subjects individually in different compartments of their home cage.  The testing apparatus was similar to \citeA{beran_maintenance_2002}.  It consisted of one bowl (containing food rewards) and one circular mark on the floor (15 cm in circumference and 41 cm from the subject's cage) in which food rewards were placed in front of the subjects.  The bonobos could easily reach their hands between the metal bars of their cage to access the circle (Figure \ref{fig:APP}).   The bonobos received grape halves (about 0.5 g) as rewards because they were a highly preferred food item. The experimenters used a stopwatch to measure time intervals and videotaped all trials for behavioral analysis. Data were analyzed using R statistical software \cite{r_manual_2009}. Data and R code are available as supplementary materials.

\subsection{Training Phase}
All sessions occurred between 0915 and 1130 h about five days a week, with one session per subject per day.  A `control' experimenter tested subjects immediately after their clicker training in the morning.  In these trials the experimenter sat about 1 m from the subject's cage and placed single grape halves from the bowl into the subject's circle (Figure \ref{fig:APP}). Initially there was no waiting period between placing the rewards, so the experimenter placed the rewards in the circle at a rate of about one per second (interreward interval = 1 s).  A daily session consisted of 8 trials with a maximum reward of 10 grape halves per trial and an intertrial interval of 30 s. Once the subject's hand crossed the line of the circle or after the 10 grapes had been placed, the experimenter covered the bowl with a lid and placed it behind her back, stopping the flow of rewards and initiating the 30 s intertrial interval. If the subject did not place all received rewards into his/her mouth within the 30 s intertrial interval, the experimenter waited 10 s after the subject placed the last reward into his/her mouth before beginning a new trial.

After a subject completed two consecutive sessions of waiting for all of the 10 rewards in 6 out of 8 trials, we incremented the interreward interval by 1 second for the next session. We continued this procedure for each subject until he/she achieved an interreward interval of 5 s, which we used for the remainder of the experiment.  Two subjects (Vifijo and Opala) did not pass this criterion after 43 and 47 sessions, respectively, so they did not advance to the experimental phase.  The other two subjects required 7 (Santi) and 15 (Simon) sessions to pass the criterion and begin the experimental phase.  During the trials, the experimenter kept verbal remarks to a minimum and avoided eye-contact with the apes both when placing the grapes and during the intertrial intervals.

\subsection{Acclimation Phase}
After the subjects reached the training criteria for five days on the 5 s interreward interval, we introduced two new test experimenters---the low- and high-reliability experimenters---to the subjects by having them observe two training sessions.  After the initial encounters, the test experimenters participated in three pre-feeding sessions before subjects experienced their daily training session. Only one experimenter conducted the pre-feeding session each day and the two test experimenters alternated, with the control experimenter testing in between the test experimenters (e.g., low-reliability, control, high-reliability, control, low-reliability, control, etc.) for a total of 12 sessions (3 sessions for each test experimenter).  In these pre-feeding sessions, each experimenter placed a banana slice in her hand about 0.5 meters above the feeding circle for about 10 s. During this time, the experimenter alternated her gaze between the banana and the subject five times.  After 10 s, the control and high-reliability experimenters dropped the banana slice into the feeding circle.  In one randomly chosen trial the low-reliability experimenter gave the subject the banana; however, in the other five trials she took the banana away and placed it in another bowl. Thus, the unreliable behavior of this experimenter introduced uncertainty into the decision-making context. The high-reliability experimenter, in contrast, was as reliable as the control experimenter, but like the low-reliability experimenter was a novel person, thereby controlling for familiarity. After waiting 10 s after the subject put the banana piece into his/her mouth, the test experimenter began another feeding trial until she completed six trials.  Following the pre-feeding period, the control experimenter conducted a standard delayed gratification task for all sessions. A standard session consisted of 8 trials with a 5 s interreward interval and a 30 s intertrial interval. We counterbalanced the assignment of the test experimenter as a low- or high-reliability experimenter across subjects.

\subsection{Experimental Phase 1: Generalized Expectation}
In the experimental phase 1, we examined whether previous experience with reliable or unreliable experimenters would influence subject's expectations about receiving the food in the delay of gratification task. To do so, we altered which of the three experimenters (control, low-reliability, or high-reliability) conducted the entire test session.  Each test experimenter conducted one session with each subject.  One control session was conducted between each experimental session, and the experimental session order alternated between test experimenters, with the initial session counterbalanced across subjects.  The control experimenter was present in the building for all sessions, but the test experimenters were only present for the experimental sessions.

All experimental sessions began with a pre-feeding period as described above.  In the control and high-reliability sessions, the experimenters gave the subject six pieces of banana.  In the low-reliability sessions, the low-reliability experimenter gave the subject only one of six pieces of banana.  Afterwards, the control experimenter gave the subjects the remaining five pieces of banana to equate consumption across conditions.  Following this pre-feeding period, the experimenters conducted a regular delayed gratification session.

\subsection{Experimental Phase 2: Social Competition}
Next, we examined whether introducing a competitive element would alter the bonobo's expectations about uncertainty in the delay of gratification task. This phase was similar to phase 1, except the low-reliability experimenter gave food to a bonobo in a neighboring cage (in visual contact about 1-1.5 m away), instead of placing it in another bowl. After this pre-feeding period, the experimenter conducted a delayed gratification session (with no visual contact with a neighbor). Both test experimenters conducted one session with each subject in a similar order as in phase 1.

\subsection{Experimental Phase 3: Direct Experience}
Finally, we examined whether direct experience with unreliable experimenters in the delayed gratification task would influence intertemporal choice. Instead of a pre-feeding period, the control and high-reliability experimenters conducted standard delayed gratification session in which waiting to the end resulted in receiving 10 grape halves. In the low-reliability session, the experimenter interrupted all trials by removing 5 grapes instead of placing the 8th, 9th or 10th grape (randomly selected) into the feeding circle. After the removal, the experimenter waited for the intertrial interval and continued to the next placement up to 10 placements. If the subject waited until the end of a trial, he received a maximum of four rewards (for expected values, see Table \ref{tab:EV}). Both test experimenters conducted three sessions with each subject.  Test experimenters alternated the order of testing with the control sessions in between (as in the acclimation phase).

\section{Results}
% R code for results

\subsection{Phase 1: Generalized Expectation}
In phase 1, we tested whether uncertainty in a pre-feeding period generalized to a subsequent delayed gratification task.  When aggregating across subjects the number of grapes before stopping (Table \ref{tab:INDIV} and left panel in Figure \ref{fig:AGGREGATE}), we do not see a strong difference across experimenters.  When separated by trials and subject (left panel in Figure \ref{fig:INDIV}), however, experimenter condition did influence choice for one of the subjects.  Simon waited for 10 grapes in all trials with the control experimenter and in all but the first trial with the high-reliability experimenter.  Yet, in the last half of the session with the low-reliability experimenter, Simon stopped after only six or seven grapes.  Santi, in contrast, waited for all 10 grapes in all conditions with the exception of the first trials. The median waiting responses did not differ between the control and high-expectancy experimenters. Individual and aggregated descriptive statistics are shown in Table \ref{tab:INDIV}.

\subsection{Phase 2: Social Competition}
Phase 2 replicated phase 1 with one difference: in the pre-feeding phase, the low-reliability experimenter gave undelivered food to a nearby group mate.  As in phase 1, experimenter condition did not influence choice at the aggregate level (Table \ref{tab:INDIV} and center panel in Figure \ref{fig:AGGREGATE}), but one subject stopped early with the low-reliability experimenter (center panel in Figure \ref{fig:INDIV}).  This is the same subject that stopped early in phase 1, and again the effect seemed to increase over the session.  The median waiting responses did not differ between the control and high-expectancy experimenters. The other subject treated all experimenters similarly.

\subsection{Phase 3: Direct Experience}
In phase 3, we dispensed with the pre-feeding phase, and had subjects directly experience uncertainty in the delayed gratification task.  Here, in both the aggregate data (Table \ref{tab:INDIV} and right panel in Figure \ref{fig:AGGREGATE}) and in the individual data (right panel in Figure \ref{fig:INDIV}), subjects stopped earlier for the low-reliability experimenter than the other two experimenters.  In this phase, both subjects responded to uncertainty by stopping early with the low-reliability experimenter but not with the control or high-expectancy experimenter.  The mean ($\pm$ 95\% confidence intervals) number of grapes before stopping dropped from 8.9 $\pm$ 0.8 to 6.1 $\pm$ 1.3 and 7.8 $\pm$ 0.9 across the three sessions.

Given the levels of uncertainty faced with the low-reliability experimenter, we can calculate the expected value of stopping after each grape.  For instance, in this phase, the expected value of waiting for the tenth grape was no longer 10; instead, it was 4 because a subject did not receive 1 grape on the interrupted trial and then lost 5 grapes.  Table \ref{tab:EV} gives the expected values of stopping after each grape, using the actual overall interruption probabilities experienced.  From these calculations, we see that stopping after seven grapes yields the highest expected value.  This approximates the number of grapes at which the bonobos stopped (right panel in Figure \ref{fig:INDIV}).

\section{Discussion}
Our data highlight two main points: uncertainty influences delayed gratification and individuals differ in their responses to uncertainty.  To summarize the experimental results, one of the two bonobos generalized about an experimenter's unreliability in the pre-feeding period with bananas to the delayed gratification task with grapes.  Namely, this bonobo became more impulsive (that is, began consuming grapes before all were distributed, thereby foregoing larger rewards) with an unreliable experimenter.  A similar pattern emerged in the second phase when we added social competition to unreliability: the same bonobo exhibited similar levels of impulsivity towards the low-reliability experimenter as he exhibited in the first phase, thus generalizing about uncertainty across tasks. In the final phase, both subjects responded to the direct experience of uncertainty in the delayed gratification task.  When interrupted during a trial, both bonobos chose more impulsively, matching optimal choices for maximizing their expected food intake.

This work provides evidence of uncertainty influencing intertemporal choices in animals.  Though theoretical and empirical work relates uncertainty and intertemporal choice \cite{prelec_decision-making_1991,rachlin_subjective_1991,green_discounting_2004}, previous direct tests have failed to find a connection.  Notably, \citeA{henly_discounting_2008} offered blue jays (\emph{Cyanocitta cristata}) a standard `self-control' task in which the birds chose between a smaller, sooner and a larger, later reward.  Henley and colleagues added a constant probability of interruption per unit of time delay such that, as delays increased, the probability of losing the reward increased.  Choosing longer delays increased the risk of not receiving the reward, so uncertainty positively correlated with time delay.  Despite time acting as a proxy for uncertainty, the blue jays did not choose the smaller, sooner option more as the interruption risk increased. Rather, they either did not change their preferences across interruption rates or even slightly increased their preference to choose the larger, later option at higher interruption rates.

In contrast to the Henly study, our data demonstrate that uncertainty does influence intertemporal choices in bonobos.  When uncertainty is embedded directly into the temporal task, both bonobos responded by reducing delayed gratification under uncertainty.  Interestingly, one of the bonobos even responded to the expectation of uncertainty by generalizing about an experimenter's reliability from a non-temporal, non-choice task to an intertemporal choice task.  Both the temporal element and the choice element differed across the pre-feeding and delayed gratification tasks.  Thus, like the children in Mahrer's \citeyear{mahrer_role_1956} task, one bonobo passively acquired information about uncertainty associated with an individual in one context and applied it to a novel context in which information about uncertainty was potentially relevant. Surprisingly, rather than starting off impulsive, this bonobo tended to start sessions in both phase 1 and phase 2 by waiting for the full amount of grapes and only became impulsive in the second half of trials with the low-expectancy experimenter.  This may have resulted from the sheer strength of previous reinforcement in this task.  Both subjects experience between 20-27 practice and acclimation sessions before beginning the test phases, so the impulsive response had to compete with an overlearned waiting response.

To increase attention to the uncertainty associated with the pre-feeding task, we included a phase in which the low-reliability experimenter gave non-received food items to a social competitor (phase 2).  Surprisingly, this manipulation did not influence the choices of either subject: Simon stopped early at the same rate as in phase 1 and Santi continued to disregard the unreliability of this experimenter.  There are at least two possible reasons for social competition not affecting choices.  First, bonobos are a very tolerant species, showing food sharing and other cooperative behaviors \cite{fruth_bonobos_2002,hare_tolerance_2007,hare_bonobos_2010}, so perhaps the social competition was not as salient as it might be for other species such as chimpanzees.  Second, the social competition does not directly bare on the delayed gratification task.  It could be that both subjects already attended to the pre-feeding task in the first phase, so social competition did not
increase attention further.

The primary weakness of this experiment is, of course, the small sample size.  Though the sample size is not uncommon in experimental studies of bonobos, we would like to see more bonobos tested. Nevertheless, we do see differences between the individuals in this experiment.  First, we began the experiment with four subjects, but only two passed the training criterion.  The two that could not wait 5 s between grapes were the youngest and lowest ranking male and female of the group.  Second, we saw differences between the two subjects that completed the experiment.  One subject became more impulsive in the delayed gratification task in response to generalized unreliability, whereas the other subject showed no effect of unreliability until it directly applied to the delayed gratification task.

One potentially interesting way to think about individual differences comes from the psychology of expertise literature \cite{ericsson_expert_1996,cokely_cognitive_2009} in which researchers distinguish between capacity and strategy.  A \emph{capacity} typically refers to the maximum ability to perform a mental task, whereas a \emph{strategy} is a set of actions used to accomplish a task \cite{vandenbos_apa_2006}.  Much work in psychology attributes individual differences to differences in capacity, where it is treated as a fixed, rather hard-wired trait.  However, many individual differences may result from applying different strategies.  For instance, in our experiment, one might postulate that one bonobo did not respond to unreliability in phases 1 and 2 because he could not generalize across contexts (he lacked the capacity).  Yet, he may have had the capacity to generalize but simply did not employ the strategy of becoming more impulsive because the contexts differed (namely, there was actually no unreliability in the delayed gratification task).  In our case, this strategy actually yielded a higher payoff.

Another application of the capacity/strategy distinction is important for species organized in dominance hierarchies.   In many cases, most individuals of a species may have similar psychological capacities.  However, the nature of dominance hierarchies results in pressure to employ different strategies depending on where an individual falls on the hierarchy.  Subordinates, for instance, may have to demonstrate strong inhibitory control \cite{amici_fission-fusion_2008,aureli_fission-fusion_2008} and patience \cite{stevens_patience_2008} in the face of dominants because grabbing food in front of the dominant can be quite costly.  Nevertheless, in the absence of dominants, subordinates may act extremely impulsively to take advantage of the opportunity to feed without harassment \cite{stevens_food_2002}.  Thus, depending on the context, individuals will employ different strategies.  This may explain why our subordinate bonobos did not delay gratification enough to pass our training criterion.  That being said, humans show enormous variation in delayed gratification \cite{mischel_delay_1989,frederick_time_2002}. In fact, decades of study have revealed that an individual's tendency toward impulsivity or patience predicts such diverse life variables as likelihood of substance dependence, marital success, academic achievement, and cooperativeness \cite{mischel_delay_1989,ayduk_regulating_2000,duckworth_self-discipline_2005,harris_delay_2002}. Though not well studied yet, there likely is variation in delayed gratification in other animals that cannot be attributed to dominance status, sex, age, etc.  Recent work in the animal personality literature proposes adaptive reasons for individual differences \cite{dall_behavioural_2004,sih_behavioral_2004,wolf_life-history_2007}, but much work in this area remains.

%%%% REFERENCES %%%%
\bibliography{bonobo}
\vspace{3cm}

%%%% TABLE EV %%%%
\begin{table*}[ht]
\caption{\label{tab:EV} Expected values for planning to stop after each food placement}

\noindent
\begin{tabular}{l*{10}{c}} \hline
Number of placements & \hspace{1mm}1\hspace{1mm} &  \hspace{1mm} 2 \hspace{1mm} &  \hspace{1mm} 3 \hspace{1mm} &  \hspace{1mm} 4 \hspace{1mm} &  \hspace{1mm} 5 \hspace{1mm} &  \hspace{1mm} 6 \hspace{1mm} & 7 & 8 & 9 &  \hspace{1mm} 10 \hspace{1mm} \\ \hline
Expected value of grapes ($EV$) & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 5.63* & 4.85* & 4 \\ \hline
\end{tabular}
\small
*$EV_{8}$ = (1-$p_{8}$)$\cdot$8 + $p_{8}\cdot$2 = 5.63 and $EV_{9}$ = (1-$p_{8}$)(1-$p_{9}$)$\cdot$9 + (1-$p_{8}$)$p_{9}\cdot$3 + $p_{8}\cdot$ 2 = 4.85,
where $p_{8}$ and $p_{9}$ represent the probabilities of being interrupted in the eighth and ninth grape, respectively ($p_{8}$= 0.4 and $p_{9}$= 0.38). \normalsize
\end{table*}
\vspace{3cm}

%%%% TABLE INDIV %%%%
\begin{table*}[ht]
\caption{\label{tab:INDIV} Descriptive statistics for number of grapes before stopping in each phase}

\noindent
\begin{tabular}{cccrrrrr} \hline
Subject & Phase & Condition & Median & Mean & Std. Dev. & 95\% CI & Trials \\ \hline
Simon & 1 & Control & 10 &
    10 &
    0 &
    0 &
    16 \\
 & 1 & High & 10 &
    8.88 &
    3.18 &
    2.66 &
    8 \\
 & 1 & Low & 8.5 &
    8.12 &
    2.03  &
    1.7  &
    8  \\
 & 2 & Control & 10 &
    9.81 &
    0.75 &
    0.4 &
    16 \\
 & 2 & High & 10 &
    9.88 &
    0.35 &
    0.3 &
    8 \\
 & 2 & Low & 8.5 &
    8.25 &
    1.58 &
    1.32 &
    8 \\
 & 3 & Control & 10 &
    9.92 &
    0.45 &
    0.13 &
    48 \\
 & 3 & High & 10 &
    9.62 &
    1.84 &
    0.78 &
    24 \\
 & 3 & Low & 7.5 &
    7.21 &
    2.59 &
    1.09 &
    24 \\ \hline
Santi & 1 & Control & 10 &
    10 &
    0 &
    0 &
    16 \\
 & 1 & High & 10 &
    9.5 &
    1.41 &
    1.18 &
    8 \\
 & 1 & Low & 10 &
    8.88 &
    3.18  &
    2.66  &
    8  \\
 & 2 & Control & 10 &
    10 &
    0 &
    0 &
    16 \\
 & 2 & High & 10 &
    8.88 &
    3.18 &
    2.66 &
    8 \\
 & 2 & Low & 10 &
    10 &
    0 &
    0 &
    8 \\
 & 3 & Control & 10 &
    9.92 &
    0.45 &
    0.13 &
    48 \\
 & 3 & High & 10 &
    9.96 &
    0.2 &
    0.09 &
    24 \\
 & 3 & Low & 8.5 &
    8 &
    1.77 &
    0.75 &
    24 \\ \hline
Combined & 1 & Control & 10 &
    10 &
    0 &
    0 &
    32 \\
 & 1 & High & 10 &
    9.19 &
    2.4 &
    1.28 &
    16 \\
 & 1 & Low & 10 &
    8.5 &
    2.61  &
    1.39  &
    16  \\
 & 2 & Control & 10 &
    9.91 &
    0.53 &
    0.19 &
    32 \\
 & 2 & High & 10 &
    9.38 &
    2.25 &
    1.2 &
    16 \\
 & 2 & Low & 10 &
    9.12 &
    1.41 &
    0.75 &
    16 \\
 & 3 & Control & 10 &
    9.92 &
    0.45 &
    0.09 &
    96 \\
 & 3 & High & 10 &
    9.79 &
    1.3 &
    0.38 &
    48 \\
 & 3 & Low & 8 &
    7.6 &
    2.23 &
    0.65 &
    48 \\ \hline
\end{tabular}
\end{table*}


%%%% FIGURE APP %%%%
\begin{figure*}[ht]
\centering
\includegraphics[width=4in]{figures/apparatus_sm.jpg}
\caption{Experimental apparatus. Experimenters placed grapes in a circle within the bonobo's reach.  Once the bonobo's hand crossed the circle, the experimenter stopped placing grapes.}
\label{fig:APP}
\end{figure*}

%%%% FIGURE AGGREGATE %%%%
% R code

\begin{figure*}[ht]
\centering
\includegraphics[width=6in]{figures/bplot.pdf}
\caption{Choices aggregated over trials and subjects.  The number of grapes placed before stopping did not differ across the control, high-reliability, and low-reliability conditions in phases 1 (Generalized Expectation) or 2 (Social Competition).  In phase 3 (Direct Experience), however, subjects stopped earlier in the low-reliability condition.  Lines in boxes represent medians, diamonds represent means, boxes represent 25-75\% quantiles, and whiskers represent 1.5 times the interquartile range.} \label{fig:AGGREGATE}
\end{figure*}

%%%% FIGURE INDIV %%%%
% R code

\begin{figure*}[ht]
\centering
\includegraphics[width=6in]{figures/iplot.pdf}
\caption{Choices separated by subject and trial number.  Though Simon stopped early in all phases, Santi did not stop early until the direct experience of uncertainty in phase 3.  Phases 1 and 2 include one session, and phase 3 represents the mean over three sessions.  Dashed line represents optimal stopping point in phase 3.} \label{fig:INDIV}
\end{figure*}

\end{document}
