\documentclass[man,fignum,english]{apa}
\usepackage{babel}
\usepackage{Sweave}
\usepackage{lineno}


%opening
\title{Spatial discounting of food and social rewards in guppies (\emph{Poecilia reticulata})}
\author{Nelly M\"{u}hlhoff$^{1}$, Jeffrey R. Stevens$^{1*}$, and Simon M. Reader$^{2}$}
\affiliation{$^{1}$Center for Adaptive Behavior and Cognition, Max Planck Institute for Human Development, Berlin, Germany; $^{2}$Behavioural Biology, Department of Biology and Helmholtz Institute, University of Utrecht, Utrecht, The Netherlands.}

\shorttitle{Spatial discounting in guppies}
\rightheader{Spatial discounting in guppies}
\leftheader{M\"{u}hlhoff, Stevens, \& Reader}

\abstract{In temporal discounting, animals trade off the time to obtain a reward against the quality of a reward, choosing between a smaller reward available sooner versus a larger reward available later. Similar discounting can apply over space, when animals choose between smaller and closer versus larger and more distant rewards. Most studies of temporal and spatial discounting in nonhuman animals use food as the reward, and it is not established whether animals trade off other preferred stimuli in similar ways.  Here, we offered female guppies (\emph{Poecilia reticulata}) a spatial discounting task in which we measured preferences for a larger reward as the distance to it increased relative to a closer but smaller reward. We tested whether the fish discounted reward types differently by offering subjects either food items or same-sex conspecifics as rewards. Before beginning the discounting tasks, we conducted validation tests to ensure that subjects equally valued the food and social stimuli in the quantities provided. In the discounting task, subjects switched their preferences from the larger to the smaller reward as the distance to the larger reward increased (spatial discounting), but the pattern and magnitude of discounting did not differ across the two reward types. These findings indicate that guppies show similar patterns of discounting food and social rewards in a spatial task. In an analysis of travel times, however, the fish swam faster to food rewards than to shoaling partners. This difference in travel times implies that fish temporally discounted social rewards less steeply than food rewards.   Thus, reward type influences temporal discounting, suggesting a dissociation between temporal and spatial discounting. Our results illustrate how animals adjust choices and travel times depending on both the type of cost (time, distance) and benefits (food, social partners).\\
\textbf{Keywords}: discounting, grouping, intertemporal choice, reward types, shoaling, spatiotemporal choice}

\acknowledgements{We thank Alex Kacelnik for the suggestion to estimate the apparent size of the rewards at different distances; Henk Schriek, Ko van Rootselaar and Henk Westland for assistance with animal care; Utrecht University's High Potentials Programme and the Netherlands Organisation for Scientific Research (NWO) Evolution and Behaviour Programme for funding to SMR; and the Max Planck Society for funding to JRS.
}

\ifapamodeman{
\note{\begin{flushleft}
\textbf{*Correspondence:}\\
Jeffrey R.\ Stevens\\
Center for Adaptive Behavior and Cognition\\
Max Planck Institute for Human Development\\
Lentzeallee 94\\
14195 Berlin, Germany\\
jeffrey.r.stevens@gmail.com\\ \vspace{5mm}
   \end{flushleft}}
}
{
\note{Draft of \today \\}
\vspace{1cm}
}

\begin{document}

\maketitle 
\linenumbers

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Introduction
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Chacma baboons (\emph{Papio ursinus}) in South Africa will walk by less desirable food patches on the way to more desirable food \cite{noser_2007}.  This phenomenon represents a case of spatiotemporal choice, in which the baboons choose a higher quality reward delayed in time and at a greater distance over a lower quality, immediate reward.  Researchers have studied the temporal component of these choices (termed intertemporal choice) in a number of animal species, including honey bees, pigeons, starlings, chickens, blue jays, parrots, rats, monkeys, and apes \cite{ainslie_1974, tobin_1996, bateson_1996, richards_1997, stephens_2001, cheng_2002, green_2004, abeyesinghe_2005, stevens_2005a, evans_2007, rosati_2007, pearson_2010, vick_2010}.  Less research, however, has investigated spatiotemporal choice. 

Work on intertemporal choice demonstrates that species differ in their preferences for delayed rewards \cite{stevens_2009}.  Even within species, individuals vary in their preferences across contexts.  Blue jays (\emph{Cyanocitta cristata}), for example, choose delayed rewards more often when the choice is framed as continuing to forage in a patch or advancing to a new patch rather than a simultaneous choice between two options \cite{stephens_2001}.  In addition, chimpanzees (\emph{Pan troglodytes}) wait longer for a larger food reward when provided with toys than without toys \cite{evans_2007}.  Thus, the decision context can influence temporal preferences, and animals often employ ecologically rational decision strategies \cite{todd_2007}, depending on the environment.

An important issue that has not been explicitly investigated in nonhuman animal species is the type of reward used in the intertemporal choice.  Almost all studies to date have used food as the reward, and a few studies have used other consumables such as water and juice \cite{richards_1997, kim_2008, pearson_2010}.   Consumable rewards have important properties: organisms require them on a regular basis for survival yet have maximum limits of consumption.  Thus, the question of intake rate is important for decisions between options occurring at different times \cite{kacelnik_2003, stevens_2009}.  Other types of rewards have different properties that may influence intertemporal choices.  Studies in humans have directly compared various reward types.  Most studies of human intertemporal choice use money as a reward, and humans can wait rather long delays for money.  Yet, when choosing between food options, their preferences shift more towards immediate payoffs \cite{odum_2006, rosati_2007}.  Moreover, economists and psychologists have tested other currencies such as health outcomes and environmental outcomes \cite{chapman_1995, odum_2006, hardisty_2009}.  Again, the different currencies result in different preferences, perhaps due to currency-specific properties such as satiation and opportunity costs.

This study aims to test whether animals exhibit the same preferences with two different reward types.  Do animals make domain-general choices across currencies or do the specific properties of different currencies shape reward-specific preferences?  To investigate this question, we tested two relevant and rewarding stimuli for domestic guppies (\emph{Poecilia reticulata}): food and conspecifics. Food is a well studied reward for fish.  Access to conspecifics is also likely to be rewarding to many fish, in light of their strong preferences to form groups or `shoal' \cite{krause_2002}.  In fact, visual access to non-aggressive conspecifics can act as a reinforcer for fish \cite{al-imari_2008}. We did not, however, offer the guppies a standard intertemporal choice task.  Instead, we offered them a ``spatial discounting'' task in which they chose between a smaller, closer and larger, more distant reward. Discounting refers to a mechanism of choice in which the subjective value of a reward decreases as some form of cost increases.  In temporal discounting, value of a reward decreases as the time delay to receiving it increases.  In spatial discounting, value decreases as the distance required to travel to that reward increases \cite{smith_1975, perrings_2001}.  If animals always choose the more preferred food reward, a sigmoidal preference pattern is predicted.  At short distances to the larger reward, we predict a strong preference for it.  As the distance continues to increase, the value of the larger reward decreases because the cost to access it increases. At the point where the values of the smaller and larger rewards are equal, the subject is predicted to become indifferent between the two options. As soon as the value of the smaller reward exceeds that of the larger reward, the subject is predicted to prefer the closer option. Thus, animals are predicted to prefer the larger reward up to the indifference point, and then switch to prefer the smaller, closer reward: a sigmoidal pattern. 

The question of spatial choice is not separate from temporal choice because time is typically embedded in traveling: farther distances take longer times to travel.  Thus, though we refer to this as a spatial choice, it remains a spatiotemporal choice.  Studying spatial choice in animals is important for two reasons.  First, they frequently face these choices in nature: go a short distance to a less desirable food or travel further to a more desirable food.  This type of spatial choice has been tested experimentally with primates in the field and the laboratory.  In the field, Janson \citeyear{janson_2007} varied the amount of food available at feeding platforms distributed throughout the home range of brown capuchin monkeys (\emph{Cebus apella nigritus}).  Like chacma baboons \cite{noser_2007}, capuchins bypassed lower quality food for more distant, higher quality food.  In a laboratory task, cotton-top tamarins (\emph{Saguinus oedipus}) and common marmosets (\emph{Callithrix jacchus}) made binary choices between traveling to a smaller, closer reward or a larger, more distant reward \cite{stevens_2005b}.  The marmosets switched from the larger to the smaller reward as the larger reward was moved farther away.  The tamarins, in contrast, traveled to the larger reward over all distances used in the experiment.  Second, these spatial choices can offer more naturalistic examples of decision making.  Though there are clear cases in which animals must wait for time delays, such as hunting and caching \cite{stevens_2009}, they may not frequently face discrete choices between simultaneously available options.  Instead, animals may face more sequential choices, such as when to leave a patch of food \cite{stephens_2001,shapiro_2008}.  Simultaneous choice may be more common in the spatial domain.  Despite the ubiquity of spatial choice in animals, this is rarely tested in experimental studies of choice. 

Fish are a particularly good system for studying spatial choices because numerical discrimination and spatial distances have been tested in a number of species \cite{tegeder_1995,agrillo_2007b,buckingham_2007,shapiro_2009}.  We tested guppies because they must make choices about both food and shoaling partners.  In the wild, guppies feed mainly on algae and invertebrate larvae \cite{dussault_1981, magurran_2005}, both rather immobile food sources.  Thus, they likely face situations in which they must choose between patches of food differing in quality and distance.  Though clearly relevant for foraging, these spatial choices apply also in the social domain.  Many species, especially fish, prefer larger groups over smaller ones \cite{hager_1991, ashley_1993, krause_1994, pritchard_2001} due to anti-predator and other benefits that larger groups can provide \cite{krause_2002}.  For example, female guppies prefer to follow a larger over a smaller shoal of fish \cite{lachlan_1998}.  When moving, subgroups may emerge, requiring individuals to choose which group to join.  If these groups are at different distances from the individual, this becomes a spatial choice. \citeA{tegeder_1995} have demonstrated that three-spined stickleback (\emph{Gasterosteus aculeatus}) will travel farther to approach larger shoals.

In this experiment, we tested whether guppies spatially discount differently depending on the reward type. We offered subjects choices between a smaller reward at a fixed and close distance versus a larger, more distant reward.  We varied the distance to the larger reward to measure spatial discounting.  In one condition, subjects chose between numbers of food items (`food rewards') and in another condition between numbers of shoaling partners (`social rewards').  We use the term `shoaling partner' to maintain generality of the concept; however, this does not imply that the fish had previous interactions.  We predicted that guppies would demonstrate spatial discounting of both reward types, due to the costs of traveling to a more distant reward. 


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Methods
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Methods}
% R code

\subsection{Subjects}
We tested domestic guppies (\emph{Poecilia reticulata}) bred in the Biology Department Aquarium, Utrecht University, The Netherlands from August 2009 to February 2010. We used a single sex to avoid mating interactions during the experiment.  Due to their slower satiation rates and higher shoaling tendencies than males \cite{magurran_1994, dussault_1981}, we used only females, all of approximately equal body size (mean $\pm$ standard deviation body mass = 0.56$\pm$0.04 g) as subjects and shoaling partners. From an initial group of 56 fish, we selected 19 individuals as subjects on the basis of their active participation and performance in the training and evaluation phases of the experiment (see below). Fourteen subjects completed the experiment---six completed both reward-type conditions and eight completed one condition.  We identified individual subjects by their distinctive coloration.  Shoaling partners came from a pool of 80 guppies. Shoaling partners and subjects were reared in separate tanks since birth and thus were unfamiliar with each other. For the visual control task (see below), we tested two subjects from the main experiment, plus three na\"{i}ve subjects.  

All fish were housed in 90 x 40 x 25 cm tanks containing copper-free water (depth 20 cm) for at least three days before beginning training. All tanks were maintained at a water temperature of $25\pm2^\circ$C and exposed both to a 12/12 h artificial light/dark cycle, with lights on at 07:00 h, and to natural daylight. We housed shoaling partners and subjects in separate tanks during the experiment. The shoaling partners received standard tropical fish flaked food (TetraMin, Tetra, Melle, Germany) twice a day.  The experimental subjects received shrimp paste (Tetra FreshDelicia Brine Shrimps) during the experiment and standard tropical fish flaked food one hour after the last subject finished its daily experimental session. The experiment was approved by the Utrecht Ethics and Animal Care and Use Committee under protocol number DEC 2009.I.06.045 and conforms to the Animal Behavior Society/Association for the Study of Animal Behaviour Guidelines for the Treatment of Animals in Behavioral Research and Teaching.

\subsection{Materials}
The testing apparatus consisted of a 160 x 40 x 20 cm rectangular aquarium (water depth: 17 cm). The tank had a white back wall and contained white gravel to provide contrast between the fish and the background for videotaping from above. We attached sliders, each with two slots, to the inside of the tank walls at 20 cm increments (Figure \ref{fig:tank}). 

% Figure 1
\begin{figure*}[ht]
\centering
\includegraphics[width=5in]{figures/muehlhoff_fig1.pdf}
\caption{Experimental tank, plan view.  We presented subjects with a choice between two versus six food items (food-reward condition) or two versus six same-sex conspecifics (social-reward condition). The 160 x 40 x 20 cm tank included vertical sliders every 20 cm to allow insertion of barriers at different distances from the subjects' starting location.  We attached food items or a container containing conspecifics to the barriers.   We could thus vary the distance to the more numerous food items or shoal.  Figure illustrates food-reward condition with the food items (F) both at 20 cm from the starting location (S). Numbers indicate the distance to each reward.  During the intertrial interval, the subject was in a transparent cylinder 20 cm from the smaller reward.}
\label{fig:tank}
\end{figure*}

For an experimental session, rewards were placed at one of the positions shown in Figure \ref{fig:tank}.  In the food-rewards condition, we placed 2 x 1 cm strips of green electrical tape on white plastic barriers (37.5 x 23.5 x 0.25 cm) as markers for each food item. These rectangles roughly matched the size of the fish, thus equating the visual surface area covered by shoal fish and food reward markers. For each quantity of food items, we placed the same irregular pattern of rectangles on the plate. We placed small amounts of the shrimp paste, a highly preferred food item, in the center of the green rectangles using a single-channel pipette (Gilson Pipetman) and allowed them to dry before placing them in the water. We placed food rewards in the tank by inserting the feeders into the sliders mounted inside the tank at the respective positions.  We inserted white plastic plates behind the reward barriers in the second slider slot. These `back walls' stayed at their positions during the entire condition and thus served as a neutral background at the reward sites even when the feeder plates were removed. In the social-rewards condition, rewards consisted of shoals of other female guppies within transparent plastic containers (13 x 10 x 12 cm). The boxes attached to the back walls via two metal hooks. Since odors could diffuse from the food items, the shoal containers had small holes to allow dispersion of odor cues from the fish as well.

We transferred experimental fish from their housing tank to the testing tank using a net. We placed subjects at the starting point within a transparent, plastic cylinder (9 cm in diameter). This cylinder allowed free rotational movement of the fish, thus giving the subject an opportunity to orient towards the favored reward side.

We used reward amounts of two and six food items and two and six shoaling partners for four reasons: 1) previous discounting tasks with primates used these amounts \cite{stevens_2005a, stevens_2005b, rosati_2006, rosati_2007}, 2) these quantities are within the numerical discrimination ability of other fish species \cite{agrillo_2007a, agrillo_2007b, agrillo_2008, buckingham_2007, gomez_2011}, 3) the items could be consumed in a relatively short time without excessive satiation, and 4) the shoal sizes fall within those observed in wild guppy populations \cite{magurran_1991}.  We measured time intervals with a standard stopwatch and videotaped all sessions with a wide-angle web cam (Philips SPC10300NC) mounted above the testing tank.

\subsection{Procedure}
Subjects experienced four phases in this experiment.  First, they were habituated and trained to the testing apparatus.  Second, they completed an evaluation phase in which we measured the relative value of food and social rewards.  Third, the subjects participated in the spatial discounting task.  Fourth, they completed a visual control task that determined whether subjects could visually discriminate the rewards at large distances. In testing sessions (i.e., all phases except the training phase), subjects experienced 10 trials in a session: two forced-choice trials followed by eight free-choice trials.  In forced-choice trials, only one option was available, whereas in free-choice trials, both options were available. Subjects experienced one testing session per day and approximately five sessions per week.

\subsubsection{Phase 1: Habituation and training}
Prior to the experiment, we trained all subjects to the feeders for four days by feeding them exclusively from feeder plates placed in their housing tanks. Each day, we inserted one feeder plate at a time (with two or six food rewards), then switched to the other reward size.  The order of presentation was randomized each day.  During this phase, we selected individuals that responded strongly to the feeders (i.e., directly and quickly swam to the green food markers as soon as a feeder was placed in the tank). These selected subjects then experienced a test session individually in the testing tank. The test session consisted of two forced-choice trials in which we only placed the six food items in the tank followed by two free-choice trials in which they could choose between two and six food items.  We presented both food amounts at a 20 cm distance from the starting point. To advance to the main experiment, subjects had to consume all six food items in both forced-choice trials within one minute and choose the larger reward in at least one of the free-choice trials.  Thus, we selected fish that fed in the testing tank and had experience choosing the larger over the smaller food reward. Twenty-two of the 56 fish (39\%) passed the selection criterion and entered the evaluation phase. Thus, the generality of our findings is restricted to a subset of the population.

Shoaling partners were habituated by placing them in batches of six individuals in a container in the testing tank for one training session per day for four days. Shoaling partner training sessions began with an initial duration of five minutes. The following days, we doubled the duration, resulting in a total duration of 40 minutes after four days, which matched the maximum time per day they would spend in the container during the experiment. Shoaling partners did not show stress responses \cite{smith_1992} such as fast movements or immobility during the experiment

\subsubsection{Phase 2: Evaluation phase}
Before beginning the spatial discounting task, we wanted to ensure that any differences in discounting responses to the two reward types conditions did not result from individuals differentially valuing the reward types. That is, a comparison of how subjective value changes over time and space is only possible when the different rewards have the same immediate value for an individual. To achieve this, subjects experienced an evaluation phase in which they chose between the same number of food and social rewards. 

Before each session, shoal fish and subjects acclimated to the testing tank for three minutes. During this period, subjects were placed in a transparent plastic cylinder at the starting point. A daily session consisted of two initial forced-choice trials followed by eight free-choice trials. In the forced-choice trials, the subject were given one reward type on one trial and the other reward type on the other trial, with order and sides counterbalanced. In the free-choice trials, the subject could choose between two different rewards that were simultaneously presented at different ends of the tank, with the sides counterbalanced. Trials were separated by an intertrial interval of 30 s. Twenty seconds into the intertrial interval, the experimenter placed both rewards in the tank simultaneously and waited for another 10 s before releasing the fish. The experimenter released subjects by gently pulling the cylinder out of the water. Subjects then had 60 s to approach a reward. If the fish crossed a line 5 cm from a reward, a choice for that reward side was recorded (see Movie 1 in Supplemental Materials). Once the fish made a choice, the experimenter removed the other option from the tank, preventing the fish from receiving both options. The fish then had 90 s to consume the food or 20 s to stay near the shoal. After this period, the experimenter removed the feeder/shoal from the tank and the subject was coaxed back to the starting position using the transparent plastic cylinder. As soon as the subject reached the starting position, the intertrial interval began. If the fish failed to make a choice within 60 s, we considered the trial invalid and terminated it. If a session contained one or more invalid forced-choice trials and/or two or more invalid free-choice trials, we did not use that session for analysis and retested the fish under the same conditions the following day. To ensure all subjects experienced the same testing procedure, we completed sessions even if they reached the criterion for an invalid session.

Nineteen of the 22 subjects that began the evaluation phase completed this phase. We excluded three subjects that showed stress responses. About half of the subjects ($N = 9$) initially received testing sessions in which they chose between two food pieces and two shoaling partners, each at a distance of 20 cm from the starting position. The remaining subjects ($N = 10$) initially chose between six food items and six shoaling partners.  We tested whether subjects had a preference for one reward type, and, if necessary, we could alter the quantity of one reward type until subjects exhibited similar preferences for both types.  A subject completed the evaluation phase for two or six stimuli when it completed two consecutive sessions demonstrating no strong preference for one reward type over the other (i.e., it chose one reward type five or fewer times in the eight free-choice trials).  Once a subject completed the two-stimuli evaluation phase, it proceeded to the six-stimuli evaluation phase, or vice versa. 

In fact, it was not necessary to adjust the reward quantities because all subjects chose equally between two food pieces and two shoaling partners (mean percent choosing food$\pm$CI: 54$\pm$4\%), as well as between six food pieces and six shoaling partners (mean percent choosing food$\pm$CI: 51$\pm$3\%). Thus, all subjects chose between two versus six food pieces and two versus six shoaling partners in the spatial discounting task. Upon completing evaluation phases for both reward sizes, subjects proceeded to the spatial discounting task. 

\subsubsection{Phase 3: Spatial discounting task}
The spatial discounting task used the reward values established from the evaluation phase (two versus six rewards for all subjects). Half of the subjects began with the food rewards and half began with the social rewards. Here, we tested each subject at six distance increments to the larger reward. Subjects experienced two consecutive sessions per distance with the distance increments constantly increasing. Testing sessions were structured in an identical way to the evaluation phase.  

For the first sessions, both rewards were placed at the smallest distance of 20 cm from the subject's starting position. This 20 cm distance is further than the distance which would be considered as shoaling \cite<five body lengths or approximately 10 cm:>[]{magurran_1994}.  To ensure that subjects discriminated the reward sizes, they had to choose the larger reward in at least six of the eight free-choice trials (or five out of seven free-choice trials with one invalid trial) for three consecutive sessions at the 20 cm distance. Subjects required a mean$\pm$standard deviation of 4.3$\pm$1.5 sessions to complete this criterion.  After demonstrating discrimination of the reward amounts and a preference for the larger reward at this initial distance, in the next session, the experimenter moved the larger reward to the next distance, but the smaller reward remained at 20 cm. This continued until the subject had experienced all six distances (20, 40, 60, 80, 100 and 120 cm) to the larger reward for two consecutive sessions. Following completion of the initial reward type condition, subjects directly switched to the other reward type, for a total of 12 social-reward sessions and 12 food-reward sessions.

Before the experiment began, we randomized the order in which we tested subjects over a day.  We maintained this order for the entire experiment. Thus, each subject was tested at approximately the same time each day during evaluation and discounting trials, in an attempt to control for possible variation in feeding and shoaling motivation over the day. A daily session consisted of 10 trials at the same reward distance. We randomly assigned the side of the testing tank for smaller and larger rewards for each session. For each individual, we measured the percentage of choices for the larger reward at each distance, taking the mean over the two sessions. In the social-reward condition, we selected new groups of shoaling partners at random for each subject each testing day, with shoaling partners used a maximum of once per day. 

Our primary measure of interest was percent choice for the larger option.  However, we also assessed the subjects' travel times by measuring the time from the point of release to the choice line.  For each subject, condition, and distance, we randomly selected two free-choice trials from our video recordings in which the subject chose the larger reward. As ten subjects completed each condition (i.e., six completed both conditions, four the food-reward condition only, and four the social-reward condition only) and there were six distances, this resulted in 240 recordings..  If no free-choice trials were available, we used forced-choice trials.  Due to a technical problem, we did not have video for the 20 cm distance of the first condition for each subject.  For these values, we used choices for the smaller reward (at 20 cm) in the subsequent session in which the larger reward was at a 40 cm distance.

\subsubsection{Phase 4: Visual control task}
We conducted a visual control task to ensure that the subjects could visually discriminate the smaller and larger rewards at the farthest distance. This would exclude the possibility that, if trade-offs over distance were observed, they simply resulted from diminished visibility of the more distant rewards. A pilot study ($N =$ 5) had established that subjects overwhelmingly chose six items at 120 cm over no reward at 20 cm (97$\pm$3\% of choices for the larger reward). Subjects showed similar responses in the food (96$\pm$5\%) and social reward conditions (98$\pm$3\%). 

For the visual control task, we tested two subjects that completed the spatial discounting task (S21 and S22).  In addition, we tested na\"{i}ve fish recruited specifically for this task.  Only three of 30 fish (10\%) passed the selection criterion. We offered the five subjects a choice between smaller and larger rewards at 120 cm distance.  We separated the 160 cm-long testing tank lengthwise into two equally wide sections with a 120-cm-long opaque partition (Figure \ref{fig:visual}). Subjects were placed in a transparent compartment that allowed visual access to both rewards and then released by removing a transparent plastic barrier. Once released, the subjects could swim on the left or right side to access the smaller or larger reward. We counterbalanced the sides of the smaller and larger reward amounts between trials.

% Figure 2
\begin{figure*}[ht]
\centering
\includegraphics[width=5in]{figures/muehlhoff_fig2.pdf}
\caption{Visual control task tank, plan view. We presented subjects with a choice between two versus six food items (F, upper panel) or two versus six same-sex conspecifics (C, lower panel). The 160 cm experimental tank was divided lengthwise by an opaque partition (solid line).  Subjects remained in a triangular starting space (S) behind a solid, transparent barrier (dashed line) during the intertrial interval.  Upon removing the barrier, the subject could swim 120 cm to either the smaller or larger reward.}
\label{fig:visual}
\end{figure*}

To familiarize the subjects with the new setup and task, initially the experimenter presented both reward quantities on either side of the tank at 20 cm from the subjects' starting point. Once the subjects showed a clear preference for the larger reward (75\% or more choosing the larger amount per session) for two consecutive sessions, the experimenter moved the reward quantities to the 120 cm distance for another two sessions. Sessions consisted of 10 trials and followed a similar structure to the discounting task.  We first conducted this procedure using food rewards and then repeated with social rewards.  A reward size preference in this task would demonstrate that the subjects could discriminate the different reward amounts at the farthest distance used in the discounting task.

All subjects showed a clear preference for the larger reward with a mean$\pm$CI of 84$\pm$3\% choices (range = 75-100\%) in all free-choice trials over both food and social conditions.  Again, behavior in the food (86$\pm$6\%) and social reward conditions (81$\pm$0\%) was similar. Despite the small sample size ($N = 5$), the consistency across individuals and the fact that the confidence intervals do not span 50\% suggest that, at a distance of 120 cm, the subjects visually discriminated two from six rewards for both food items and shoaling partners.

\subsection{Statistical analysis}
For descriptive statistics, we report means$\pm$95\% confidence intervals (CI).  For effect sizes, we calculated generalized eta squared $\eta^{2}_{G}$ \cite{bakeman_2005}.  We analyzed the data using R statistical software version 2.12.2 \cite{r_manual_2010} and the epicalc \cite{epicalc_2010}, Hmisc \cite{hmisc_2010}, lattice \cite{sarkar_2008}, and psych \cite{psych_2010} packages.  Data and R code are available in the Supplementary Materials and on the Dryad data repository (http://datadryad.org/).  The original \LaTeX \hspace{0.1mm} document, with Sweave-embedded R code \cite{sweave_2002} to allow reproduction of analyses \cite{de_leeuw_2001}, is available from JRS.  

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Results
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Results}
The spatial discounting experiment involved extensive testing per subject, and several subjects did not complete all tests.  Six of the 19 subjects that participated in the discounting task completed both the food and the social conditions. Eight more subjects finished only one of these conditions, with four subjects in the food and four in the social condition. Thus, 14 fish completed at least one condition, and 10 completed each condition.  The other five subjects did not complete any conditions because they stopped making choices, stopped consuming food, or died in the course of the experiment.

\subsection{All-subjects analysis}
We begin with analysis of all 14 subjects, including those that only completed one of the two conditions.  Figure \ref{fig:allchoice} illustrates two interesting results.  First, the subjects' preferences for the larger reward declined as the distance to it increased.  When the subjects could choose between a smaller and larger reward at the same distance of 20 cm, they chose the larger one in 81$\pm$6\% of the trials in the food condition and 83$\pm$6\% of the trials in the social condition (pooled over all 14 subjects). At the farthest distance increment (120 cm), subjects chose the larger reward less often, in only 17$\pm$6\% of the trials in the food condition and 18$\pm$6\% of the trials in the social condition. Thus, the fish showed evidence of spatial discounting.  They did not, however, show a sigmoidal preference function as predicted by discounting.  Instead, the preference function appears to be more linear.  Because aggregating different sigmoidal responses over subjects can result in a linear pattern, Figure \ref{fig:subjchoice} shows the data for individual subjects.  Here we see, with the possible exceptions of two subjects (S2 and S14), that  most subjects' response patterns were approximately linear. 

% Figure 3
\begin{figure*}[ht]
\centering
\includegraphics[width=6in]{Rwork/figures/muehlhoff_fig3.pdf}
\caption{Effect of reward type on spatial discounting using aggregate data. We presented subjects with a choice between two versus six food items (food-reward condition, blue circles) or two versus six same-sex conspecifics (social-reward condition, yellow triangles). The smaller reward remained at 20 cm. Choice for the larger reward decreased with the distance to the larger reward. The linear regression equations (not illustrated) for the food and social conditions were $y = -0.5x  + 82.9$ and $y = -0.6x  + 90.6$, respectively.
We used all subjects in this analysis ($N = 14$), thus including subjects that experienced both conditions ($N = 6$) and subjects that experienced only one condition ($N = 8$).}
\label{fig:allchoice}
\end{figure*}

Because the two reward options varied in their distance from the subject, they also varied in their apparent size on the retina.  It is possible that the fish might only use the simple cue of apparent size to make their choices.  The points of indifference (the distance at which subjects choose smaller and larger rewards equally) and the shape of the discount function, therefore, could result from the difference in apparent sizes on the retina of the two reward options rather than on an evaluation of distance.  To explore this, we calculated the total visual area of the rectangular food markers for the two close rewards and the six rewards at each distance. Because we used the same number of smaller and larger rewards for both food and social rewards, the relationship calculated for the food reward also holds for the social reward.  We calculated the retinal area $A$ for one marker as $A = \tan(2\arctan(\frac{w}{2d})) * \tan(2\arctan(\frac{h}{2d}))$, where $w$ = width (2 cm), $h$ = height (1 cm), and $d$ = distance (20, 40, 60, 80, 100, 120 cm).  We then multiplied the area by the number of markers (two and six) and compared across reward sizes.  First, the retinal area of the closer reward exceeded that of the larger reward when the larger reward was farther away than 35 cm.  Therefore, if retinal area constrains their choices, the subjects should show indifference between the 20 and 40 cm distances.  Instead, linear regressions of the aggregated data showed indifference points of 70.6 and 69.2 cm for the food and social conditions, respectively.  Second, the difference between the total retinal areas decreased hyperbolically with distance rather than linearly.  Thus, the linear pattern of discounting likely does not result from the difference in retinal area.  

The second striking result is that preferences are similar between the two reward type conditions (Figure \ref{fig:allchoice}).  There is a slight difference around 80-100 cm, but examination of individual preferences suggests that two unusual subjects (S2 and S14) primarily drove this effect (Figure \ref{fig:subjchoice}).  The other subjects behaved similarly regardless of the reward type. Moreover, they behaved very similarly to one another.

% Figure 4
\begin{figure*}[ht]
\centering
\includegraphics[width=6in]{Rwork/figures/muehlhoff_fig4.pdf}
\caption{Effect of reward type on spatial discounting for each subject. Choice for the larger reward decreased with the distance to the larger reward for each subject both in the food (blue circle) and social conditions (yellow triangles).  Six subjects experienced both conditions, and eight subjects only experienced one condition.}
\label{fig:subjchoice}
\end{figure*}


\subsection{Within-subjects analysis}
To test the effect of distance and reward type with inferential statistical analysis, we restricted the sample to subjects that completed both reward-type conditions ($N$ = 6).  We analyzed the data using repeated-measures analysis of variance (ANOVA), with distance and reward type as within-subject factors. We arcsine square-root transformed the data for the ANOVA to correct for a slightly non-normal distribution of residuals (Shapiro-Wilk normality test on raw data: $W$ = 0.97, $p$ = 0.07; arcsine, square-root transformed data: $W$ = 0.98, $p$ = 0.41; Levene test of homogeneity of variance on raw data: $F$ = 1.36, $p$ = 0.21; arcsine, square-root transformed data: $F$ = 1.5, $p$ = 0.16).

The frequency of choosing the larger reward strongly decreased with distance (ANOVA: $F$(5,25) = 27.3, $p$ < 0.01, $\eta^{2}_{G}$ = 0.67), but there was no main effect of reward type ($F$(1,5) = 1.9, $p$ = 0.22, $\eta^{2}_{G}$ = 0.07). There was, however, an interaction between distance to the larger reward and reward type ($F$(5,25) = 3.7, $p$ = 0.01, $\eta^{2}_{G}$ = 0.17).  Again, this difference emerged from the higher preference for the larger food reward at the 80 and 100 cm distances for subjects S2 and S14.  

\subsection{Travel times}
Spatial discounting did not vary across reward types for most subjects. However, the time required to travel to the larger reward may vary with reward type.  As expected, the mean and median travel time increased as the distance to the reward increased (Figure \ref{fig:traveltime}).  In addition, the travel time depended on the reward type.  Travel time increased relatively slowly with distance in the food reward condition but increased more quickly in the social reward condition.  Consequently, subjects took longer to reach social rewards. For example, at a distance of 120 cm, subjects reached food rewards in 60\% of the mean time taken to reach the social rewards.

% Figure 5
\begin{figure*}[ht]
\centering
\includegraphics[width=6in]{Rwork/figures/muehlhoff_fig5.pdf}
\caption{Travel time as a function of distance to reward and reward type. Travel time increased with the distance to the reward.  As the distance increased, subjects swam to food rewards faster than to social rewards.  We used all subjects in this analysis ($N = 14$). In the boxplots, diamonds represent the mean, lines represent the median, boxes represent the interquartile range, and whiskers represent 1.5 times the interquartile range.}
\label{fig:traveltime}
\end{figure*}

Since the fish showed similar spatial discounting across reward types but varied in the time required to access the different rewards, this suggests that they may temporally discount food versus social rewards differently.  That is, if we look at choice as a function of travel time rather than distance, we would expect differences across the reward types since travel time varied across reward type.  Figure \ref{fig:timedisc} plots choice as a function of travel time for both reward types. Subjects chose a larger reward less often at longer travel times. Thus, guppies demonstrated not only spatial discounting but also temporal discounting.  Although preferences decreased with travel time at the same rate across reward types (i.e., the slopes of the regression lines are approximately parallel), at any given travel time, the subjects chose the larger option more in the social reward condition (i.e., the social condition regression line has a higher intercept than the food condition regression line).  For instance, in the food condition, subjects were indifferent between smaller and larger rewards at a travel time of approximately 13.1 s: the food-condition regression line crosses 50\% preference at 13.1 s.  In the social condition, in contrast, subjects were indifferent at 23.4 s.  We found similar results when excluding subjects that showed possible reward type differences in the spatial task. Thus, subjects swam for longer to reach a larger shoal than to reach a larger food reward.  Social rewards appear to hold their value over longer times, suggesting that the fish temporally discounted or devalued the social options less steeply than the food rewards.

% Figure 6
\begin{figure*}[ht]
\centering
\includegraphics[width=6in]{Rwork/figures/muehlhoff_fig6.pdf}
\caption{Preference for larger option as a function of travel time and reward type. Data points represents the mean travel time and mean preferences for smaller versus larger option for each subject, distance, and reward type.  Because each subject is represented multiple times, the data points are non-independent.  Solid lines represent linear regression equations for food ($y = -1.0x  + 63.1$, blue) and social ($y = -1.1x  + 74.8$, yellow) conditions.  The dashed line represents choice indifference. We used all subjects in this analysis ($N = 14$).}
\label{fig:timedisc}
\end{figure*}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Discussion
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Discussion}
In this study, we tested how reward type influences spatial discounting in guppies by offering them a choice between a smaller, closer versus larger, more distant reward using food items and shoaling partners as rewards.  For both reward types, the percentage of choices for the larger reward decreased with increasing distance. This finding matches Stevens et al.'s \citeyear{stevens_2005b} results on spatial discounting in marmosets.  Additionally, aggregating the data over subjects shows little difference in the choice patterns between the two reward type conditions. Most subjects chose similarly for food rewards and social rewards.  This does not necessarily exclude the possibility that guppies discount differently depending on the nature of the reward; however, the aggregate choice data and the data for most individuals did not reveal different patterns across reward types. Yet, some individual differences emerged from our data.  Though most subjects exhibited similar patterns of choices for both reward types, two appeared to travel farther for the larger food rewards compared to the social rewards. Further studies with a larger sample size and variation in the relevant factors could test the robustness of such individual differences.

Though discounting of food and social rewards did not differ over space for most individuals, it did differ over time.  Subjects reached food rewards more quickly than social rewards.  When analyzing choices as a function of travel time rather than distance, the guppies traveled for longer times for the social rewards.  Therefore, they temporally discounted social rewards less than food rewards.

In the evaluation phase before the discounting tasks, we offered subjects a choice between the reward types (e.g., two food items versus two shoaling partners) to equate the value of the reward types.  All subjects showed no preference for the food over the social reward type when the rewards were the same quantity. This finding suggests that food and social rewards hold equal values when available immediately in this experimental context. Therefore, if we were to find any differences in the discounting task, this could not be attributed to a difference in immediate value rather than a difference in the effect of space or time on choice.

The trade-off observed in the discounting task could result from diminished detection of the rewards at greater distances. However, the visual control data indicate that guppies visually discriminated the reward amounts used in this study at a distance of 120 cm. Discrimination on the basis of odor is unlikely given the distance of the rewards and that reward locations were counterbalanced across trials. Thus, guppies can distinguish the number of shoaling partners and food patches over the largest distance used in this experiment, suggesting that subjects indeed made choices based on the quantity of and distance to the rewards.  Moreover, the pattern of discounting we observed in guppies could not have resulted only from differences in apparent size on the retina as distance increased because visual area decreases hyperbolically rather than linearly, with a much earlier predicted indifference point.

The shape of the discounting curves demonstrated by the fish was surprising.  We predicted a threshold response in which fish preferred the larger reward until a point where they abruptly switched to preferring the closer reward. Instead, most individual discounting curves were linear. Discounting curves sharply dropped between the equidistant rewards (20 cm) and the first distance increment (40 cm). The early drop off might result from the fact that subjects experienced more than two sessions with equidistant reward amounts, a procedure conducted to familiarize the subjects with the task and establish a preference for the larger reward. Since we trained subjects in the equidistant situations for several sessions, a sudden increase in distance to the larger reward may have been perceived as proportionally larger. Interestingly, \citeA{tegeder_1995} and Stevens et al.\ \citeyear{stevens_2005b} found similar initial drop offs with stickleback and marmosets in spatial discounting tasks.

Future studies should develop models that incorporate perceptual and neurocognitive mechanisms to explore the shape of the choice function. Although we have referred to this task as a spatial discounting task, we have not established the mechanism underlying the results we observe: discounting is only one of several possible mechanisms that could be used to solve this task.  We excluded the possibility that the pattern observed was directly related to the apparent size of the rewards on the retina, but other mechanisms could be acting.  Underlying mechanisms are rarely established in studies of animal and human intertemporal choice and remain a topic of debate  \cite{scholten_2010,stevens_2011}.  

\subsection{Travel time and costs}
Most guppies did not differ between reward types in their preference for larger rewards at each distance provided.  They did differ between reward types, however, in the time required to make a choice by swimming faster to food rewards than to social rewards. Though we did not test temporal discounting directly by imposing a delay for a reward, instead measuring travel time, our data suggest that the fish temporally discounted the two reward types differentially. Examining choice as a function of the travel time (ignoring spatial distance), we see that the guppies would not wait as long for food as for shoaling partners. This indicates a potential dissociation between spatial and temporal discounting in this species and raises the possibility that guppies temporally discount consumable versus non-consumable rewards differently. 

Our data suggest that guppies temporally discount the rewards differently even though they value them equally, as demonstrated by the evaluation phase.  This suggests possible differences in either motivation or decision mechanisms used to make foraging versus shoaling choices. Subjects swam faster to food than to a shoal, perhaps due to the competitive nature of finding food \cite{laland_1999b, griffiths_2003}.  Latecomers to a food source may miss out on foraging opportunities, whereas a shoal does not devalue over time in this manner.  Additionally, to reach a distant, larger shoal, subjects had to leave an area relatively close to the small shoal. That is, in the shoaling tests they had to leave an area of relative safety to cross an open area, and thus may have been hesitant to do so, potentially increasing their travel times. Another possible explanation is that shoals are more dynamic and potentially more ambiguous stimuli than are food items. A shoal is constantly moving, with individuals possibly obscuring each other. The guppies' perception of the group may have been constantly changing, or guppies may have hesitated when not followed by the small shoal, both possibly leading to increased travel times. In contrast, the feeders used in this study remained a stable, immobile cue to which the guppies had been pre-selected to respond. 

The results suggest that animals may disassociate travel time and travel distance in some circumstances. For example, encounter rates with ambush predators may increase with distance traveled but not necessarily with time away from a safe area. Encounter rates with other predators, in contrast, may increase with both time out of safety and the distance covered. Moreover, there may be trade-offs between vigilance and travel rates that vary with predator type \cite{trouilloud_2004}. 

Energetic considerations might also have influenced the spatial choices. In temporal discounting, only time and reward amount are relevant aspects of a decision. But when dealing with choices over space, more is at stake. Indeed, traveling a distance to a reward takes a certain time, but it also involves metabolic costs, which are minimal in temporal discounting tasks. Starlings (\emph{Sturnus vulgaris}), for example, adjust their locomotion type (walking or flying) to the energetic value of a reward in relation to the expenditure required to obtain it \cite{bautista_2001}. Aged honeybees (\emph{Apis mellifera}) with damaged wings that can only perform a limited number of wingbeats accept inflorescences of poorer quality and with fewer flowers, suggesting that they reduce the amount of time they spend in flight \cite{higginson_2004b}. Thus, animals attend to the costs associated with travel and will accept smaller rewards to avoid travel. 

The role of energetics has also been explored in the motivation and economics of demand literatures \cite{lea_1978,dawkins_1990,niv_2006}.  The amount of work expended for and the rigor of responses directed to a reward signals the value of that reward.  This is often measured experimentally by increasing the number of operant responses required before receiving a reward or by increasing the amount of effort that an animal must exert (e.g., displace weights) to receive a reward.  In many cases, the reward in question is food, but this literature has considered other reward types, such as aggression and access to water, nest sites, larger space, and toys \cite{hogan_1970,sherwin_1996,mason_2001}.  For example, domestic pigs ({\em Sus scrofa}) required to press a plate repeatedly to access a reward responded more often for food than for social contact \cite{matthews_1994}. Very similar results were reported in Siamese fighting fish ({\em Betta splendens}) trained with food or the opportunity to make aggressive displays \cite{hogan_1970}.  Thus, there is evidence that animals value social and consumable reward types differently when the cost involves physical effort. We observe some parallels with our results, with guppies swimming faster to food than social rewards, but also differences, with subjects becoming indifferent for food sooner than for social rewards.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Conclusion
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Conclusion}
This study suggests that guppies discount food and social rewards similarly over space but not over time.  Though spatial choices were similar for food and social rewards, this is likely sensitive to the reward amounts, distances, order of presentation, motivation levels, and perceived predation risk, among other relevant factors.  

The difference in travel times across the two reward types provides two interesting insights.  First, it suggests that reward types are treated differently in choice situations.  Though this has been demonstrated in the motivation and demand literatures, it has not been well documented in studies of intertemporal or spatial choice.  This finding highlights the need to examine more reward types than just food and other consumables.  With the exception of studies using water or juice \cite{richards_1997, kim_2008, pearson_2010}, we know virtually nothing about how animals trade off other types of rewards \cite<but see>[]{shapiro_2009}.  Food is an easy and salient reward type to manipulate, but animals make decisions over many other currencies.  Based on work on number discrimination \cite{agrillo_2007a, agrillo_2007b, agrillo_2008} and shoaling preferences \cite{hager_1991, lachlan_1998, day_2001, krause_2002}, this study manipulated the number of shoaling partners that subjects could approach.  Choosing which group to join is an important, naturally occurring decision that many animals regularly face \cite{krause_2002}.  Yet, there are other rewards that are important to animals, including mating opportunities, territories, and nest-building materials, each of which will involve a varied suite of costs and benefits and socio-ecological variables.

The second implication from the travel time difference is that there is a fundamental difference between spatial and intertemporal choice.  Though spatial choice inherently includes a temporal component, it is not simply a combination of time and energetics.  Spatial and intertemporal choices are disassociated in our data because subjects make similar spatial choices across reward types but require different travel times to make these choices.  Stevens et al.\ \citeyear{stevens_2005b} found a similar disassociation between spatial and temporal choice in marmosets and tamarins.  In that case, marmosets preferred the larger rewards in the temporal domain, and tamarins preferred the larger reward in the spatial domain.  Therefore, though they are tightly linked in many ways, animals may use different mechanisms to make choices over space and time.


%%%% REFERENCES %%%%
\bibliography{fishdisc}

\newpage
\section{Supplementary Material}

\noindent\emph{Movie 1.} Food spatial discounting test. Video recording from above of a subject presented with  two food items at 20 cm and six food items at 60 cm. 

\noindent\emph{Data sheet 1.} Evaluation phase data.

\noindent\emph{Data sheet 2.} Discounting data.

\noindent\emph{Data sheet 3.} Travel time data.

\noindent\emph{Data sheet 4.} Visual control data.

\noindent\emph{Data sheet 5.} R code for data analysis.

\end{document}
