The differential biology reader

Companies with work that can be broken into short, repetitive tasks yet still require those discerning humans to complete them are turning to cloud labor, distributing tasks to workers throughout the world. Amazon's Mechanical Turk was the first to define the field, followed by more targeted and nimble services like CrowdFlower and txteagle.

I have noticed a number of researchers using these tools to recruit participants and collect data. I opted to try CrowdFlower because they have a completely unintimidating sign-up procedure that encourages you to play with designing tasks before you push your survey out into the world.

Once you are in, you can go about creating a job. If you are making a survey or trying to field a psychological instrument, the interface suggests that you can get started without first adding data. Under the more task/job orientation of most users of these types of services, you need to populate the Job with the information you want workers to use (such as a list of URLs to visit). This is not quite what we want, but I have found the rest of the system doesn't work if you don't populate the Job with some sort of data.

So what you can do is just create a two line .csv file with something like a survey identification number.

Place this in a plain text file and upload it to CrowdFlower (if your browser blocks Flash, whitelist crowdflower.com; the uploader depends on it).

Now you're ready to get cracking. CrowdFlower has a very nice little form editor (under the Edit tab). However, you'd probably like to avoid pointing, clicking, and dragging as much as possible. You also surely have the item content for your questionnaire, ready to go. The thing to do, then, is to skip the GUI form editor and head straight for the CrowdFlower markup language, which gives you XML tags for formatting the content of your survey. This is the real gem of CrowdFlower's platform.

Once you are done with the layout, you are ready to Order Judgments (sounds serious, right?). You'll want to skip the calibration step, which presents you with a dummy copy of your survey and times you as you complete it. You should already know how long it takes to do your questionnaire.

So while CrowdFlower has many nice features, it isn't quite suited for psychological surveys. This is hardly a criticism, since it wasn't designed for this type of task. The main problem is the screen participants see once they've submitted the task. It isn't what I'd describe as a good debriefing. That said, there is a type of task that this interface is quite suited for, which is assessing personality in nonhuman primates. Like the kind of jobs that CrowdFlower was designed for, having a number of raters assess the personality of some apes or monkeys precisely getting a number of judgments by each worker (the raters) on the assigned units (the primate subjects).

Lastly, a few tips

• In the CSV file you use to import the units, don't just use variables that will appear to the raters in the task information, but also metadata that will help you sort and organize the data later. Examples are stud numbers or database IDs for each animal.
• You cannot edit the survey content once the job is running, so get it right before starting.
• In CML, you can provide an alternative name that will show up as the column label in the dataset. Capitalization of this label won't be preserved.
• The output gives you some other information about the workers, such as an ID (which I assume is somehow tied to cookies in their browser) and the city they are connecting from.

photo cc-by Amodiovalerio Verde -

The foundational advice for entrepreneurs also applies to research ventures

Entrepreneurs are almost always pushing the envelope of reason, because big opportunities are derived from doing something that either few have thought of, or many have rejected as ill-advised. So unless you have at least one major detractor, then you are probably not on to something big. In fact, if everyone thinks it is a wonderful thing to do, then probably a legion of competitors is on the launch pad.

Isenberg. The 2-Minute Opportunity Checklist for Entrepreneurs . The Conversation, Harvard Business Review

photo cc-by Kibondo

Nathan at Flowingdata turned me on to Tableau Public, a Software as a Service application for data sharing and visualization. The key feature, in addition to a graphical interface for data exploration and graphing, is embedding the visualizations on the web. Tableau takes care of hosting and gives you an snippet of Javascript that will render an interactive version of the visualization.

As a trial run, I extracted some data from the National Health and Nutrition Examination Survey. Among all the other physiological and psychological data they collect, they ask about marital status and sexual behavior. One oddity that the study authors point out is that the number of people who are or have been married but who claim never to have had sex has increased over the last few years:

Dashboard 1

That sort of gets the point across and it took longer to export the data from SPSS than to start taking stabs at the visualization. It did take me a while to figure out how to get Tableau to display the timeseries (I think because 'year' was a dimension rather than a measure at first).

Sharing interactive graphics on the web is a great place to end up, but I could only manage to produce graphs by trial and error. When manipulating the interface, it isn't really clear what the results of most of your actions will be, though at least they are displayed immediately.

Edit: Posterous seems to be stripping out Javascript from the post. The NHANES sex visualization can be seen at Tableau. :(

Chasing pregnant women, congressional district × budgeting shenanigans, Big Science (with a capital B and a capital S) comes to child health and development as the National Children's Study seeks to following children from birth to age 21. The ambition and scope is a bit staggering, but they intend to collect hundreds of phenotypic, genotypic, and environmental measurements for each participant and compare them against outcomes in pregnancy, development, behavior, and health with a particular focus on asthma, obesity, and injury.

I guess we have 21 years to look forward to the results of gene × environment interactions on temperament and emotional regulation, though from the wording in the research plan on exposures and outcomes (pdf) I cannot tell if they intend to collect personality data on the parents as well.

But the big problem the Times is reporting is with recruitment. Mothers and communities are wary of investing in the project when it isn't clear how the data are to be used or what the benefit will be. This is a problem in smaller scale research, too, such as with the hapless psychology study participant who might get some cake, or £5, or even as little as 5¢ if they wander into the peat bog of Mechanical Turk. There is no reason to invest because there is no way to track your returns. And with an enterprise as massive as the National Children's Study, carried over such an extensive period of time, we have to be comfortable with the possibility that the study will affect the very exposures it is designed to detect. Mother's are right to ask: what's in it for me and my baby?

photo: Shoshone Falls, Snake River, Idaho. by Timothy H O'Sullivan

This is why personality is a bit more than just behavior

Personality traits can be employed to guide understanding of trajectories to health and longevity, but long-term longitudinal study and multifaceted assessment of healthy aging are crucial. Following up on the life span study initiated by Lewis Terman, we assessed 4 validated factors of personality in young adulthood in 1940, constructed a multifactor measure of participants' healthy aging in 1986, and collected death certificates through 2007 (to determine longevity) on a sample of 1,312 Terman participants (732 men). Neuroticism predicted worse physical health and subjective well-being in old age and, for women, higher mortality risk, but for men, neuroticism predicted decreased mortality risk. For both sexes, extraversion predicted old-age social competence, whereas conscientiousness predicted men's old-age productivity. Differential patterns of association between personality traits and healthy aging components are informative about individual personality characteristics and long-term health outcomes.

subjective well-being tended to be not associated with mortality risk when separated from other aspects of health…The causes of this variation are still unknown, but one might speculate about persons who feel good but are carefree and ignore medical care or prescribed treatment or have unhealthy habits. Studies of subjective well-being or positive affect also often inadvertently capture many confounding variables such as current physical health, socioeconomic status, health behaviors, social integration, and more. It may be nothing about subjective well-being (happiness, positive mood, life satisfaction) per se that is important to longevity in a causal sense.

photo cc-by √oхέƒx™

The Wild Animal Modeling Wiki has a number of tutorials and links to software for carrying out quantitative genetic analyses in wild populations. They provide example data and guides for ASReml, MCMCglmm, and WOMBAT.

Papers that are very how-to oriented and the use of a wiki like this is a great step in the doing of science. The analysis subsections of papers are often a bit inscrutable when you've never performed that kind of analysis before. If you are not in a lab group doing this kind of work, it can be hard to figure out exactly the best way to affect your analysis, especially if the software has lots of little fiddly parameters. No Methods section can beat a straightforward tutorial with sample code.

Wilson et al (2009) An ecologist's guide to the animal model. Journal of Animal Ecology doi: 10.1111/j.1365-2656.2009.01639.x (pdf)

photo cc-by Gabriela Camerotti

Lots of studies of personality have either looked at individuals in captivity or the wild, but not both. To see whether personality traits that seem like they should be the same but occur in different situations (such as activity and movement in a novel cage versus discovering new feeders in the wild as measures of exploration–avoidance), Herborn et al. tested blue tit personality across and within contexts in captive and outdoor settings.

Exploratory tendency and neophobia were not correlated with each other, in either the captive or the wild context. Therefore they are independent traits in blue tits, in contrast to many species. Finally, exploratory tendency and neophobia measured in captivity positively predicted the analogous traits measured in the wild. Reflecting differences in the use of feeding opportunities, personality in captivity therefore revealed relevant differences in foraging behaviour between individuals.

It's lovely that researchers have turned to these basic issues of construct validity for measuring traits in tits. The correlation between wild and captive exploratory behavior demonstrates convergent validity, as the behaviors seem to consistently measure the same underlying, latent trait (exploration–avoidance). The two measures should be related in theory, and the data confirms that they are. Similarly, the lack of correlation among activity, exploratory behavior, and neophobia shows discriminant validity (each is a measure of a different latent trait).

It is a happy fact that ecologists are so keen on individual differences and it is even better that they are settling on using the word personality (née behavioral syndromes), but let's not leave psychometrics behind with the rest of the 1950s.

Herborn et al. Personality in captivity reflects personality in the wild. Animal Behaviour (2010) doi: 10.1016/j.anbehav.2009.12.026
Campbell et al. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull (1959) vol. 56 (2) pp. 81-105

photo cc-by Rob Baldwin Photography

PS: Can we get some loess curves in those plots, even the one that didn't show a significant relationship?

It is wonderful to see ecologists asking structural questions about personality using measures of many behaviors but I am a bit puzzled by the statistical analysis in this paper by Wilson et al on house crickets:

We tested whether laboratory-reared male and female European house crickets, Acheta domesticus, exhibited behavioral syndromes by quantifying individual differences in activity, exploration, mate attraction, aggressiveness, and antipredator behavior. To our knowledge, our study is the first to consider such a breadth of behavioral traits in one organism using the syndrome framework.

My first question is: what software were they using that didn't throw up an error when doing a PCA on only two variables (in general you need at least 3 variables, otherwise you are just getting the correlation between the two variables).

Second, why not instead do a PCA on all the raw variables? This would be another way to look for behavioral suites across contexts. There may be latent variables here that span the presupposed behavioral context.

Wilson et al. Behavioral correlations across activity, mating, exploration, aggression, and antipredator contexts in the European house cricket, Acheta domesticus. Behav Ecol Sociobiol (2009) doi:10.1007/s00265-009-0888-1

photo cc-by alles-schlumpf

Loehlin:

There are at least 18 ways in which personality-oriented behavior geneticists are concerned with environments. They include three forms of gene-environment correlation, eight varieties of shared environment, three of unshared environment, three forms of gene-environment interaction, and the environment of evolution.

Loehlin details the ways that environmental variance might be partitioned into shared and unique pre- and post-natal environments and how parents and children can be environments for each other. He also lists the "Environment of human evolution" which isn't something that will fall out of a variance component model but should be considered from comparisons across species: our personality structure is nothing new or unique.

Loehlin, JC (2009) Environment and the behavior genetics of personality: let me counts the ways. Pers Indiv Dif doi:10.1016/j.paid.2009.10.035

photo cc-by G u i d o

A report from the folks at Hunch (Mac vs PC People: Personality Traits & Aesthetic/Media Choices) tells us what we already knew: Mac users like to be different while PC users prefer to stick with what they're comfortable with. While the questions are nominally about aesthetics, media and consumer choices, and "personality," for the most part they've actually made a questionnaire that captures several facets of the personality dimension Openness to experience.

A lot of the questions hit on the facet of Artistic interests ("Which type of art do you prefer?", mostly through the choice of Modernism, and I would hazard that most folks who chose the Impressionist painting were picking 'flowers' rather than 'Monet'; all of the items about aesthetic or design preferences) although there are a few questions related to Extraversion ("How often do you throw parties?") and Agreeableness (attitude to authority). Other questions probably hit on both high Openness and low Conscientiousness at the same time (task preference).

NB: the sample size for some of these items is huge! (N > 50,000) Of course, as I tell my students, a little more methodological details would be nice:

Summary findings in this report are noted when there is a statistically significant difference in the answers of the two subsets being compared.