Can real behavior of users, when it comes to security decisions, be observed in lab studies? A recent paper from my research group sheds light on this question.
Initially, our goal was quite different. We replicated and extended a 2008 study conducted at CMU that investigated the e effectiveness of SSL warnings. To achieve better ecological validity, we adjusted the experimental design: allowing participants to use their web browser of choice and recruiting a more representative user sample.
At the end, we found what we were not looking for. During our study we observed a strong disparity between our participants actions during the laboratory tasks and their self-reported “would be” actions during similar tasks in every day computer practices. Our participants attributed this disparity to the laboratory environment and the security it offered. In a paperrecently presented at SOUPS we discuss our results and how the introduced changes to the initial study design may have affected them. Also, we discuss the challenges of observing natural behavior in a study environment, as well as the challenges of replicating previous studies given the rapid changes in web technology. We also propose alternatives to traditional laboratory study methodologies that can be considered by the usable security research community when investigating research questions involving sensitive data where trust may influence behavior.
See more details in the paper.