Qualitative Research Methods

In the second period of 2024 I followed the (mandatory) Qualititive Research Methods (QRM) course, given by professor Judith Masthoff. Qualitative research deals with subjective data collection that cannot be directly numerically measured, like interviews or case studies. The purpose of the course is to teach about the various research methods that exist, how to design and conduct them and how to analyse & subsequently report on that data. A lot of attention was also given to ethics and privacy.

The course is set up quite uniquely; the emphasis of the course lies in a big two-person project where an entire qualitative research project should be designed, actual data should be gathered from participants and this gathered data should be analysed, all while keeping privacy in the highest regard. At the end of the course, a big 'conference' would be held where all groups can present their research, all completely organised by the class.

On to what we actually did!

After some brainstorming we found ourselves interested in the topic of password managers (PMs). We set out to design interviews for understanding people's reasons for using or not using them, their subjective requirements they would have for such tools and to understand the current ways they are handling passwords. Of course, privacy is very important conducting and analysing the interviews, especially because of the sensitive nature of our research. We devised the following RQs: - RQ1: What considerations do individuals have towards personal password security? - RQ2: What considerations do individuals have for using or not using a password manager?

We found it quite challenging to scope our research and create a consise script of questions we'd like to ask. Because of this, and because we would like to go more in depth on some topics depending on a participant answer, we decided to perform semi-structured interviews. This means there will be a question script, but it's alright to deviate from this if an interesting topic comes up.

We picked participants using a screening survey, asking about if they are currently using a password manager and - if yes - what kind: browser based or software based. As expected, the overwhelming majority of possible participants that were screened used browser-based managers. This is exactly why a screening survey for possible participants proves useful: ideally, participants should come from diverse backgrounds to be able to generalise results and deter a bias. In our case, people currently using a password manager might have completely different views on password security than people currently not seeing the need to use a PM. Things like this really made me see how intense it could be to design a research: there is a lot of thought that has to go into every step of the process to be able to get sound results.

In the end, participants from three equally sized screeninggroups of participants were invited for individual interviews. Alan and I both took turns in conducting the interview, while an audio recording device was there to record participant responses. I noticed how difficult it actually is to conduct an interview, specifically because you cannot help the participant too much by giving examples, as this may influence them and their answers.

Using the audio recording, a transcription was made. Then came the most time consuming part: annotating the data using self-made labels (codes). Every transcription was carefully analysed for phrases that could fall within a code.

For the remainder of the analysis, we determined some trends that we encountered using the things that participants mentioned and fell within our codebook. This led to our final conclusions that participants who do not use PMs generally have security data privacy concerns and participants who do use a PM prefer convenience. We also found a lot of participants mentioning that remembering passwords themselfes was important to them, and that they think of new passwords by re-using old ones.

I think this project was very informative and for us led to interesting insights. It taught me a lot about interviewing techniques and other research methods, where the form of the course really contributed to being able to apply this knowledge to your project.