Qualitative Research and Audit Trails

Why care about audit trails? It is considered a way of insuring and assuring quality in less standardized qualitative studies.

The reason I mention it is the clarification and self-reflection required on the part of the researcher, resulting in a more rigorous research process and thus, theoretically, results. It includes the recording of the whole process as it relates to decisions made with respect to the resulting products and processes of your research, decisions we should automatically document for defense of our research.

Akkerman et al (2006) provide in Table II the audit trail components which, if you are interested, you should review along with the text associated with this Table. The authors’ include defining the problem, the conceptual framework or paradigm within which you will be working (useful for consistency in interpretation throughout the paper), the process document, etc. that all should reflect decisions taken along the research process. It seems a bit obvious when reading it, but while performing research, and if you lack experience in performing research as I do, this becomes a valuable guide to what you should be considering, retaining and recording as you progress.

They do speak to recommended strategies for documenting what you’re doing. These include: “…compulsive recording, use of a comfortable but consistent recording system, detailed description, immediate recording prior to digestion or discussion, and ongoing cross-indexing between notes and data.” In my case each time I made what I considered significant changes to my text, I kept a copy of the old word file. I think this fits nicely within the “compulsive recording” aspect. My weakness was in that I did not explicitly record all my decisions, though selected decisions are commented on in the first of my two articles. Robinson (2003, 175) mentions Smith’s (1999) “reflexive journals” which may provide a method for compulsive recording of thoughts. I’ll have to go check.

The intent of an audit trail is that an auditor can follow what you have done and may perform any checks required to ensure and assure the quality of the results also. I have a suspicion auditing by a third person will mostly occur within the context of a Masters or a PhD because of time constraints. But my focus, to repeat, is the assistance this activity may provide me in the creation of rigorous research and its related products.


Akkerman, S. et al (2006). “Auditing quality of research in social sciences.” Quality & Quantity, pre-publication copy available from SpringerLink for those with subscriptions, or from the Igitur Archive from the University of Utrecht.

Robinson, Z. (2003). “Exploring the audit trail for qualitative investigations.” Nurse Educator, 28(4), July/August, p. 175-178.

more on subtexts in LIS: theoretical sociological frameworks

D’you recall when I was burbling on about how there seemed to be no subtext behind many LIS articles (Sept. 28 and Oct. 22, 2006 posts)? I’ve been reading about research methods and discovered the following statement which speaks to my comments on a lack of subtext (meaning an historical perspective of ideas and theory) from a sociological point of view:

“Usually a discipline has a dominant paradigm, although this can and often does change over time. Because of the diverse nature of information science there is no truly dominant paradigm today, although it is fair to say that at the time of writing ‘interpretivist’ research is without doubt the newest paradigm and is only just beginning to emerge as a significant force in research and communication (Pickard: 2007, xvii).”

This author further states that there are only three major research paradigms in information science (LIS) today: positivist, postpositivism, and interpretivist. Very brief, simplistic explanations that require more context than is currently available in this space are:

Positivism: “…assumes the existence of an objective, independent and stable reality, which is available for discovery and analysis.” These “[s]ocial facts are seen to exist independently of human interaction just as natural laws exist (8).”

Postpositivism: “[o]bjective social facts do exist independently of and external to human beings, but these facts are subject to uncertainty and probability.” “…all discovery is subject to interpretation; it is the responsibility of the researcher to demonstrate objectivity during the discovery process…and is demonstrated by external validity (10).”

Interpretivism: Empirical interpretivism “…deals with investigation in natural settings of social phenomena; …[and critical theory] engages in ideologically oriented investigation, examining current thought and social structures.” Interpretivists believe “…that realities are multiple, constructed and holistic. There is no single, tangible reality, instead there are only the complex, multiple realities of the individual (11).”

So where does this leave us/me? Within the sociological context, which subtext/paradigm am I most interested in and wish to promote in my work? Or perhaps I am flexible enough to change approaches, though consistency in approach is implicitly preferred within a research environment. Plus, what about the science subtexts? Remember the “is LIS a sociology or a science” question? Are the science subtexts still relevant in the field or was there a mix-up of positivism with science in LIS’ inception?

Alison Jane Pickard. (2007). Research methods in information. London: Facet Publishing.

mental frameworks and values

I ran across an article that argues “…that when people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it.” “Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities” (Kruger and Dunning, 1999: 1121).

All of this seems self evident to me. There will always be someone more competent than myself – it is like your older sibling who will always be your older sibling and you may never catch up because they have a head start (assuming all other experiences etc remain equal).

The study tests for humour, grammar and logic, and what cognitive dissonance exists in self-perception in these areas. It is fascinating in that it does not address what mental frameworks we hold. And I’m thinking of those people whose skills, training and mental frameworks are based on a professional training that existed for librarians 20 or even thirty years ago that have not been able to (or may not desire to) move with the changes in emphasis and perspective in the field. These people have been trained in a different time for the needs of that time and are competent to that time frame. It is only when one time travels forward, or even backwards for that matter (Woohoo! Punch cards anyone? Oops, suddenly I’m incompetent) that one recognizes the inadequacies of skills and training – assuming one is competent enough to recognize change and incorporate it into a newly revised mental framework.

And there’s an important point – if all it takes improving skills to increase metacognitive competence then why are those librarians isolated in their own category instead of being in with the rest of use? Because it takes more than improving skills to shift, revise and reintegrate new realities. Some might say it requires certain behavioural characteristics. I would add it also takes a willingness to change. Further, you can force exposure to our cognitive frameworks but these librarians won’t shift from their cognitive frameworks. The relevant question is why? I don’t know. Fear?

When change challenges any values on which we sustain our images of ourselves that our flexibility and the capacity to adapt comes to a screeching halt. If one was taught that knowledge of specific tools and attention to detail are the hallmarks of an x librarian and we have constantly prided ourselves on these, then any challenge to these is doomed to failure.

How about a Cataloguing Department whose managers have emphasized for decades that the best way to serve the library’s users is dependent on the accuracy of MARC records? How does that Department then deal with students and reference librarians who state the emphasis is on finding answers, not necessarily items, and that a comprehensive description of the item is less important than a skeleton record with a link to the content of that item, in order to find the answer as expeditiously as possible. Another example might be one of the new library positions: a mix of administration, collections, acquisitions and project management with a dependence on flexibility, the ability to turn on a dime, the capacity to manage experts when you lack the in-depth knowledge of those you are managing, and the ability to bring together members across functional units to achieve specific goals for the life of the intended purpose. How do we reconcile ourselves to the ephemeral nature of this change?

I’m not sure I’m communicating well but I hope this at least brought about a few new thoughts.

Kruger, Justin and David Dunning. (1999). Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, v. 77(6): 1121-1134.