When is Enough Reading Enough for a Doctoral Thesis?

Photo: dermabg

by Ian Brailsford, Doctoral Writing SIG: https://doctoralwriting.wordpress.com/2018/05/21/when-is-enough-reading-enough-for-a-doctoral-thesis/

Ian Brailsford is Postgraduate Learning Adviser in the Libraries and Learning Services at the University of Auckland. Here he shares insights from his recent research into thesis bibiliographies.  

By Ian Brailsford

If I had a dollar for every time in the last decade I’ve responded to a question in a doctoral writing workshop with a succinct ‘it depends’, I wouldn’t be able to retire. But I could go on a nice overseas vacation. My answer is commonly voiced when running sessions for new doctoral candidates embarking on the literature review when the question relates to how much reading is required. This is one of the core ‘how do I know when it’s enough?’ research dilemmas (recently identified by Dr Inger Mewburn) that experienced postgraduate learning advisers are familiar with.

When it comes to the ‘how much literature is required (or expected)’ question, I have teased out my ‘it depends’ answer with a few questions of my own, not just to the person asking the question, but to the entire group of workshop attendees: are you finding mostly journal articles, books or conference proceedings through your database searching; is your research topic in a fast-moving part of scholarly publishing (such as computer science and bioengineering) or slower paced, such as my former stamping ground of history; is most of the literature in your folders (electronic or paper) from this century or over a longer timespan stretching back into the last decades of the twentieth century (and possibly even earlier); how do you know something that appears in your search results list is actually relevant for your doctoral topic? And so on.

I fully acknowledge that the correct answer to many of these foundational research questions is a version of ‘it depends’. To give a definitive one-size-fits-all answer as to how much reading is expected or required for a doctorate would be far too reductionist and, taken literally, do more harm than good. However, in the last year or so I’ve increasingly come to the conclusion (possibly influenced by my history background) that the body of doctoral work in our university library repository can offer new candidates tangible parameters to work with. ‘It depends’ can be supplemented with ‘but if you are working in this research domain, it’s probable that “x” number of references might be enough’. This knowledge, combined with what we know about thesis examiners’ expectations of the literature review in relation to relevance and coverage, can assist doctoral candidates make sensible choices about how much to read.

To this end I’ve surveyed the reference list or bibliography from a random selection of twenty 2017 University of Auckland doctoral theses. Every item was coded (and then counted) using a simple classification system: academic journal article; book/monograph; chapter in an edited collection; conference proceeding; unpublished thesis/dissertation; and ‘other’ cited scholarly items. ‘Other’ included ‘grey’ literature such as technical reports, official demographic, health, and education statistics compiled by government agencies, non-governmental advocacy groups, university working papers etc. found in open access repositories.

Based on this sample a University of Auckland doctoral thesis – on average – had: 191 academic journal articles; 27 books/monograph, 17 book chapters; 4 conference proceedings; 4 theses/dissertations; and 20 ‘other’. So the non-existent ‘typical’ doctoral thesis has 263 items; the one with the fewest had 80 references and the most 538. In both instances this was ‘enough’ for the two independent thesis examiners.

The academic journal article was the most frequently cited item, approximately 72% of all references in the 20 doctoral theses. There were, not surprisingly, disciplinary differences. In one case (biological sciences) the thesis comprised 363 references of which all but four were journal articles (98.9%). In contrast, a thesis in mechanical engineering drew from 26 conference proceedings and 45 ‘other’ (technical reports), resulting in a lower percentage of journal articles overall (55 journal articles out of a total of 137 items, 40%). Theses in education, Asian studies, and dance studies were more reliant on books and book chapters than those in STEM fields. For example, the education thesis had 224 items in the reference list: 108 journal articles (48%), 52 books (23%) and 41 edited book chapters (18%). Conference proceedings were frequently cited in theses from mechanical engineering, computer science, and engineering science but infrequently elsewhere. In a similar vein four theses (education, education psychology, dance studies, and sociology) had extensive referencing from unpublished theses or dissertations but these were the exceptions to the rule. ‘Other’ was very much subject specific rather than field of study, but if there was a pattern, those in applied research areas tended to draw from ‘grey’ scholarly but not necessarily peer-reviewed sources more frequently. As an example, a thesis from economics had 48 (42%) items out of 114 references from ‘grey’ sources.

Based on this desk-top research I’m confident giving new doctoral candidates more nuance to the ‘it depends’ answer to how much reading is required. I will be telling workshop participants that aiming for between 200 to 300 references would be a prudent reading goal for a doctoral project with about two-thirds derived from journal articles. Once they have this nominal total in mind they can calibrate with more precision by following Cally Guerin’s advice of reading doctoral theses (in their own discipline) to write their thesis and also talking with their supervisors to ensure quantity of reading is balanced with quality.

When it comes to how much time they should be reading, I use a nominal 10% to 15% of their total study time. My ‘ball park’ figure derives from the fact that thesis examiners devote about one-tenth of their written reports to commenting upon the literature review (Holbrook et al., 2007). Moreover, Paul Thompson’s (2009) study of literature review chapters in applied PhD theses indicated that literature review chapters were approximately 15% to 20% of the whole thesis. This ‘rule of thumb’ reading time is about 1200 hours over the whole doctorate. As Dr Ricardo Morais, in his introduction to the ‘Idea Puzzle: design and defend your PhD’ software, points out, the first six months of the doctorate is the ‘best part, the bohemian phase’ where new candidates can immerse themselves into the literature. But this ‘divergent’ phase has to stop at some point. The bulk of this intensive reading time needs to take place in the first few months.

So as an adviser I will carry on with my recommendation of aiming to read for 20 hours per week in the first few months, potentially generating a working (and preferably annotated) bibliography of 100 to 150 items by the six-month mark. ‘It depends’ is fleshed out. However, it will still be qualified by the reminder that one of the most important doctorates of all time, John Nash’s 1950 Princeton PhD dissertation ‘Non-cooperative games’, which gave the world the ground-breaking ‘game theory’ (and subsequently a Nobel prize for its author), contained only two references, one of which was a paper written by John Nash. In this instance two items in the bibliography was enough.

I’m using this blog post to crowd-source the tenor of my advice to both doctoral candidates and advisors: (does it ring true, it is helpful, potential unintended consequences?) and to see if anyone else has calculated the number of references in the doctoral theses in their repositories. Is 200 to 300 items in the reference list or bibliography probably enough?

References:

Holbrook, A., Bourke, S., Fairbairn, H., & Lovat, T. (2007). Examiner comment on the literature review in Ph. D. theses. Studies in Higher Education, 32(3), 337-356.

Thompson, P. (2009). Literature reviews in applied PhD theses: Evidence and problems. In Academic Evaluation (pp. 50-67). Palgrave Macmillan, London.