John's Philosophy of Science Page: How to do Proper Online Research

Introduction

On this page I list common errors people make in doing online research, that is, gathering data online for the purpose of ultimately writing a research document of some sort, e.g., an academic paper, a factual book, etc. I hope this page is helpful to you. If you think it needs some additional pointers mentioned on it, please let me know.

Please note this page contains the author's opinions only and may differ from the Rules and Regulations of the University. Please consult the University Rules, which overrule these.

http://www.cracked.com/blog/5-easy-ways-to-spot-b.s.-news-story-internet

Interpreting and Choosing Sources

The most important thing is to use reputable, recognised, accurate sources. There is a lot of information on Internet which is of questionable provenance. In particular, you should avoid making the following mistakes:

Do not assume that because a page is online, its content is worth referencing. The content may be completely inaccurate, false, misleading, biased, or sensationalist. You can find anything you want on Internet that would agree with a view you may have, no matter how outrageous the view is. So, for example, there are Holocaust Denial sites, there are conspiracy theory sites, and there are sites that post misinformation deliberately - either because the author mistakenly believes that his prejudices are true (e.g., racial supremacist sites), or because the sites are tongue-in-cheek humour or satire sites. Learn to recognise these sites and avoid using their material (unless you're specifically doing a study of crackpot sites).
Do not only use one source for your information. Many web sites just copy and paste from each other without citing the source of the information. If you find a passage cited on two different websites, and the wording is identical, you must assume that the wording originates elsewhere. Try find out where it comes from, originally, by looking until you find a quote of the paragraph which cites the reference. If you cannot find a reference for a quote, do not use it, no matter how good it is.
Do not just appeal to one authority. Make sure you compare and contrast two or more disagreeing authorities. Do not give your opinion: just cite at least one or more authorities that you side with, but be sure to give the opposing point of view and why you disagree with it, which said opposing point of view must be of equal status to the point of view you support. I.E. You can't compare a journal research article's view to a conspiracy site and then support the conspiracy site against the superior research.
Double-check all assertions against reputable sites. If you see some statement made on a dot-com, or dot-net, or dot-org site, you need to be careful. These are often sites set up by private citizens, whose views may be uneducated, biased, or even worse, bigoted. That's not to say that authors on reputable sites cannot ever be guilty of the same things, but just that they're less likely. You should consider university sites in the First World (i.e., .edu, .edu.au, .ac.nz, .ac.uk, and so on), to be reliable. You can also generally trust sites of academic journals, which will often be .org sites. Use your discretion. NGO sites, which are also often .org sites, can sometimes be trusted, as long as they have statistical studies to back up their claims. Some NGOs are fear-mongerers and sensationalists. I won't say who, but you should be conservative whenever you hear a strong opinion.
Assume that the truth in any debate is somewhere in-between two strong opinions. Remember: if two professors disagree, and neither are idiots (because they're professors), then there must be some misunderstanding, linguistic or conceptual obstacle, or an honest mistake - e.g., misapplication of statistical methods, etc. The job of academic research is first and foremost to try resolve disputes between strong opinions, and arrive at a reasonable, well-attested opinion (note: opinion).
Do not fall for the "I saw it on TV" explanation, or other appeals to authority which are not cited clearly. For example, references to YouTube videos, or TV shows, or Newspaper websites, particularly opinion pieces - should all be regarded with suspicion. Their aim (videos, newspapers) is to draw an audience, and audiences are best drawn by alarmist sensationalism. If the appeal to authority is made to a well-known and respected authority, i.e., a university professor or similar - then you can consider that opinion to be well-supported. Other than that, it's mere hearsay and should be treated with considerable circumspection and double-checking.
Do not trust books just because they are books - especially Ebooks - books that you can download. If the book was written any time in the last fifty years, it may just be a conspiracy theory written by some hack. (I say "in the last fifty years" because recent advances in technology have made "knowledge" production a very democratic exercise, no longer the preserve of ivory-tower academia.) You should therefore trust books primarily from academic or well-known publishers, e.g., Blackwell, Routledge, Oxford, Cambridge, Penguin, etc. Books published by obscure publishers should be regarded with suspicion. If you get a book from an obscure publisher, view its contents with the same degree of skepticism as any random website that you might find; in other words, check that it references all its claims, that its argument is balanced and not one-sided, and that its references are all to legitimate, respectable sources, rather than to conspiracy theorists or similar sites.
When you view a page, and it has references, check that those references are respectable, or from academic or scientific sites, articles, books, or journals. Do not assume that if someone has referenced something, that the source is credible.
Beware of anecdotal evidence. If a website talks about experiences of individuals and draws generalisations from that, you must discard that evidence. A statistical sample of one (or two) individuals does not warrant any generalisations being drawn. If, for example, you find a website that advertises a new diet, and you're researching diet efficacy, do not assume that because the site has pictures of smiling people quoted as saying that it worked for them, that it is in fact an efficacious diet. If, however, the company has an open paper that you can read, which shows that the diet worked for 95% of people in a sample of 1000 people, then it is more reliable.
Check who funded the research. If it is a medicine and the research was funded by the company that sells the medicine, regard it with suspicion. Of course they will say that their products work! You need to check their research by seeing if (a) it was published in a well-known peer-reviewed journal (i.e., an impartial judge had a look at it), and (b) what the critics in that peer-reviewed journal had to say. See if anyone else has run an independent study to see if they can replicate the findings of the study. If the study was not published in a peer-reviewed journal, and no critical follow-up studies were done, and no attempts at replicating the results were done, you cannot trust those results.
Beware of democratic claims - e.g., "A million people believe that P is the case" or "95% of Americans accept that Q is true". Just remember that 1000 years ago, 99.99% of people thought the world was flat. Truth isn't democratic.
Beware of unqualified generalisations - e.g., "The sky is blue." These remarks are dangerous because they sound superficially plausible until you realise that there are a whole load of other things that the truth of the remark depends on, e.g., the sky is not blue at night, the sky on Mars is not blue, the sky is not uniformly blue at sunset, the sky is not blue when the weather is overcast, etc. Whenever you see an unqualified interesting generalisation, e.g., "Women score higher on spatial skills," you need to not only check the research behind it, but also what the conditions were under which that statement was true. e.g., women would score lower on spatial skills while wearing a blindfold, or while suffering from myopia, or they might score lower in a different culture, etc.

Be careful of assuming that because something has statistical, mathematical, or scientific reasoning or evidence behind it, that it is true. Check that the experiment has been replicated - ie., whether someone else, apart from the researcher, has done the same experiment and achieved the same results. Check the mathematics. Ask yourself whether the statistical model applied was the right one, or whether the person used the wrong model. Ask questions about the sample population. Was it a representative population? Were the individuals involved biased in some way? Were there enough persons in the sample (e.g. 1000, not 10)? Were they from a cross-section of the society, or was the study focusing on a particular subgroup?
Be careful of assuming that because a professor or PhD researcher says something is true, that they're right. Remember that most issues in academia are still under debate. You should make a point of finding at least one researcher who disagrees with the person whose view you prefer. Find someone's view that annoys you, especially if they come from a credible academic background, and see for yourself whether their view is plausible. Make sure you mention their view in your research and explain why you still disagree with it. Don't be one-sided or biased, in other words. Show both sides of the argument. In particular, don't keep going to the same sites, and sites that link you over to related sites with similar content. That's called "preaching to the choir". Challenge your own assumptions and beliefs by looking at what the opposing point of view says.
The hardest thing to avoid is confirmation bias, that is, where you read only things that support your view, or look for evidence which supports your view (only), and you write with the intention of supporting your view, and you do not give serious and careful consideration to views that disagree.
Do not assume that because someone is a bad person that their argument is bad. This is called the "fallacy of ad hominem" - "to the man" - or "a personal attack". A lousy researcher with a dubious past may still produce an oustanding piece of work.
Be careful of reductionist or oversimplified conclusions or explanations. Be careful of definite conclusions that are not supported by massive statistical data. So if you see, for example, a diet that cures any diseases AND is supported by testimonies of a few smiling people on the website, ignore it. It's a hoax. There's generally not only one good explanation for a phenomenon, and there is generally little evidence to support any one particular conclusion. You should only accept conclusions that are heavily supported by statistics (e.g. sample sizes of 2000+ participants), that have been peer-reviewed, that have been replicated, or that follow logically (formally) and mathematically, by entailment. So the following is a BAD argument: "Drinking only red wine will cure any diseases, because red wine contains antioxidants and antioxidants have been shown to cure all diseases in my family". The following is a GOOD argument: "Drinking some red wine may help prevent certain diseases because red wine contains antioxidants and there is some statistical evidence in a sample of persons tested with a control group that 80% of participants experienced fewer disease X symptoms when taking a weekly glass of red wine, and the article cited argues that it is the presence of the antioxidants in red wine which assisted in disease prevention (Reference: some academic journal)". See the difference?

Do not assume that because a view is a minority view, that it is wrong, or that a view that is a majority view is right, even in academia. For hundreds of years, Newton's formulae were the majority view. One man, Einstein, overturned those equations. He was the minority. But he was right, and Newton was less-right. Question orthodoxy, but not to the point of paranoia (i.e., to the point of becoming a conspiracy theorist).
Look carefully at the site you're examining. If it's a dot-com, and it has large font sizes, bad design, bright colours, lots of adverts, flashing banners, and sensational claims, you can be pretty sure it's a conspiracy site, and that they have put sensational claims on that site to draw advertising revenue. You should ignore its claims and find a proper source.
Plagiarism. Plagiarism, in my definition, is: the quoting or usage of any original creative or academic material without stating where it came from. If you want to use someone else's work, you can save yourself from the accusation of plagiarism, and possibly disciplinary action (e.g., lawsuits or expulsion from university), by doing two very simple things: (a) put quotation marks on either side of the quoted paragraph, and/or (b) putting, immediately after it, a reference. (In other words, in parentheses or a footnote, saying where you got it from. For the most part, the author name, date, and page number is adequate, provided that the full reference appears in your bibliography or reference list at the end of your research. If you don't have a bibliography, cite the entire reference.)

The following is considered plagiarism:
Chances are, someone, somewhere, has already said that. Find out who they are, and quote them. The following is NOT considered plagiarism:
Notice that in both cases, you've used the same wording, but the first is not permissable, and the second is perfectly correct. Remember: you're not the only person who knows how to use Google. If a critic reads your research and Googles your paragraphs, especially ones that are well-written, and he finds that it is a direct quote of another source, but that you have not referenced it, you will be, by default, guilty of plagiarism. Journalists can be fired for this, students can be expelled for this. All it takes to solve the problem is quote marks.

Reputable Sources

You should avoid getting material from sites who do not have a known reputation for good academic content. The following is a list of sites that I regard to be reputable or largely reliable:
Any university site, e.g., Plato.stanford.edu, www.ox.ac.uk
Wikipedia.org
Any journal article site, e.g., www.philpapers.org, www.jstor.org, etc

Sources that are not considered reputable are:
Most .com sites
People you know
Yourself
Hearsay
TV, DVDs, and other entertainment media.
Regarding newspaper sites and newspapers, you can cite these sources and quote from them provided that you make it clear that it is a newspaper that is being cited, and that therefore (a) the informative content is suspect, and (b) the opinion stated is merely an opinion, and is not considered authoritative, and (c) that the article may contain misrepresentations, hype, sensationalism, or other exaggerations. Generally, citing newspapers is primarily useful only for discourse analysis - i.e., showing the sort of discourse that is/was prevalent in society at the time. It is, at best, anecdotal evidence, unless, for example, you find 1000 newspaper articles, all with very similar content (that can be shown, formally, syntactically and semantically, to be similar), and you then apply a statistical mathematical analysis to the prevalence of the key concepts contained therein.

References
Always cite your references, no matter how trivial the statement is that you're making.
The only statements that you do not have to reference are:

- Your own opinions that you do not know anyone else to hold, e.g., "I believe that Smith is mistaken in his theory."

- Opinions that follow logically from a previous argument, e.g., "Socrates is a man, all men are mortal, therefore Socrates is mortal."
Logically necessary truths, e.g., "All bachelors are unmarried men," "1+1=2," etc.
Well-known facts which are beyond disputation, e.g., "The sky is blue most of the time during the day on earth," "analog clocks have twelve hour-marks," etc.
Cite your reference immediately after each statement. For the most part, the author name, date, and page number is adequate, provided that the full reference appears in your bibliography or reference list at the end of your research. If you don't have a bibliography, cite the entire reference (in such cases, rather use footnotes than parentheses, as complete references get very long to read past). Note that you can use footnotes, exclusively, if you wish. Here are some examples of referencing:

"Smith says that P is not the case." (Smith, 2009, p60).

or:

"Smith says that P is not the case." (2009: 60).

or:

"Smith says that P is not the case."¹

¹Smith, T. (2009). An analysis of fundamental logic. Oxford University: Oxford. p60.

Anything suitably similar to these examples will count as acceptable or adequate referencing. Remember: the point of referencing is so that your reader can check up where you got your ideas from and make 100% sure you're not misrepresenting the person that you're quoting, or quoting them out of context. If your references do not check out, you completely lose credibility.

Note the following abbreviations:
et al.- "and others". Used when you've cited a long list of author names already, and don't want to re-type all of them. But you have to list them at least once. So, for example (Smith, J., Jones, T., 2009, p60) can become, on the second instance: (Smith et al., p60).
et seq.- "and following". Used when you've referred to an argument or evidence which goes on for a few pages. Eg.: (Smith, 2009, p100-200) can become: (Smith, 2009, p100 et seq.) Avoid using this; it means that you can't say exactly where you saw it, which looks sloppy.
cf.- "confer - compare". Used when you want the reader to read a similar argument or passage in another author that makes a similar point. Eg.: (cf. Smith, p90). I have seen this used for the same author, as well.
qv.- "which see". Same as cf., basically, but usually to refer the reader to another spot inside the same book or article.
ibid.- "in the same". This is used when the reference is the same as a previous one. Eg.: (Smith, 2009, p60), can thereafter be followed by: (ibid., p61), (ibid., p62). If, however, another author is cited, then you have to drop the "ibid" and go back to the full version again, at least once, otherwise you'll be saying that the reference belongs to the new author. I recommend you avoid using ibid, because when you edit your paragraphs and move them around, you might move a new author into an older author whose paragraph is littered with ibids.
op. cit.- "work already cited". Same as ibid., basically.
When referencing a web page, cite your references as you would for any journal article, except, instead of providing the publisher city, provide the URL. e.g.,

Smith, T. (2009). An analysis of fundamental logic. Oxford University: http://www.ox.ac.uk/analysis_of_logic.pdf
Remember that if you cite something on the web, it is likely to disappear, because people do not tend to keep things on the web for very long, so you should probably save the article, preferably as a PDF so that the fonts and images are retained in one file. You should name the saved file something like this:
so that it's easy for you to find the file again. You should save the URL into a file as well, eg.:
so that you can cite the reference in your research paper at a later stage, without having to Google again.

You should give the DOI reference if possible (the digital object identifier), rather than the URL. This is because if the object moves, the same DOI will refer to it properly.

You should also say when you viewed the object/page, e.g. --Accessed 20 Dec. 2013.
If a file is likely to disappear off Internet, i.e., if it's not in Jstor, then you should probably consider printing it and attaching it as an appendix, if it's not too long, to prove that you didn't just make it up. Only do this for articles that are on questionable sites, or pages that are likely to disappear, e.g., pages on .com sites. You might also only do this if the article in question has notbeen published in a proper journal.
If an article has been published in a proper journal, then you MUST cite the journal version of the article, not the web version. Most reputable pages that contain a journal article will mention that the page is a journal article and that it has been put online "with permission" of the journal publisher. This is your hint to go and find the real thing, eg., on Jstor. Or just email the academic who wrote the paper. Most academics want to be cited by other researchers, so most academics are happy to email you copies of their papers. Just be polite and do not ask for more than one paper - that's greedy.
If you are citing a webpage and it doesn't have page numbers, because it is a continuous scrolling single document, then cite the paragraph number and/or section number.
Don't put words in peoples' mouths. If an author did not say something, or you have misinterpreted an author, you can't reference it to him or her. Your reader may know your subject area very well, and if you misattribute a reference to the wrong author, you might get caught out falsifying your references.
Writing Style, Grammar, Spelling

Make 100% sure your spelling is correct. Do not assume that a spellchecker on a word processor will get the spelling right. For example, if you use "their" instead of "there," or "hear" instead of "here," "discrete" instead of "discreet," a spellchecker will not pick it up, but it will ruin the meaning of your sentence. Bad spelling creates a bad impression. It is sloppy and unprofessional.
If you even have the slightest doubt about a word's meaning, check the meaning in the dictionary. For example, evince, evidence (verb), and posit, postulate, do not have quite the same meanings.
Make 100% sure your grammar is correct. For example, I've used "their" above as the neutral singular personal pronoun, instead of he/she. This is incorrect. If you don't like he/she, then alternate "he" and "she" in your examples. Similarly, make sure your sentences agree, e.g., if your object of your sentence is plural (the noun), then make sure that the verb is plural ("are"), etc. Don't worry too much about grammar checkers fussing about passive voice; passive voice is quite commonplace in academic writing. ("The man hit the ball" is active voice, "The ball was hit by the man" is passive voice.) Also learn the difference between whom and who.
Avoid using the word "I". You should give your opinion through the voice of whomever you're citing. So, instead of "I think that P is the case," rather say, "Smith thinks that P is the case" (Smith, 2009, p60). Or make it neutral: "It may seem that P is the case, considering the aforegoing argument."
Do not commit to a position as being a fact unless it is logically entailed (1+1=2) - i.e. unless it is a demonstrable fact. Rather say "it appears," "it seems," "it may be the case that". The only time you should commit to a position as a fact if you think that your argument does demonstrate something - in which case, say, "it is the opinion of this author that," or, "from the aforegoing reasoning, it must be the case that".
Check your writing style on a website that checks style for you, e.g.: http://ed.essayrater.com/

Friday 6 March 2015

How to do Proper Online Research