How to evaluate your project
This post is all about how to evaluate your project. This is something you will do ideally about half way through your work, but more realistically towards the end of your work, just before you write up. However, it's imporant that you plan your evaluation early on, otherwise you run the risk of getting almost to the end of your work and finding that your evaluation process isn't going to give you any good results. So, the main thing you need to understand when you are planning is how evaluation works in the sciences and how to apply that understanding to your own projects.
Proof
Sometimes we see student dissertations which make claims like "this experiment has proved that ...". Almost always the student has used the word "proof" incorrectly, and will lose marks because they have given the impression that they have not understood the contribution that their work has made to the field they are working in. Very often students in this situation simply haven't understood how scientists use the word "proof".
Proof, in a scientific context, is a mathematical argument that is used to convince other mathematicians or scientists that a theorem (or a mathematical idea) is true. Proofs must never involve evidence or experiments, only arguments. There's an example proof of a simple theorem at the end of this post.
Once mathematicians are convinced that a proof is correct (and sometimes that is difficult in itself, if the proof is several hundred pages long) then it is irrifutable. This is very different to the sort of science that is advanced by experiments, where another scientist can find new data or eveidence that shows that an old idea was wrong.
So, we generally say that a theorem can be proved correct whereas a hypothesis (or guess!) can only be tested via experiments. A hypothesis might turn out to be wrong if experimental data cannot be found to support the hypothesis, or contradictary evidence is found. If a lot of evidence is found to support a hypothesis we might call it a theory. Even so, a theory cannot be proved correct in all cases. For example, if you came up with a theory that said all atoms have a particular shape, you might invent a special microscope to look at atoms and find out if they have your shape. This would provide some evidence to support your theory. You couldn't, however, test every single atom in the Universe, so your hypothesis might well become a theory, but it can never be "proved" correct.
[Aside: there's a long literature in this sort of philosophy of science. If you are really interested, read Karl Popper on Falsification, AJ Ayer on Verification and Paul Feyerabend on scientific revolutions and Imre Lakatos on Proof.]
Scientific method
Scientific method is the way that scientists decide whether a particular hypothesis (or guess) is likely to be a good model for the way the world works. If most scientists accept that the hypothesis is likely to be true, then we call it a theory. Of course, even theories have limitations, and it may be that as more experiments are carried out we find that a different theory fits the evidence better, or that the theory only works in certain circumstances. This is exactly what happened in physics to Newton's laws of motion. It turns out that Newton's laws describe the world pretty well in most cases, they can certainly tell you when your train is likely to arrive at its destination. For other circumstances, for example when you are travelling very fast, close to the speed of light, or for very small particles like quarks, other theories (like Einstein's theories or quantum mechanics) better fit the data we have gathered. Of course, much of this work in areas like physics is driven by what we can measure and observe. Better telescopes mean better theories of cosmology, and so on.
In computer science we also have hypotheses that we can test. For example "functional programming languages can run just as efficiently as imperative languages", "online learning increases student engagement", "objects and inheritance improve code reuse in software companies", and so on.
To be a true hypothesis, and not just the opinion of the author, a statement must be refutable, that is, it must be possible for experiments to determine that the hypothesis is incorrect. The opposite statement to a hypothesis is called an alternate hypothesis. Examples for the hypotheses listed above would be "functional languages are necessarily slower (or faster!) than imperative ones", "online learning has no effect on student engagement" and "objects and inheritance have no effect on code reuse in software companies".
So, to evaluate your own research questions, you need to do the following:
- Devise a hypothesis.
- Form your alternative hypothesis.
- Plan an experiment that tests whether the hypothesis or the alternative hypothesis is true.
- Conduct your experiment.
- Analyse the results of your experiments.
- If the results are conclusive, STOP. Else, re-run the experiments, or devise a better experiment and repeat.
In a student project, you may not have time to repeat your experiments, especially if they involve people, but you should design your evaluation in such a way that this would be possible, were you to continue the work.
About experiments
A good experiment should test one variable and one variable only. So, if your hypothesis is "neural network algorithms run faster in C than C++" then you will probably want to implement some neural network algorithms in both languages. You should make sure that the programs are as similar as possible, except for the language you are using. If you implement slightly different algorithms, it may be the algorithm and not the language which is causing any change in performance you observe. In this case, the programming language is called the independent variable and the algorithms are called the controlled variable and the speed is the dependent variable which is being measured.
Bad science? The case of Usability tests
Many students undertaking projects who have developed a web application, web site or content management system (often in response to a client brief) ask about the most suitable evaluation method for their project. In these cases precisely what to evaluate is often less clear cut than an experiment with an independent variable and a controlled variable (as in the "neural network algorithms run faster in C than C++" example above).
In order to settle on an evaluation method for such cases it is often necessary to return to the client and question them about their goals for the application you have built for them. The first question you need to ask is “Can my application be evaluated by automatic means?” i.e. for applications that are evaluated for technical performance (network performance, speed of execution, resource efficiency, memory performance, robustness against attack, etc) the answer is usually 'yes it can' and your evaluation may consist of running a number of automated performance tests and collating the results into comparison tables.
However, in cases where the client might be interested how humans (users) interact with the application and 'perform better' because of it, the evaluation solution is usually some form of 'user based test'. Your client may be interested in the performance of the application you have built in supporting users to achieve their goals. There are a number of parameters that could be tested:
Effectiveness: Can users actually perform and complete the (desired) specified task?
Efficiency: Can users do it quickly, without getting bored or frustrated?
Satisfaction: Is it fun, or at least pleasant to use?
Learnability: Can users 'pick up' the application without reaching for a manual or asking for help? Does it support learning? (see ISO 9241 section 11)
Different applications will have different emphases in terms of what you need to evaluate. Games, for example, need to be satisfying or challenging most of all. Terminal applications for call centres need to be efficient most of all. Most kinds of 'stand-alone' technology (Car park ticket machines, vending machines, ATM machines) need to be learnable most of all. All applications need to be effective.
At this point it is important to stress that working with real users introduces a large number of uncontrolled variables that cannot easily be 'designed out' of any user test you may undertake. The fundamental question about the 'scientific validity' of usability tests (i.e. whether you are 'really' evaluating user performance rather than the application's performance in the test) is very difficult (but not impossible) to answer.
There are two specific limitations with usability test data.
The reliability limitation: Is the data reliable?
No, if you are testing users who are not typical of the true intended user group, or if there are significant individual variations within the test group (this is made worse by small sample sizes*)
The validity limitation: Are the conditions under which the data was recorded reproducible?
No, if the test is run differently each time (users briefed differently, different equipment used, not quite the same test questions asked)
As Jeffrey Rubin points out [usability] “testing is always artificial” (Rubin, 1994, p.27). [Aside: There is a fascinating and long running debate on the scientific status of usability between Jakob Nielsen and Rolf Molich, particularly in relation to the small sample sizes championed by Nielsen. See here or here]
There is some good news about usability testing, however. Often it is entirely unnecessary to worry about the number of uncontrolled variables in a usability test because you are looking forindicators rather than proofs. In fact, if you think of a usability test as a 'design tool' rather than an 'experimental tool' you are closer to the way tests are used in the commercial world. This does not absolve you of the responsibility for designing and recording a test in which you have addressed the reliability and validity limitations OR that you have set appropriate benchmark metrics for judging the success (or not) of the application in supporting effectiveness, efficiency, satisfaction or Learnability. It does mean, however, that usability tests can, if designed properly, be a very good evaluation method for your project.
Mind your language: 'user friendly' and 'significant'.
There are two phrases you need to be very careful about using in your test reports.
User-friendly. Never use this phrase! Software cannot be friendly, it is not (yet) sentient. This is called ANTHROPOMORPHISM and should be avoided. To claim that an interface is 'user friendly' is also subjective and not testable. To claim, however, that an interface is USABLE is more sustainable as long as we measure against some predetermined metrics.
Significant. In normal English, "significant" means 'important', while in Statistics "significant" means not due to chance. For example, if I answer all questions in a multiple-choice quiz randomly, sometimes I will still pass the test. However, the number of times that I pass the test as a percentage of the number of tries I have in total should show that passing the test a few times just happened "by chance" and was therefore "not significant" in statistical terms. A research finding may be true without being important. When statisticians say a result is "highly significant" they mean it is very unlikely that the result happened by chance. They do not (necessarily) mean it is highly important. Be careful when you claim to have found 'significant' results.
(http://www.surveysystem.com/signif.htm)
Interpreting your results: correlation does not imply causation
Correlation by xkcd
When you perform an experiment, you are hoping that the outcome will lend some evidence to either your hypothesis or your alternate hypothesis. Going back to the example above, they hypothesis "neural network algorithms run faster in C than C++" has an alternate hypothesis "neural network algorithms run no faster in C than in C++". If we run an experiment to test this, and assume it's a fair experiment, and the results are that all our algorithms run faster in C, what has this told us? A naive answer would be that the experiments have confirmed the hypothesis that C is the faster language for this sort of algorithm. A more subtle answer would be that efficient neural networks are correlated with neural networks written in C. That means that when the algorithm is written in C it's likely to run quickly, which is what the experiment reported. This does not necessarily mean that the algorithms implemented in C ran quickly because they were written in C, it may be that there was some other factor involved that the experiment didn't effectively control.
In experimental work it is very important to understand this subtle distinction, otherwise you can easily fool yourself into believing that your experiments have discovered something far more conclusive than is actually possible.
To give you a better idea of how this distinction between correlation and causation works, below are some examples of incorrect conclusions drawn from perfectly reasonable correlations. See if you can work out why the conclusions are unreasonable:
- Children with bigger feet have higher reading ages. Therefore, people with bigger feet are more intelligent.
- Teenagers who text late at night have poor motivation in class (see news reports here). Therefore, using mobile phones leads to poor performance in class (see also a more skeptical analysis here).
- In the last 150 years there has been a dramatic increase in the number of people who report being abducted by aliens. There has also been a trend towards global warming. Therefore, alien abductions cause global warming.
In your own work, just be honest and straight forward about your results. If they aren't conclusive then say so and demonstrate your understanding by describing what future work could be done to gather more data.
Some basic dos and don'ts
This is some more specific advice, based on good and bad practice we have seen from students over the years:
- DO be clear and honest about what results your evaluation has obtained.
- DON'T claim to have "prooven" anything if you haven't written a formal, mathematical proof.
- DO use an appropriate experiment for your hypothesis. For example, if your work is about evaluating the performance or security of a technique, there is no need to involve real users in your evaluation. If your hypothesis is about usability you really must involve real users.
- DON'T use questionnaires unless you can guarentee to get a large sample size of answers (always well above thirty) and you understand the statistics needed to analyse the results. If you are in any doubt at all about this then seek the advice of a qualified statistician before you start your project. If you can't do that, think about using an alternative evaluation method such as semi-structured interviews.
Appendix
Example proof: The square root of 2 cannot be written as a fraction of whole numbers
Theorem
The square root of 2 cannot be written as a fraction of two whole numbers. (This is sometimes called the Theorem of Theaetetus)
Proof (by contradiction)
Imagine we could write the square root of 2 as a fraction of two whole numbers, say x/y where x and y are integers.
Let's say that x and y don't have any factors in common, so x/y is already written in its simplest form and no numbers can be "cancelled out" of the fraction.
So, we can also say that (x/y)*(x/y)=2
Therefore (x*x)/(y*y)=2
Therefore (x*x)=2*(y*y)
So we now know that x*x is even, since x is 2 times another number.
Since x*x is even, we also know that x is even (by the "Lemma" or little theorem that squares of odd numbers are never even).
Therefore, there must be a number, which we'll call z such that x=2*z
So, (2*z)*(2*z)=2*(y*y)
Or, more simply, 2*z*z=y*y
y must also be even, by the same argument that we used to say that x is even.
If y is also even, there must be some number, which we'll call w such that y=2*w
But if x/y=2z/2w then the fraction x/y was not in its simplest form like we assumed above.
This contradicts our initial assumptions, which must have been wrong.
So, the square root of 2 cannot be written as a fraction of whole numbers.
How to read research papers
These last couple of weeks I've been taking my groups of final year project students through the process of starting their literature reviews. There is a separate post on literature reviews on this site here and a post on why you should read academic literature in Computer Science here. This post isn't to do with those topics, this post is about how to read research papers. We often find that if students haven't done much of this sort of reading before their get to their final year getting started can be a bit of a shock. So, this post is designed to help you get started with academic literature and, just as importantly, to help you get the most out of the papers you read in the short space of time you have available (and it is a short space of time, believe me).
Remember the structure of a paper is just like the structure of your thesis
Other posts on this site have discussed the overall structure of your thesis, but in outline this is the sort of structure you should be expecting to produce:
- Introduction should introduce the reader to your research question and the broad context of the research.
- Literature review should describe the work that other people have carried out to answer your (or similar) research questions.
- Method should describe what you did to answer your research question (or to support your thesis, if you think of it that way), and how you went about it.
- Results should evaluate what you have done, and say what answer (to your research question) you have arrived at.
- Conclusions should summarise what you have done and how you answered the research question.
Academic writing (in the sciences) of all sorts follows something like this structure, including all of the papers that you will be reading for your project. There are a couple of exceptions to this rule. One is theoretical papers which sometimes put their "related work" (or literature review) somewhere towards the end of the paper rather than after the introduction. The second exception is survey papers. Surveys are extended literature reviews and, as such, are a good place to start in your own literature reviews. ACM Computing Surveys is a journal that publishes survey papers or you can sometimes find them in reputable journals.
Briefly review each paper for relevance
You don't have time to read everything, so it's important to make sure that what you do read is really relevant to your thesis. So, to check whether a paper is likely to be relevant to you first read the Abstract. This should give you a brief summary of the whole paper. So, at the very least the abstract should give you a good idea of what research question the authors were trying to answer. Next, read the Conclusions. This is also likely to be a summary and may well give you a better idea of what results the authors obtained and what work they did not finish but left for "future work". If that doesn't give you a good enough idea of the relevance of the paper to your own work, try reading the last part of the Introduction. This is usually where the authors summarise what is written in each of the following sections of the paper, so that should give you a much more detailed view of what the rest of the paper contains.
If, after all of that, you think the paper is irrelevant to you, then discard it and move on to something else. Otherwise, you are ready to move on with your reading...
Focus your reading on specific questions
If you just go ahead and read a paper from start to finish the chances are that you won't get very much out of your efforts. You are likely to ramble around the paper, not taking very detailed notes and at the end of your efforts you may not have learned much. A much better way to go about your reading is to keep in mind a number of clear, focussed questions and read the paper with the intention of writing down answers to these questions in your notes. That way you will finish with a clear set of notes that you can be confident will be useful to you when you start writing up.
I would recommend you use this set of questions to guide your reading:
- What research question were the authors asking?
- Why did the authors believe that their research question was important?
- How did the authors go about answering their research question?
- What results did the authors obtain or, what did the authors learn from answering their research question?
You can find a template for some notes here.
Making use of your notes
When you have finished reading you should have a stack of notes on all the papers you have read. This should be a much more concise way to start writing up than having a much bigger stack of papers and (most likely) not much memory of what was in them! So, the next thing to do is decide on the structure of your literature review chapter.
The first paragraph of your chapter should introduce the rest of the chapter. This is a good place to remind the reader of your research question and explain how the current chapter relates to it.
The last paragraph of your chapter should summarise what you have reviewed. This is a good chance to help the reader naviagte around your thesis. Briefly review what you have said in the chapter and refer the reader to the next chapter, explaining how the next chapter follows on from the current one.
The middle part of the chapter is more difficult and, since your writing will depend on your particular research question and the literature you have read, there isn't much generic advice to be given here. However, you can start by reading through your notes and looking for common themes. Think about how best to present the ideas to a reader who has not read the same literature. Do you want to take the reader chronologically through the literature, from the earliest point to the present day? Would it be easier to understand if you split the reading into particular topics that are related? When you have what you think is a good structure, write some section headings into your thesis and think about which papers go in which sections (of course, some papers may well go into several sections). Write the citations into each section using something like EndNote, Mendeley or BibTeX to format them for you. Play around with the structure until you are convinced that it will make sense then write in the details of each section. Make sure you check out this post to help you with your writing.
Why read from primary sources? Or: why reading blog posts is harder, not easier than reading papers
I've been meaning to write this post for a long, long time. Now that I have an enormous pile of marking to get through in double-quick time, I have the perfect excuse for a bit of structured procrastination.
What is a primary source?
A primary source, is an original piece of writing, describing some research and written by the person or team who performed that research. A secondary source, is a description or discussion of a piece of research by someone who has read about the research, but did not carry it out themselves. So, if an academic performs an experiment and writes it up as a journal paper, that paper is a primary source. If another researcher then quotes the paper and cites it in one of their papers, then that is a secondary source. Newspaper articles, magazine articles, wikipedia, and most websites and blog pages are secondary sources. When it comes to scientific research, only writing published in peer-reviewed conferences, journals, books and magazines constitutes a primary source.
What is peer-review and why does it matter?
Even if a paper is a primary source describing some research, that doesn't guarantee that the research is rigorous, reliable and high-quality. To ensure that all academic writing meets basic standards of quality assurance, scientists use peer-review. This means that a number of professional scientists (usually two or more) will read through the work carefully, and critique it before it is published. If the work is of very poor quality, or very badly written, it will be rejected and the authors will have to re-write their papers and try to publish them elsewhere. If the work is of a high enough standard to publish, the authors will be given a list of improvements they must make before the paper goes to print. This way, we ensure that inaccurate, incorrect, or incomprehensible work doesn't get published in high quality conferences and journals.
Why read primary sources?
Students often complain about making the leap from reading textbook-style prose to formal, academic research literature. Part of the problem is that the style of writing is different, and takes some getting used to. More deeply, though, students today have likely grown up with the web and with reading informal, secondary sources, making the change is hard work, and nerve-wrecking for some. Why waste hours wading through pages and pages of long-winded, complicated, weirdly-written prose, when you can read a quick, accessible summary on Wikipedia? Well, of course Wikipedia is a good place to start to get a basic overview of an area and help your understanding of the primary sources you are reading. However, it is absolutely essential to read the primary sources themselves. Why?
Reason 1: secondary sources editorialise
A secondary source will describe some parts of the primary source, but not others. The secondary source will take a particular point of view (i.e. the author will voice their own opinion) and will pick the parts of the primary source that are useful for that discussion. This doesn't necessarily mean that the secondary source is particularly biased (although it might be), it's more that secondary sources are selective in what they discuss. For example, if a paper on Web2.0 discusses the implementation, performance and usability of Web2.0 sites, a secondary source on the subject of usability is likely to leave out any mention of implementation and performance. So, by reading secondary sources you miss out on a lot of the detail of the original work and much of that detail may be very important to you and your work.
It is probably worth saying that there is an important exception to this: survey papers. A good survey paper should be like an extended literature review that discusses, in some detail, the literature available in a broad area of Comptuer Science. These survey papers are a good place to start when writing your own literature review. You can usually find survey papers in well established journals, or specialist survey journals such as ACM Computing Surveys.
Reason 2: secondary sources are sometimes wrong
Every academic field has a number of ideas which are passed on from one generation to the next with little reference back to the original research that generated those ideas. Be somewhat skeptical about this, most of the time there are good reasons to feel assured that this knowledge is sound, especially in fields where mathematical proof is the main way of advancing the field. However, in more subjective or experimental fields (such as Software Engineering or Usability) results can sometimes be misunderstood or misinterpreted over the years.
An example of this is Winston Royce's "Waterfall Method" which (as you probably already know) is a method for organising and planning large programming projects. The central idea in Royce (1970) is very simple and easy to understand: you split the work into a number of different "phases" (requirements gathering, analysis, design, coding, testing, maintenance) and your team performs each phase in turn. There's even a nice image to go with the idea, just to make it nice and easy to understand:
Image source: Wikipedia
For many people, this is where their understanding of the waterfall model stops. But in Royce's original paper there is a long discussion of the drawbacks of organising a project in this manner In fact Royce says that it is "risky and invites failure" (pp. 329). Moving on, the majority of Royce's paper is a list of changes to the sequential model which make it more workable. Some of these are of particular interest, for example "plan testing" is a step that Royce advocates should go with program design. In modern, more "agile" development methods we would advocate writing unit tests around this time, so Royce is presenting a very modern approach. The last modification Royce makes is to "involve the customer" at several points in the process. Again, a much more modern approach that many authors would say goes with agile or "eXtreme" development methods.
The picture Royce paints is not a simple sequential model at all, it's much more complicated than that. Tarmo Toikkanen has written an interesting blog post on this subject. He speculates that the reason people advocate for the basic waterfall method is that the diagram and analogy make it very easy to understand, so people don't delve any deeper into the details. In fact, Toikkanen points out that NATO even have a military standard (DOD-STD-2167) based on Royce's work. [Aside: If you wanted to test Toikkanen's idea that it's the diagram in Royce's paper that leads to the misunderstanding, what experiment would you devise to test that idea?]
More parochially, we often see University students writing something like "in my project I will use the Waterfall Method" sometimes even with a citation. DON'T DO THIS! Read Royce (1970) in full, understand what he's really arguing for, then use a more modern method, or at least use Royce's iterative method found at the end of the paper.
Reason 3: different primary sources may disagree
Research is all about creating and discovering new ideas. Very often primary sources disagree on how best to do that, or they have competing ideas and only through years of research and discussion does a consensus evolve. There are examples of this throughout the history of science. Whether it's the flat Earth debate, Big Bang vs. the Steady State theory, structured programming vs. object oriented programming, through debate, reason, mathematical arguments, prototype systems, models, simulation and all sorts of other techniques, the history of science is full of arguments and competing ideas.
When you read a secondary source, very often whatever "debate" has taken place is already in the past and the author of the secondary source will simply describe the consensus that has since been reached. For example, there were many good reasons for cosmologists to believe in the steady state theory before evidence for the Big Bang became overwhelming. Only by going back to this litereature can we see how the debate unfolded and why the evidence that supported the Big Bang (to do with background microwave radiation in the cosmos which was discovered in the 1960s) was so convincing.
In Computer Science there are also many of these debates. For example, most programming languages do not have a "goto" statement. In fact, Java has a keyword called "goto", but it is not used. In the late 1960s and 70s there was a heated debate about whether "goto" was a safe and useful construct and you can read through that debate in Dijkstra (1968), Knuth (1974) and plenty of other sources. Without going back to these papers, which were written well before the debate was settled, you cannot fully understand the arguments that, eventually, banished the "goto" statement from most modern languages.
Conclusions: reading blog posts is harder than reading papers
So, why did I say in the title of this post that reading blog posts is "harder" than reading papers? Actually reading blog posts may be easier, but in terms of getting a good grade in your project you are unlikely to produce a high quality literature review based on blog posts. Blogs will tend to be selective and biased in their nature. This isn't a criticism of blogs, far from it, blogs are a great place for lively debates. They aren't such a great place to descibe careful, peer-reviewed research in great detail -- that's best left for conferences and journals.
References
Top 9 writing mistakes made by project students
A large part of any final year thesis project is the write-up, but every stage of the project will involve some form of writing. Whether you are writing a proposal, an interim report, a draft report, documentation or the final thesis, a very large part of your time will be spent composing text. These are the top ten mistakes we see from students year on year, avoid them and you can save yourself a lot of time (and earn a lot of marks)...
1. Unsubstantiated claims
As a general rule, any statement of fact or opinion about your work, or your topic of study should be substantiated in some way. You don't have to worry about the bleeding obvious, we know that 1+1=2 (unless your work is in the "foundations" of mathematics or philosophy, in which case 1+1 may well be undefined), but anything less obvious should be backed up by solid evidence. That evidence can be a reference to literature, an argument you make in your writing, the results of your own experiments, or any other suitably rigorous evidence.
Avoid so-called sweeping statements that are effectively impossible to substantiate "all Object Oriented programs couple algorithms and data". Do they? All of them? "Testing improves the performance of students". Really? These sorts of claims are poor academic practice and suggest that you have been a little sloppy in your thinking about your work, and of course that's not the impression you want to give.
2. Dangling pronouns and other references
This seems to be an almost universal problem with student writing. Consider the following paragraph:
In his 1953 paper, Henry Gordon Rice proved that for any non-trivial property of a partial function there is no general decision method to determine an algorithm that computes a partial function with that property. It has far reaching consequences for compilers, static analysis and other fields in practical computing.
What does the "It" in the second sentence refer to? Rice's theorem, his paper, or something else? It isn't clear from the text, although we can guess that the author meant to discuss the theorem. Better though, to be clear about the meaning in the first place. Every pronoun ("I", "he", "she", "it", "that", "who", etc.) should clearly refer to exactly one noun. The first sentence gets this right, it is clear that "his" refers to the noun "Henry Gordon Rice" and not any other noun in that sentence. So, we could improve the paragraph above by re-writing it:
In his 1953 paper, Henry Gordon Rice proved that for any non-trivial property of a partial function there is no general decision method to determine an algorithm that computes a partial function with that property. Rice's theorem has far reaching consequences for compilers, static analysis and other fields in practical computing.
3. Using a secondary source rather than a primary one
It's much easier to read the popular press, blogs and other "informal" media than research papers. Very few marks will be awarded for this, though. In a final year project you need to show that you can perform a small academic study, so marks will be available for reading peer-reviewed academic literature. There are two major pitfalls to avoid here. Firstly, you will occasionally come across some disreputable conference or journal which does not use peer-review. Worse, it is possible to come across "articles" on the Internet which have citations and publishing records and look, to all intents and purposes, like a genuine piece of academic writing, but have actually never been submitted to a journal or conference. This is very poor practice on the part of anyone who puts this kind of thing up on their own blog or website, but it does occasionally happen. A reputable publishing venue will have some sort of statement on their website stating how articles are reviewed (look for "Instructions to Authors"). This should say that every article is reviewed by at least two people and sent back for corrections before being published. Without this sort of peer-review any poor standard of work can be "published" without anyone checking even basic standard of good practice, such as detecting plagiarism. That said, even with peer-review, some poor practice still slips through the net.
Secondly, whatever references you cite should be primary, rather than secondary sources. A primary source is one where the author(s) reports work that s/he (they) have personally carried out. A secondary source is one where an author reports on work that someone else has completed and published elsewhere. Secondary sources include news reports, magazine articles and blog posts about research completed by others.
4. Confusing structure and use before definition
Any academic writing should generally be written for a reader who is an expert in the general field of the study, but not necessarily in the specific area of the study. If, for example, your final year project is on genetic algorithms, your work might be marked by an expert in artificial intelligence, or in computer science generally, but not necessarily by an expert in GAs. So, write with that in mind, and make sure that you don't use any specific technical terms without defining them. This is often very difficult to get right at first, especially when you have been working with your own ideas for a very long time. A good plan is to swap drafts of your work with a fellow student who is on the same degree course but working in a completely different field for their final year project. If you can both understand each others work, that's fine. If not, make changes.
5. Claims of "proof"
At the beginning and end of your dissertation you will want to set out the aims of your work and describe the conclusions you have reached. Occasionally we see students writing sentences such as "this study proves that ...", "this thesis will prove ...", and so on. "Proof" is specifically a mathematical method, and if you have genuinely proven a theorem, by all means say so. If not, then don't use the word "prove" and be very careful about what you do claim. For example ...
- If you have tested a piece of software and it has passed all your test cases, then you have shown that your software is free of the specific errors you have checked for. You have not shown that it is "error-free" or "works".
- If you have performed some sort of user testing, or any other usability / accessibility testing, then you may have demonstrated that the system under study is "usable" or "accessible" as far as you have tested it. However, without a large-scale study that is as much as you can claim. Be very circumspect about reading research in the area of usability; there is much good research but also much which is over-blown. Be especially careful of authors who also run consultancy practices, make sure you cite their academic literature, and not anything that could be considered advertising. Make sure everything you cite is peer-reviewed.
- Be very, very careful about using questionnaires. I usually tell all my students to just avoid them altogether. It is very, very hard to produce a questionnaire which holds up under academic scrutiny and you will need an amount of statistical sophistication to produce sensible results. Also, you need a very large sample-size because your questions will be circumscribed (and for other reasons). This makes questionnaires very difficult to use in short, single-person projects. If you are in any doubt, then use a "semi-structured interview" to interview test subjects and the "talk-aloud protocol" or "cognitive dimensions of notation" for usability testing. Before you start, read some papers on evaluation methods, such as Hollingsed and Novick (2007) Usability Inspection Methods after 15 Years of Research and Practice, or Hornbaek and Law (2007) Meta-Analysis of Correlations Among Usability Measures.
6. Ad-hominem remarks
We see this very rarely, but just occasionally a student will forget that they should be writing about academic research, and criticise an author directly. Examples include "$X is stupid", "it would be stupid to think that..." and so on. Don't do this, stick with the research and don't criticise the person.
7. Don't be meta
If there's one piece of advise students regularly misunderstand it's that you should be cautious and critical of the literature that you read and cite. Of course, you should be critical and cautious, but you should also be sure that you are writing about the content of the research that you are reading about, not the quality of the writing. I should probably say, this piece of advice does not hold up if you are studying literary criticism, or anything similar, but for science-based subjects, stick with the science. Don't say things like:
(Foo Bar, 2009) is a poorly written paper. It is confused and hard to follow.
Also, don't say things like:
The problem with this paper is ....
If there's a problem with the research that the paper describes, then by all means discuss that, but not the writing.
8. Don't write a giant list
I've written a separate post on literature reviews, but one common mistake we often see here is to structure a thesis as if it is a list of points, not a single piece of prose. Do not write about the literature in your field one paper at a time, try to tell a story about how the field has developed, culminating in saying that there is clearly a gap in the research where your contribution can fit. So, avoid writing like this:
Foo Bar (2007) On the usability of Flibble widgets
Bar describes the Flibble widget, which is used for ...
instead, work your thoughts on Flibble widgets into a longer piece of writing on widgets. If you do need to break up and structure your literature review, make sure your headings are topics which group together related pieces of research.
9. Don't replace elegant paragraphs with bullet points
A very common error we see in project write ups is where the student writes a series of ‘bullet lists’ instead of connected and well supported prose. The problem with this is that bullet lists do not often demonstrate complex thinking, rather, they simply provide ‘shallow’ summaries of topics (when we need some depth!). For example:
“People use CSS for:- Interoperability
- Future proofing
- Accessibility
- Conforming to standards”
In this example – what does ‘interoperability’ mean? Why does CSS provide ‘future proofing’? Is CSS the only way to enable accessibility? There is SO MUCH TO SAY on these subjects! The bullet list destroys your chance to demonstrate your research and ideas.
How to choose a good BSc or MSc project
Planning stuff...
A critical part of the success or failure of any thesis project is the initial choice of what to work on. This is a surprisingly difficult part of any project, in some ways the most difficult part, and it's something that we see students struggle with year on year. Nothing is so disappointing than marking a project and coming to the realisation that with some better decisions at the beginning of the year a failing project could have passed. This is a trap to avoid, and by avoiding it you will not only improve your chances of passing your project, you will greatly improve your chances of getting a first. In fact, projects are pretty straight forward to do well in, so long as you fully understand what is expected of you. This post takes you through what you need to focus on and avoid right at the start of your project journey.
Do something you are interested in
A final year or MSc project is a six month, single person project and in most Universities students will have to study several other modules concurrently. This is a long time to be working on a single piece of coursework, so it is important to choose a project which will hold your attention for that length of time. Moreover, you will be working on other things at the same time, so ideally you need to choose a project that is compelling enough that you want to work on it, in preference to doing other things.
How to know what you are interested in
This might seem like a rather unnecessary topic -- what is "interesting" is very personal and individual. However, estimating what you might find interesting in several months time, when you are under pressure to meet deadlines is not easy. One trick to weight the odds in your favour is to choose a project which you do not, at the start of the project, entirely know how to complete. Like Einstein said: "If we knew what we were doing, it wouldn't be called research". Obviously, don't choose something that is completely outside your area of expertise. If you have spent two years studying bioinformatics then don't suddenly decide to try a dissertation in ceramics, but equally, if you know exactly how to complete every part of the work that you will need to do for your project then your idea is not "big" enough in scope. This point, really is the key to finding a project and much of the rest of this post expands upon it: a thesis or dissertation is not simply a long piece of coursework, it is an individual, self-contained work which should stand on its own. Think of it as a sort of "first job". When you leave University and apply for further study or a job, then the results of your project will be part of the professional portfolio of work you can use to convince a future employer to take you on.
Project difficulty: a difficult project is an easy project
By far and away the biggest mistake that we regularly see from students writing project proposals is choosing a project which is far, far too easy. The train of thought seems to go ... projects are difficult, I want to make the project easier, therefore, I will choose a simple idea to work on. The classic examples of this in Computer Science are "a website with a database" -- usually for a family member or friend who runs a small business -- or occasionally a website or database on their own. What's wrong with this? Well... so many things:
- By the time a student has reached the final year of their degree, they will likely already have written several databases, websites and at least a couple of websites-with-a-database. Therefore, the project is something that the student has already been awarded credit for. This means that the student will not be demonstrating that they can learn independently, and go beyond what has been taught in lectures, which is one of the main purposes of the project.
- Because the proposal is about the same size and quality as an individual module coursework, it is not large enough in scope to gain many marks.
- An individual website, in ASP, PHP, or similar, for an SME is a very old problem for which there exist a large number of "turn-key" solutions -- that is, off the shelf products that can be used to create the product. These include templating systems such as Joomla, cloud-based solutions such as Google Sites, Posterous, Tumblr and so on, wikis, and a number of other technologies. A straightforward website-with-a-database is, therefore, in no way a demonstration of the students ability to work at the cutting edge of their field.
- A website-with-a-database is not a problem, it's a solution to a problem. A project proposal should propose an interesting problem, with a suggested strategy for solving that problem during the progress of the project.
Having said this, I have seen and indeed supervised a number of excellent projects, for which the student implemented some sort of website and some sort of database. So, it's not that websites or databases are inherently bad choices as solutions to the problems posed by a project proposal, but a proposal MUST overcome the four problems outlined above.
The heading for this section said (rather confusingly) that "a difficult project is an easy project". What I mean by this is that the "difficulty" of a project is something that will uppermost in the mind of the staff marking your thesis. A "difficult" project is likely to be looked upon favourably because it will be a bigger step away from what you have already been taught, you will need to be reading more academic literature, you will be showing more independent learning, and so on. These are some of the most important factors in getting a good grade, and far outweigh factors such as finishing every part of your practical work. The up-shot of this is that if you choose a "difficult" project and complete it quite poorly, you are likely to get better marks than a student who chooses an "easy" project and completes all of their practical work. If you did choose to work on a website-with-a-database-for-an-sme then the proposal will be so easy that you will really have to complete every part of the project perfectly just to get a pass. So, choose a small but difficult project.
Have a research question
In the last section I said that a project proposal should pose a problem, not a solution to a problem. Ideally, it is best to phrase this as a research question, such as the following:
- is algorithm X more efficient than algorithm Y?
- is it possible to implement product Z on the cloud?
- can feature L be added to programming language P?
- can theorem T be proven?
- can algorithm Z be adapted to be used in conditions C?
and so on. There are several advantages to this. One is that this is a standard form of writing in academia, and your project will be marked against academic criteria. Secondly, if the aim of your project is to answer a question then you leave the issue of how to answer that question reasonably open ended. It may be that you have a very clear idea, at the start of the project, what you are going to do. That's fine, but as you progress through the project you may well find literature that enlightens your views on how your question can be answered. Thirdly, your answer to the question may not be what you expect. That's fine, it's OK to find out that actually, your algorithm isn't as efficient as you thought, or the theorem cannot be proved, so long as you give solid, convincing evidence for your answer.
Do something practical
If you are working in the sciences, it really is important that you do something practical as part of your work. For these purposes "practical" can mean experimental work or mathematical work -- it's OK to prove a theorem, for example, as the main part of the "practical" content of your work. What you should avoid though, is vague, nebulous, thought-pieces, which have no clear results and cannot be evaluated. Avoid anything with a title like "an investigation into X" or "a dissertation on Y". These sorts of writing are well accepted in the humanities, but for a scientific piece of work you need to propose a question and find some answer to it. Equally, a literature review is not really a project in itself, it needs some research question and evaluation with it to form a complete project.
Focus on evaluation from the start
Evaluating your work will likely be the last practical work you complete before finishing your dissertation writing. However, you should know from the start of your project how you plan to do this. As with unit-testing, it is best to have designed you evaluation in as much detail as possible before you start you practical work. That way, you know that what you are aiming for is something that can be evaluated in the manner in which you have planned. Remember, the purpose here is to determine whether your project has answered your original research question.
In general, your evaluation will fall into one of the following categories:
- Performance evaluation: either testing the speed, memory footprint, scalability, load-balancing, or other aspect of the performance of a program or system. This is often the easiest form of evaluation -- it can be performed by a program and so automated, the results can be analysed and presented using a statistics and you will not be reliant on users. Work in programming languages, networking, operating systems, databases, and hardware tend to suit this sort of evaluation well.
- User-acceptance testing and usability: if your project involves creating a product for end-users to test, especially if you have an industrial client, then it is essential that you perform some sort of user acceptance testing. Good options for this are the talk-aloud protocol or semi-structured interviews. NEVER, EVER, EVER think that a "heuristic" evaluation is sufficient. Heuristic methods only catch basic errors, they tell you nothing about how your users will actually experience your product.
- Formal or semi-formal methods: such as proving a theorem, using a model checker (such as SPIN), using a formal method such as B or Z to show that your work is free of particular types of errors.
Take (academic) advantage of your supervisor
Every student will have at least one supervisor, who will usually be actively involved in research, consultancy or something similar. This sort of work can provide a wealth of good ideas for projects and has several advantages. Firstly, your supervisor will propose projects that have the right scope and difficulty for your degree course. Secondly, if your supervisor has an interest in what you are doing, they will have a vested interest in seeing you succeed and of course will have a lot of relevant expertise with which they can advise you. Lastly, it is likely that your work will be used by other members of a research group which will give you access to feedback on what you have done.
Be flexible (within reason)
Remember that a project is a marathon, not a sprint. It may well be that you get part way along the journey and find out that what you had first set out to do is actually impossible, or impossible within the scope of the project. Or it may be that you find some other way of answering your research question, or you uncover some literature which shows that the question can actually be answered very simply. In this case, you should speak with your supervisor and find a way to reword or even completely change your original research question. This is quite a reasonable thing to do and happens often in "real" research projects, so you should not be worried about it. Your final project does not have to match the original proposal exactly, but you should be able to explain why the changes you made were necessary.
Summary
- DO choose a project that will hold your interest for the duration of the project.
- DO NOT choose a project that is the same size or scope as a coursework, or something that is very similar to work you have been set in a module.
- DO propose a "difficult" problem -- it is easier to pass a challenging project than an "easy" one!
- DO propose a research question, and an idea for solving it.
- DO propose a project with some sort of practical or mathematical component, DO NOT set out to write a commentary on a topic.
- DO have a very clear plan for how you will evaluate your project. This should clearly state how you will determine whether or not you have answered your research question.
- DO NOT evaluate an end-user product with only heuristic methods.
- DO test end-user products with real users.
- DO take advantage of the expertise of your project supervisor and their research interests.
- DO be flexible, if you find that your original research question cannot be answered, or if you find that a more "interesting" research question emerges during your project.
How to write a literature review for your final year thesis project
A long time ago I wrote an article on how to pass your final year thesis project, which several students found helpful. In the same vein, this post deals with a particular aspect of the final year project: the literature review.
- How many papers should I read?
- How long should the literature review be?
- Should I read books, articles, or ...?
- Is it OK to reference websites such as Wikipedia?
- Who will read my literature review and what can I assume about their knowledge of the area?
- When should I start the literature review and when should it be finished?
These questions crop up frequently and will be familiar to any readers who are starting their own project. However, when you fully understand the purpose of the literature and how to go about writing one, you begin to realise that these questions are actually not that important. This post is designed to help students make that transition, from not yet understanding what the literature review is for, to having a thorough understanding of its purpose and a clear idea of how to write it up.
- Introduction: should introduce the reader to the broad context of the research and explain why this is an interesting area to work in. So, if your thesis is something to do with mobile computing, you might say something here about why mobile phones are important, why mobile computing is an interesting and important area, and broadly what other researchers are working on. At the end of the chapter you will want to introduce your specific research question, having said why the area you are working in (and therefore your question) is important.
- Literature review: Now you have introduced the reader (who will likely not be an expert in your exact area) to the broad research agenda in the field, and your research question, you can start writing more specifically about your own project. In this chapter you will survey the work that other researchers have done to answer your research question, or related questions. At the end of the chapter you should briefly explain how your own work builds on and differs from the work that has gone before it.
- Method: this chapter should describe what you did to answer your research question (or to support your thesis, if you think of it that way), and how you went about it. You should describe your work in sufficient detail that another researcher could recreate your work to check your results.
- Evaluation: here, you should evaluate what you have done, and say what answer (to your research question) you have arrived at. It may be that in your method you describe some experiments, and this section records your results and analysis of those results. This is an important section -- most students gain or lose marks in either their literature review or evaluation. Key to producing a convincing evaluation is to plan very early in the project what information you will need to write this section. More on that in another blog post.
- Conclusions: should summarise what you have done and how you answered the research question. It may be that your work produced a very clear answer to the question, or it may be that your work points to a need for further research to clarify or confirm your answer. You should refer back to the literature review and summarise how your research differs from (hopefully improves on) the work described in the literature. Make sure you also say what research you would do if you were to continue working on your project.
- References: a list of publications cited in the main text, in Harvard style or similar format.
It is likely that most chapters will be roughly the same size, although the introductory chapter and conclusions are usually slightly shorter than the others. Try to let the lengths of each chapter be guided by the amount of useful and important information you have to convey to the reader, don't impose artificial word limits on yourself.
The area of pervasive, or ubiquitous, computing was founded by Wieser (1991) [ referenced] who predicted that computers would one day be integrated into everyday objects and interact with people seamlessly. Although few such products are available today Weiser’s work has led to the creation of a number of research areas, including ambient intelligence (Eli and Epstein 1998), smart dust (Khan et al, 1999) and the Internet of Things (Brickley et al, 2001). [Sets the historical context of the area and defines related areas.]
An early application of pervasive computing was the active badge location system, described by Want et al (1992), in which users and objects were tagged with an "active" badge which could locate and identify them. This system was based on ultrasound locationing, whereas later systems might use RFID technology to achieve the same effect. [describes how the field has changed over time] Uses of the active badge system included routing phone calls, email alerts and so on to the physical location of the receiver. [contextualises the fundamental research]
...
How to pass your final year thesis project
I've been thinking for a while that it would be good to distill some of the advice that colleagues and I give to students doing final year thesis project, so here it is -- enjoy!Think of yourself as an academic.Whatever you want to do when you leave University, your work will be written for academics and marked by them. A thesis project is like a small research project and to get the best marks available you should run your project with that in mind. So, you need to perform a thorough literature survey, follow a sound methodology, critically analyse your results and so on.Have a hypothesis (or a thesis statement or a research question).A lot of students approach their projects by saying "I want to do this", like "I want to invent a Foo, written in VB". This is not the best way to get good marks. Your project needs to have some sort of clear purpose and in the academic world it's best to state that purpose as a hypothesis, thesis statement or research question. These three are really just different ways of stating the same thing and which you choose is just a matter of taste. So, if your project is a study on the ratio of smokers which develop cancer (e.g. "I want to do some statistical analysis on smoking") then you might phrase that in one of these ways:
- Smoking is correlated with cancer -- hypothesis
- Smoking is correlated with cancer -- thesis statement
- Is smoking correlated with cancer? -- research question
Or, perhaps you want to design a new GUI widget to replace list boxes:
- Users will find WidgetFoo easier to use than ListBoxes -- hypothesis
- Users will find WidgetFoo easier to use than ListBoxes -- thesis statement
- Is WidgetFoo easier to use than a ListBox? -- research question
Part of the point of phrasing your work like this is that it should change the way you think about the work. You now have a very clear goal to reach -- to prove your hypothesis / thesis statement or to answer your research question. Everything you do in your project should now be directed towards this one goal.
Also, you have reduced your risk of failure. What if WidgetFoo turns out to be rubbish? Well, then you have still answered the research question or disproved the hypothesis -- you've got a result. In your write-up you can probably say a lot about why WidgetFoo wasn't as good as you thought, how you achieved your results and how other researchers can use your work to invent even better widgets.Produce something useful to others.So, you've got a hypothesis to answer. Great. But your teachers will still want to be convinced that the hypothesis you've chosen is worth six months of your time and lots of hours of their time. How do you know if your work is useful? Firstly, make sure you've read the relevant literature. Use Google, go to the library, talk to potential users. There should be a group of people in the world who have a clear reason to be interested in what you're doing. That might be a section of the research community, a user group, a company, whoever -- but there must be someone who wants your hypothesis validated or disproved. You should convince whoever's marking your thesis that this group of people exists by explaining in your thesis what the current literature says about the area you are working in and how your work contributes to this area.Do something (a bit) novel.
You're not expected to win a Nobel Prize on the back of your undergraduate work. However, there's no point in doing something that everyone has done a million times before. The worst example of this in Computer Science is a project which is basically "I'm going to write a website with a database". Usually the website is for a friend or relative and the database keeps track of users. There's nothing wrong with that if there's some real novelty in it (e.g. you've just invented a new sort of database and this is a demonstrator for it). But often this is something that most first years could complete for a coursework (so, it isn't stretching you) there are simple, free tools that can do the job for you (it's not novel) and the user is bogus (noone's interested in it). Choose wisely!
Have clear success / failure criteria.This should be taken care of when you write your hypothesis (or thesis statement or research question). However, it's important to know what will constitute a "successful" project for you. What results do you want to end up with? Once you know that, you can choose the appropriate methods to use in your work (e.g. will you be running a focus group? Writing a questionnaire? Writing some programs -- and testing them somehow?). You can also think about reducing the risks in your project. What if you don't finish part of your work on time? Is there some catch-up time in your schedule? What if early tests go badly? Is there time to repeat them?
Don't change the world.
Don't choose a project which is just far too big. If you're going to invent a new computer, writing an OS from scratch and a windowing environment for it is not a one-person, six-month, part-time job. Keep it small and give yourself some scope for extending the project if it goes better than you thought.Even if you've chosen something small with a clear hypothesis, once you've written out a project schedule it will still feel like far too much to do. You will feel overwhelmed. The trick to reducing your fear early on (and making sure you work to schedule) is to break down your work into small tasks. Each task should have it's own goal and deliverables with it's own success criteria and a deadline. So, if your hypothesis is "WidgetFoo is easier to use than ListBoxes", your sub-goals might be:
- Produce a literature survey on list widgets.
- Write comprehensive unit tests for WidgetFoo in the Gtk widget set.
- Implement a WidgetFoo in the Gtk widget set.
- Debug (goal is to pass all unit tests).
- Devise usability experiments (deliverable: methodology document).
- Write application software for testing in the experiments.
- Perform experiments.
- Analyse data.
- Write-up thesis.
Note that these tasks are dependent on one another. WidgetFoo cannot be written before you know how to test it. The experiments cannot be run until you have designed them and decided how to analyse the data they will produce.
Your new list will make life a lot easier, but you will probably still feel that it's too much to handle. To feel better about your work and motivate yourself, it's a good idea to take each of your sub-goals and write out the next physical action you need to perform to carry out that task. So, for the literature survey the next action might be "Google for 'list widget'". For the unit testing your next action might be "Find documentation about the UnitFoo testing framework". And so on. Most people find action lists much more motivating than task lists. Scroll down to see more stuff on productivity and where to keep your lists!Choose a project that will maintain your interest.
Six months is a long time. If you choose a boring project (maybe you think it'll be "easy") then you'll quickly lose interest, get bored, stop working and possibly fail. In some ways, it's good to choose a project with a lot of scope (so you can change direction a bit and still address your hypothesis) in an active area of research (so there's lots of work to build on). On the other hand, some people would find that sort of project unfocused and confusing.When you plan your project, think hard about what motivates you. Is it reaching towards a really interesting goal, or the fear of failing really badly? Either way (or both) you need to keep that motivation in mind whenever you're working. So, choose something interesting to do, give yourself at least a little scope for changing the details of the project over the year and keep in mind why you want to succeed (or not fail!).
Don't be too dependent on clients.If you have a client to work for, especially one from industry, be careful about how you plan your project. If you are expecting resources from your client what will you do if they don't turn up on time? If you want the client to test your work, what do you do if they get a major order in when you're ready to test and noone has the time to help you out? What if the client goes out of business? What if you have to sign a non-disclosure agreement -- can the University still mark your work?Having a "real" industrial client can be a big bonus. For one thing it's very easy to say that someone is really interested in what you're doing for the project. However, you need to make sure that if the client pulls out or doesn't cooperate you'll still have a viable project. Plan well and you won't have any problems.
Understand that research is not reading and a thesis is not a report.Most undergraduates (at least in Computer Science) seem to be pretty confused about what research really is. It certainly isn'tabout using Google or reading in the library. Research means adding something new to the body of knowledge on a particular subject. This is why it's so important to know what work has already been done (so you know your work is novel), to have a clear hypothesis (so you know what new understanding you're adding) and to write up your work well (so other researchers can use it).Also, understand the place of your thesis. You are not writing a report which tells people what you did. You are writing a thesiswhich tells people about the research you have done. This can be structured in whatever sensible way you prefer, but it needs to have the following parts:
- An introduction. What's your hypothesis? Why is your work interesting? What are your trying to achieve?
- A literature survey. What have other people done? What new knowledge will your work add? What is the current state of the art missing and how are you going to address that?
- Your methodology. How did you go about validating / disproving your hypothesis? Why is your method sound? Why should anyone trust your results?
- Your results. What did you do? How?
- Your analysis of your results. What do your results mean? Why are they interesting? Did you validate your hypothesis or disprove it?
- Conclusions. What did your work contribute and how could it be continued by others?
Eat well, sleep well, get some exercise and take a day off every week.Basically, look after yourself. To work productively you need to be in good physical and mental condition. If you feel ill or your not coping well with life, slow down a bit and take a break. Don't eat junk food all the time or you'll feel sleepy and miserable. Cut down on caffeine and alcohol or you'll get stressed and sleep badly. Sleep a sensible number of hours every night -- consistency is important. Get some exercise because you need endorphins to keep you happy and some oxygen getting to your brain. Omega-3 and the sorts of minerals that aren't found in hot dogs will keep your brain working sensibly.Most of all, take a whole day off University work every single week. It doesn't matter what you do with that day (have fun, earn money, write a novel, whatever) but working every day will limit your creativity massively. Most very creative (and productive) people find that their best ideas come after a day off. This gives your mind a chance to consolidate all the material you have learned, synthesise it and solve some of the problems you've been considering. In fact, to revise for an exam or solve an interesting problem, it's a good idea to spend a few days working really hard at reading everything you need to know and taking notes, then take a day off directly before the exam or the day before you're going to write the solution to your problem. This will give you the best chance to properly understand everything you're working and be creative about it.Be productive but don't spend time on productivity.You need to organise yourself well, which is a difficult problem in itself. However, if you spend even five per cent of your time on productivity management (e.g. using Microsoft Project!) then that is far, far too much and a massive waste of your most important resource -- your time.Good productivity strategies are effortless, effective and fun to use -- and take almost no time at all. I can recommend David Allen's Getting Things Done strategy and RememberTheMilk for managing lists of action items.
One thing you can do to really give yourself a head-start in the workplace is to estimate how much time it'll take you to do every single task in your project. You'll start out finding that your estimates are stupidly far out, but as your project progresses you'll get better and better at correctly estimating your tasks. Joel Spolsky has a nice essay on how to do this simply, which you can use with RememberTheMilk
