The COMP33511 Open Courseware Site
COMP33511 runs as a 'flipped classroom' in that I expect you to read the notes before the lecture, and then in the lecture we discuss the topics covered in the notes as well as possible exam questions, problems you are having etc.
Assessment is via both coursework and examination. Coursework comprises three short assignments (reading and a 250 word report each) worth 30% total (3 x 10%). The exam is therefore worth 70% and comprises ten multiple choice questions and five/six longer questions. Both coursework and exam are completed by electronic assessment and all questions are compulsory.
HCI Society is a group of likeminded people interested in the interaction between humans and all forms of information technology. We plan on organising our own social events, take part as a society to social events and hackathons, but also we encourage non computer scientists to join the society if they are interested in this research field.
Since the early days of computer science, with the move from punch cards to QWERTY keyboards, from “Doug Englebart's” mouse and rudimentary hypertext systems, via work on graphical user interfaces at Xerox PARC, to the desire to share information between any computer (the World Wide Web), the human has been at the heart of the system. Human computer interaction then, has had a long history in terms of computer science, but is relatively young as a separate subject area. In some ways, its study is indivisible from that of the components which it helps to make usable, however, as we shall, key scientific principles different from most other aspects of computer science, support and underlay the area; and by implication its practical application as UX.
User experience (UX or UE) is often conflated with usability but some would say takes its lead from the emerging discipline of experience design (XD). In reality, this means that usability is often thought of as being within the technical domain. Often being responsible for engineering aspects of the interface or interactive behaviour by building usability paradigms directly into the system. On the other hand user experience is meant to convey a wider remit which does not just primarily focus on the interface but other psychological aspects of the use behaviour. We’ll talk about this in more detail later, because as the UX field evolves, this view has become somewhat out of date.
This unit comprises twenty teaching sessions with one extra to cover revision topics, and one self revision lecture (which might also be a make-up lecture to cover unexpected circumstances).
You will be expected to devote further time for your own study and for the completion of your coursework (this is expected for all courses and is detailed in the course / programme handbook.).
The twenty traditional lectures will be interspersed with three discussions in which the material for the coursework (coursework will take the form of three, 250 word, discussions of key UX topics.) will be examined, after you have completed and submitted it for marking.
Submission Dates will be:
We will also be having between 1 and 2 industrial talks from organisations such as the BBC, Thoughtworks, and Barclays.
In this case the unit is composed of discussions through: (a) the principles, tools, and techniques required; (b) These course notes to expand on those principles and techniques; (c) the discussion topics to teach and test your ability to summarise, and then form a view based on a complex UX topic1; and finally, (d) the secondary text which will hopefully teach you something about the general science of the area while also enabling you to critique areas and understand how and why your view has been formed (including the ability for people to have different views which may coexist).
In addition to the unit topics, there will be an unseen quiz at the start of each lecture, along with a discussion on a question -- based on the course text -- posed at the end of the preceding lecture.
I consider that you are all adults and I will treat you as such. Attendance for all contact hours is entirely optional -- however from past years experience there is a direct correlation between students who attend and those who get over 56% overall.
I consider you all to be adults, attendance is therefore optional, however don't expect that you will be able to monopolise my `open house' near the end of the course if you haven't attended, contributed or put in some effort consistently over the full course.
Hopefully you will choose to attend every lecture, in this case I do not expect you to be asleep, reading a newspaper or some other magazines, be talking to your friends, or be using your mobile -- remember if you do not want to be at the lecture, be somewhere else.
I am here to help, if you have any problems with the course itself, the work you are expected to do, problems in general (not course related), a need for more feedback either from your coursework or from the questions posed within the course lectures, or anything else you are not clear on, then come to see me either at the end of each lecture or privately in my 'open house' sessions. If you feel you need more help, if I can't help you, or if you do not feel comfortable talking to me (maybe because you have a problem with my teaching) then you can talk in confidence with your personal tutor, or your third-year supervisor. If these tutors cannot help you, then you can talk -- again in confidence -- to the Head of Year3, or the Director of Undergraduate Studies4.
We are all here to help whatever happens do not sit on a problem, the sooner we know there is a problem the sooner we can help you address it:
These discussion topics are provided to help you form your opinion about a certain aspect of UX which needs closer inspection, while at the same time teaching you how to read, summarise, and critique2 a piece of work such that you can form your own opinions about that work.
Both the School and University take plagiarism very seriously, in the context of your coursework let us just revisit what this actually means. Plagiarism is presenting the ideas, work or words of other people without proper, clear and unambiguous acknowledgement. It also includes self-plagiarism (which occurs where, for example, you submit work that you have presented for assessment on a previous occasion), and the submission of material from essay banks (even if the authors of such material appear to be giving you permission to use it in this way). Obviously, the most blatant example of plagiarism would be to copy another students work. Hence it is essential to make clear in your work the distinction between: the ideas and work of other people that you may have quite legitimately exploited and developed, and the ideas or material that you have personally contributed. To assist you, here are a few important dos and don'ts:
You should note that a quote in quotation marks, either indented or not, is not plagiarism as long as you cite its source. Further, it will not be included in the 250 words (details coming up) and neither will the citation or the references.
As we have already seen 30% of the assessment structure for this unit is made up from the three coursework components based on our discussion topics. These discussion topics are provided to help you form your opinion about a certain aspect of the course which needs closer inspection, while at the same time teaching you how to read, summarise, and critique5 a piece of work such that you can form your own opinions about that work.
You should submit all 250 word (plus or minus 10% or 25 words) coursework assignments via Blackboard. These will be checked for length and plagiarism via the Turn-It-In system; after which point they will be graded. You will receive feedback and grades within two weeks of the submission deadline. I am strict on hand-ins -- no exceptions and no exemptions. Remember you can complete all coursework as soon as you like -- the deadlines are your last possible chance to submit.
Remember, No exceptions, No exemptions! The printer will be busy -- this is not an excuse -- you don't need to print; if the power goes off or the computer explodes -- this is not an excuse -- back-ups. If your Hamster dies in a horrible 'seed-choking' accident -- this is not an excuse. You haven't got a dog -- it cannot chew your answers. And no, your USB pen has not been abducted by aliens! If you get hit by a bus; suffer a gruesome chain-saw accident; are attacked by a loose buffalo along Oxford Road I may be lenient. Finally, remember No Plagiarism, both the School and University take plagiarism very seriously.
'Understanding, Scoping and Defining User Experience: A Survey Approach'[Law et al., 2009] (10 Marks) -- this work will enable you to understand the scope and the inconsistencies still present within the UX domain. It will enable you to understand that the definition of UX is not yet fixed and is someway based on the interpretation of the practitioner;
'Designing the Star User Interface' [Smith et al., 1982] (10 Marks) -- the Star interface is really where all GUI interfaces began. It takes the user as a first and primary priority in the design and it is inconceivable that you do not have an awareness of these classic design principles as perspective computer science graduates; and
'Voice Loops as Cooperative Aids in Space Shuttle Mission Control' [Watts et al., 1996] (10 Marks) -- this paper shows just how far UX and the techniques which it inherits from human computer interaction can go. We are mainly concerned with systems and objects which are purely commercial, however, in this case failures in the human interface can have serious consequences for a real-time mission, including the loss of the vehicle. Further, these kind of UX techniques can also be found in other critical interface components such as those controlling nuclear power stations or fly-by-wire aircraft.
So the 'question' that you need to answer for each of the three assignments is that which is asked of UX'ers in most companies.
The questions from managers often are phrased thus:
So the skill is to be able to summarise a paper while also adding in your expertise and original thought - coming up with 250 words (the industry standard) of insight which someone without your training in UX or CS could not produce.
You interpret the paper, add your insight (using experience created from your UX/CS training), and produce a 'mash-up' of the two focusing on aspects of the paper you think are important, rationalising why, and linking it to other work you have read, work you have done or seen, prior knowledge, or real world experience.
It is difficult to not just produce a summary (but a summary is often never required). Think to yourself, what do I bring to this 250 words (± 10%), could anyone have created the 250 words (± 10%) by just reading the paper and without your training and insight. If the answer to the last question is 'yes' then you need to change adding your insight based on your expertise.
You should submit all 250 word (± 10%) coursework assignments via Blackboard. These will be checked for length and plagiarism via the Turn-It-In system; after which point they will be graded. You will receive feedback and grades within two weeks of the submission deadline. I am strict on hand-ins -- no exceptions and no exemptions. Remember you can complete all coursework as soon as you like -- the deadlines are your last possible chance to submit.
Students tell me that when trying to write the coursework they constantly find that 250 words is completely insufficient to properly make any point that they would like to make about the paper that they have read. They are not suggesting that these be made into 1000 word essays as that would just be unfair and way too much work. They do however suggest that perhaps the guideline length to 250-500 words with a 10% tolerance at either end meaning an overall length of between 225 and 550 words would be better. They think this would allow for some more in depth observations without the pressure of such a low word limit as it often doesn't seem right to make two to three (perhaps flimsy, due to the word cap) points and to sum your opinions all in the space of 225 to 275 words.
I quite understand their point; 250 words is very short. However, you may have guessed by now that I don't do very much without a reason. So the reason why it is 250 words is simple - this is the industry standard. You need to be able to summarise in 250 words - and I fear you've not been exposed to this in CS up to now. Remember, you don't need to say everything - you need to pick out the most important stuff based on your experience. It may be tough now - but you'll be expected to write it in 250 words in business - and I'd rather this occurs now and you don't have to learn within a company settings. If you don't believe me have a look at the ACM DL for abstracts - it's the same for all Science, Engineering, and Medical domains - 250 words.
You should note that a quote in quotation marks, either indented or not, is not plagiarism as long as you cite its source. Further, it will not be included in the 250 words and neither will the citation or the references.
Your submission will be marked according to the following rubric, which has three components marked from A--E:
Students have been asked to discuss each of the following papers and make 250 word submissions for each, according to the timetable , and based on the rubric you will use to mark them.
to turn around feedback for each submission, with a final feedback deadline two weeks after the coursework submission deadline.
You will need to have read each paper, and formed your own opinions about it before marking the work. The objective is not to slavishly follow a set template but to make sure the students know the content of the work, and understand that content in enough detail to form a critical opinion of that work. As there are no boolean right of wrong answers the marking of each abstract will take longer than a conventional marking process; but this will, hopefully, give us better learning outcomes than parrot-fashion (thought free) assessments.
Each submission will be submitted to the Turn-It-In Plagiarism checker - if there are signs of repetition of: phraseology, ideas, or the like - beyond that which we could normally expect - when we will define this as plagiarised work. In this case you should alert me.
If you have any questions on any work as you are marking it then please forward them to me as soon as possible - so that we can maintain our two week target for feedback.
The feedback for this work should not just be via the grade, but written and informative enough for the students to understand their mistakes and derive some beneficial learning experience from the feedback -- so as to make their exam work better. The 'Grade-mark' system will be used with the rubric providing the more universal comments, then general comments entered into the system, and finally any quick comments and off-the-shelf remarks dragging to the relevant position in the text. The students should be able to get to all this information, but after marking each assignment we need to download the pdf feedback form and email it to each student.
Student 1 with no citations: A definition for User Experience (UX) is difficult. It's a new area and people's own agenda influences their views on it. Industry people believe it's about the users' experience as a whole and their view of the company/brand and answering the simple question 'Does it do what they want'. Whereas academics take into account the users emotional state, the complexity of the system and the situation they are using it in. The article sets out an important definition, that by UX we refer to the use of a particular product/service, not an overall experience of a company/brand. User Experience is subjective, we are all different and prefer different things, but maybe not completely subjective, for example, computer scientists are likely to find a command line interface acceptable. However an art student would prefer a more graphical interface. This would suggest we can group users into social communities. Experts would agree with this as the survey suggested they feel it's not subjective and I can understand this as we are all human and have similarities., such as we make mistakes, so being able to recover from them, would be a good aspect of an experience. So maybe to some extent we can apply a template to UX. It's interesting to think of UX not just as the time it occurs, but before and after. This thinking could lead to a better UX. Maybe UX is not an exact science and not a completely subjective either, but somewhere in between, with interesting avenues still to explore and that in its self will be an experience.
Student 2 - with external pointers citations: It is clear that the research in this paper aimed not only to collate the opinions among academia and industry to agree on a definition of UX, but to also measure their participant's responses and attitudes towards finding a definition. The paper appears to conclude that the work carried out by the researchers will be of benefit towards defining UX, outlining the nature of UX being one of an abstract, subjective and emotive area, though these objectives appear to somewhat contradict with their findings. For instance, their research found that most respondents disagree with there existing a 'definite need for a standardised definition of the term UX' (http://www.awebsite.com/whatimciting). As the research aims to assist work in creating a definition, this mood amongst industry and academia indicates to me that there are many interpretations of UX and perhaps these variations lead to a reluctance among practitioners to compromise on their own understanding in favour of an objective and fixed definition. Another example of this variation exists from the geographical differences1 in attitude which is briefly highlighted in the paper's conclusions, adding to the subjective nature not only of UX, but it's understanding in HCI community globally. Perhaps in order to better define UX we should be aware of it's subjective nature arid allow this very new area of work to develop free from the constraints of definitions and objectivity to lead to a more fluid and flexible domain similar to the dynamic effects it produces in users.
1. http://www.anotherwebsite.com/whatimciting
Student 3 with informal citations: The paper attempts to define User Experience (UX) by presenting the perceptions of a wide range of people on a wide range of statements and potential definitions for UX. It fails to acknowledge that coming to a consensus on the scope of UX would require coming to a consensus of the definitions of 'user' and 'experience'. The paper instead focuses on some factors that contribute to the generation of an experience, and some characteristics of a user1. The factors and characteristics are secondary to the supposed problem; what is a user, and what is an experience? Furthermore, what is a good experience? One could argue that any experience must be had by some thing, and that this thing„ by experiencing something, is consequently a user. Does overgrowth provide a good UX for a crumbling building in a forest, in that it prolongs the structural integrity of the building, or vice versa, because buildings aren't alive? Furthermore, the purported advantages of having a universal definition for UX are instead the advantages of having a definition of good UX (which is hopeless). Issues with the discourse, evaluation and teaching of UX originate from the subjective question of what makes good UX, but not what UX is [1]; these problems are not unique to the field. The paper also fails to acknowledge the benefits that come from UX having a dynamic scope and definition, such as allowing interdisciplinary approaches to problems [Various 2015], enhanced versatility and applicability, and future malleability. Any universal definition of UX would be so general as to be useless.
1. http://www.anotherwebsite.com/whatimciting Accessed 01 Jun 2015.
References
[1] Various, 2015, ITV News 12 http://www.anotherwebsite.com/whatimciting Accessed 01 Jan 2017.
[Various 2015] Various, 2015, BBC News 24 http://www.anotherwebsite.com/whatimciting Accessed 01 Jun 2017.
Student 4 with formal references: There is no definitive way to classify UX [Brown 2013]-- it is a challenging concept to render. We see that our job is to uncover the semantics of interactions between the user and system, as to allow for better user experience. That is, what do we need to do to make sure user satisfaction is maximised? I comprehend its difficult because they're elements to UX which we can't easily determine and therefore transform into a 'working model', which tells us how we should stimulate the user's attention fully. Users' perception of a system can be down to personal subjectivity, determined by a series of individual emotions and sensations. Perception can also be influenced by social contexts' they're in; different communities of people which affect their experience. It's clear that different people (practitioners, scholars etc.) in fields relating to UX have various ideologies as to what UX is. Taking a range of different perceptions and gathering them carefully into a unified definition is one approach to solving this problem. Trying to map qualitative information (open definitions) of what people understand about UX into quantitative information (statistical information) may give rise into helping define what UX is [1]. I don't believe this is trivial as people's apprehensions are subjective, and could be very difficult to find an adequate scope to cover all possible areas of UX. We do know, however, that the very concept is dynamic which stems from individual human responses, and involvement in varying social contexts', to the use of an artefact.
References
Brown 2015 - Brown, A., and Harper, S. Dynamic injection of wai-aria into web content. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (New York, NY, USA, May 2013), W4A ’13, ACM, pp. 14:1–14:4.
[1] US Department of Labour. Bureau of Labor Statistics. 2015-06-25. URL:http://www.bls.gov/data/. Accessed: 2015-06-25. (Archived by WebCite at http://www.webcitation.org/6ZY9hwsm3) (06 2015).
Student 5 with quotation (Quotations in quote marks and indented are no plagiarism as long as they are cited, and are not counted in the word count): This paper offers an informative insight into the thoughts and views on UX from both academics, and people working within this domain in industry. The results of this paper show that in some areas, both academia and industry are 'agreed', and in others their views differ somewhat. The real challenge appears to be the ability to find a 'consensual definition of UX.
"The much vaunted Web 2.0 sees once static pages evolving into hybrid applications. Content that was once simple to surf is now becoming increasingly complicated due to the many areas of dynamic content "dotted" throughout the page. In previous studies, we have shown that unlike younger users, older users have more varied interaction patterns when using dynamic content." -- Lunn 2011
The survey approach used in this paper shows that whilst all parties seem to agree that a definition is needed, it also shows that there are several variables to take into account, not all of which seem to have the same weighting of importance to each individual. This shows is that finding an 'ultimate' definition is going to be challenging and will not satisfy everyone's ideals. In order to create a shared definition, we must first consider the 'emotional, affective, experiential, hedonic, and aesthetic' [1] variables governing the concept of UX We must also clarify what exactly we are defining, for example, is it a single aspect of an application, or is it the usability of the full system. Only once these have been addressed., can a shared definition be put forward. I feel that several definitions will be required in order to satisfy all the 'criteria' (a definition for various single aspects as well as a definition for a full system). Due to the varied nature of applications, it seems to me that if a shared definition were to be agreed on, then it would need to be relatively 'vague', so as to be applicable for the majority of applications. [1] Understanding, scoping and defining user experience: a survey approach.
References
Lunn 2011 -- Lunn, D., and Harper, S. Improving the accessibility of dynamic web content for older users. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (New York, NY, USA, April 2011), W4A ’11, ACM, pp. 16:1–16:2.
Student 6 with no citations: There exists vast ambiguity when precisely defining the User Experience domain. The paper attempts to address this issue through knowledge elicitation involving industry/academic professionals. A survey formalised responses, answering questions covering definitions, opinions and respondents' background. Statistical analysis was then used to highlight patterns in an attempt to derive an accurate definition. This is important due to the current disparity in describing what is UX? Currently the UX domain encompasses everything from the explicit end-user interactions with a company/system to the tacit emotions elicited through reflection. Within this scope the research highlighted a geographic divide in opinion on the 'subjectivity' of UX, describing disagreements on the importance of user perception over performance. These results indicate fundamentally different approaches to UX within the USA and Europe further strengthening the argument for standardisation. Interestingly, although many of the background variables proved fruitless, there existed an inverse proportionality between the need for standardisation and an individual's level of expertise. Indicating the value in a precise definition would most benefit those who are inexperienced within the UX field. Clearly there is widespread ambiguity in defining UX within the community. The recommendation is to impose constraints on the domain to establish a definition based upon user-interface interactions, which places general experiences out of scope. Reining in what UX encompasses will benefit the field, allowing for focused research on 'measurable aspects' to drive innovation. Reflecting, it is clear I have come to appreciate the significance in scope and temporal properties that contribute to the 'user experience'. Accentuating that UX reaches far deeper than simply a cognitive process.
My Attempt: I understand that it may be difficult to realise just exactly what I would like from your 250 word discussions. In this case I think an example is appropriate, and I have chosen a very famous human factors research paper, written in the mid--Fifties and concerning Psychoacoustics -- 'The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information' [Miller, 1956]. This paper is interesting because it has spawned, an often incorrectly understood, usability principle -- being that our working memory can only handle seven (±) items at any one time and therefore we should only make menus or lists a maximum of seven items long. Here is my discussion:
"Miller's paper is a summary/analysis of research findings couched in the form of bits per channel. He amusingly, relates the fact that the number seven seems to be plaguing his every move because seven seems to appear repeatedly in regard to the amount of information that can be remembered or differentiated by humans. However there are two caveats to this assertion: firstly, that the user is required to make absolute judgements of uni-dimensional stimuli and secondly that that stimuli is not clustered. This last point is quite important because using clustering means that we can remember or distinguish more than the unitary seven. For instance, we can remember seven characters in sequence, seven words in order, or seven phrases. In reality then, we have trouble differentiating uni-dimensional stimuli such as audible tones played without reference to each other, but we can differentiate more than seven tones when played in a sequence, or separately when multiple dimensions such as loudness and pitch are varied. Further, we are able to remember more then seven things within a list especially if those things are related or can be judged relatively, or occur as part of a sequence. So the well found psychological finding that working memory can handle seven (±2) arbitrarily sized chunks of absolute uni-dimensional stimuli, becomes the often quoted but mostly incorrect usability/HCI principle that you should only include seven items in a menu, or seven items in a list... and on and on."
If you have any questions on any work as you are marking it then please forward them to me as soon as possible - so that we can maintain our two week target for feedback.
Once you've completed this course you will have covered a number of School/University stipulated learning outcomes. These count towards your final degree:
Learning outcomes are variously assessed by Examination (70%) and Coursework (30%):
As we have already seen 30% of the assessment structure for this unit is made up from the three coursework components, the remaining 70% is from the final examination. This examination will be 1h:30m long and will be in two parts. The first part will be composed of 10 compulsory multiple-choice questions (no negative marking); while the second part will be a choice of five/six longer questions again compulsory. The questions on the second part will require longer answers and will be made up of sub questions; with the topics drawn randomly from the unit book. This method has served well over the last few years giving averages of 60% inline with differences from the students average performance of 1.3.
The structure of both parts will comprise: bookwork; discussions (and examples); application of technique; and original thought. You will not be able to attain a first (over 70%) without being able to demonstrate your ability to apply original thought to a problem, and you will not be able to achieve an upper second class (2:1, 60%) without being able to demonstrate a good grasp of the application of various techniques.
In a break with the tradition let us look at the kinds of questions you will be expected to answer when it comes to the completion of this exam:
You'll notice that I've left last years posts as an archive just so you can look ahead as we go if you'd like. You'll also notice that there is information that you should look at immediately under the 'Important' menu item, and this will be updated as the unit progresses.
[Law et al., 2009] Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P., and Kort, J. (2009). Under- standing, scoping and defining user experience: a survey approach. In Proceedings of the 27th international conference on Human factors in computing systems, CHI ’09, pages 719–728, New York, NY, USA. ACM.
[Miller, 1956] Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review, 63(2):81–97.
[Pirsig, 1974] Pirsig, R. M. (1974). Zen and the art of motorcycle maintenance: an inquiry into values. Morrow, New York.
[Roto et al., 2011] Roto, V., Law, E., Vermeeren, A., and Hoonhout, J. (2011). USER EXPERIENCE WHITE PAPER: Bringing clarity to the concept of user experience. Technical report, AllAbouUX - http://www.allaboutux.org/uxwhitepaper.
[Watts et al., 1996] Watts, J. C., Woods, D. D., Corban, J. M., Patterson, E. S., Kerr, R. L., and Hicks, L. C. (1996). Voice loops as cooperative aids in space shuttle mission control. In Proceedings of the 1996 ACM conference on Computer supported cooperative work, CSCW ’96, pages 48–56, New York, NY, USA. ACM.
[Smith et al., 1982] Smith, D. C., Irby, C., Kimball, R., Verplank, B., and Harslem, E. (1982). Designing the star user interface. BYTE, 7(4):242–282.
In 2003, Rob Howard described a meeting with Bill Gates
The first thing I notice as the meeting starts is that Bill is left-handed. He also didn’t bring a computer in with him, but instead is taking notes on a yellow pad of paper. I had heard this before – Bill takes amazingly detailed notes during meetings. I image he has to, given all the information directed at him. The other thing I noticed during the course of the meeting is how he takes his notes. He doesn’t take notes from top-to-bottom, but rather logically divides the page into quadrants, each reserved for a different thought. For example, it appeared that all his questions were placed at the bottom of the page.
Indeed, as UX is a new area you will need to be able to read and form an opinion about the work as it is published.↩
'Critique is a method of disciplined, systematic analysis of a written or oral discourse. Critique is commonly understood as fault finding and negative judgement, but it can also involve merit recognition, and in the philosophical tradition it also means a methodical practice of doubt.' - wikipedia↩
Tim Morris, Rm 2.107.↩
Toby Howard, Rm 2.96.↩
'Critique is a method of disciplined, systematic analysis of a written or oral discourse. Critique is commonly understood as fault finding and negative judgement, but it can also involve merit recognition, and in the philosophical tradition it also means a methodical practice of doubt.' - wikipedia↩