class: centre, middle, inverse name: UX from 30,000ft specific: COMP33511 topic: Designing Your Evaluations (part 2) # {{name}}: {{topic}} ### Lecture 16 (50 minutes) ### [@sharpic](http://twitter.com/sharpic) .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] --- layout: true class: left, middle name: UX from 30,000ft noteid: Ch 09 - Designing Your Evaluations specific: COMP33511 website: http://sharpic.github.io/COMP33511 author: [@sharpic](http://twitter.com/sharpic) --- class: middle topic: Hypothesis Testing .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} .av[
.caption[Can You Solve This? .credit[Veritasium]]] --- class: middle topic: Evaluation Design and Analysis .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} 1. Experimental Design; 1. Data Collection and Tools; 1. Data Analysis; mostly 1. Statistical Analysis. --- class: middle topic: Evaluation Design and Analysis .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} 1. Descriptive Statistics; 1. Inferential Statistics. -- 1. Internal Validity; 1. External Validity; and 1. Confounding Variables. --- class: middle topic: Sampling (Participants) .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} 1. **Simple Random Sampling** *Probabilistic* --- Simple random sampling equates to drawing balls at a tom-bola. The selection of the first has no bearing, and is fully independent of, the second or the third, and so forth. This is often accomplished in the real world by the use of random number tables; 1. **Systematic Sampling** *Probabilistic* --- Systematic samples are a variation of random sampling whereby each possible participant is allocated a number, with participants being selected based on some systematic algorithm. In the real world we may list participants numbering them from, say, one to three hundred and picking every seventh participant, for instance; 1. **Stratified Sampling** *Probabilistic* --- Stratified samples are used to reduce the normal sampling variation that is often introduced in random sampling methods. This means that certain aspects of the sample may become apparent as that sample is selected. In this case, subsequent samples are selected based on these characteristics; and 1. **Multistage Sampling** *Probabilistic* --- Multistage sampling is a strategy for linking populations to some kind of grouping. --- class: middle topic: Sampling (Participants) .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} 1. **Quota Sampling** *Non-Probabilistic* --- Almost all non-governmental polling groups or market research companies rely heavily on non-probability sampling methods; the most accurate of these is seen to be quota based sampling. Here, a certain demographic profile is used to drive the selection process; 1. **Snowball Sampling** *Non-Probabilistic* --- The process of snowball sampling is much like asking your participants to nominate another person with the same trait as them; 1. **Convenience Sampling** *Non-Probabilistic* --- The participants are selected just because they are easiest to recruit for the study and the UX'er did not consider selecting participants that are representative of the entire population; and 1. **Judgmental Sampling** *Non-Probabilistic* --- This type of sampling technique is also known as purposive sampling and authoritative sampling. Purposive sampling is used in cases where the specialty of an authority can select a more representative sample that can bring more accurate results than by using other probability sampling techniques. --- class: middle topic: Evaluation'++' .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} 1. Single Group, Post Test; 1. Single Group, Pre-Test and Post-Test; 1. Natural Control Group, Pre-Test and Post-Test; 1. Randomised Control Group, Pre-Test and Post-Test; 1. Within Subjects; but there are 1. Others. --- class: middle topic: Practical Ethical Procedures .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} ### The Ethical Process > A critical component of good evaluation design because it encourages the UX specialist to focus on the methodology and the analysis techniques to be used within that methodology. --- class: middle topic: Organisations .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} 1. The American Psychological Association's (APA), 'Ethical Principles of Psychologists and Code of Conduct'; 1. The United States Public Health Service Act (Title 45, Part 46, Appendix B), 'Protection of Human Subjects'; 1. The Belmont Report, 'Ethical Principles and Guidelines for the Protection of Human Subjects of Research'; 1. The Council of International Organisations of Medical Sciences, 'International Ethical Guidelines for Epidemiological Studies'; and finally 1. The World Medical Association's, 'Declaration of Helsinki -- Ethical Principles for Medical Research Involving Human Subjects'. --- class: middle topic: in Brief... .noteids[{{noteid}}] .credits[ {{author}} | UX from 30,000ft | {{specific}} | {{topic}}] .controls[[SAQ](saqs.html) [D](discuss.html) [OH](oh.html) [C](coffee.html) [P](coffee.html#2) [SLIDES](http://sharpic.github.io/COMP33511/#slides) [↩](#)] ## {{topic}} 1. **Respect**: Assess you participants autonomy and capability of self-determination, treat participants as equals, ensure their welfare; 1. **Benefits**: Maximising benefits and minimising possible harms according to your best judgement, seek advice from your organisations ethics committee; 1. **Justice**: Research should be undertaken with participants who will benefit from the results of that research; and 1. **Trust**: Maintain trust, anonymity, confidentiality and privacy, ensure participants fully understand their roles and responsibilities and those of the experimenter. -- 1. **Responsibility**: You have a duty of care, not only to your participants, but also to the community from which they are drawn, and your own community of practice.