Affiliate Positions
- Adjunct Associate Professor, University of Washington Department of Computer Science & Engineering
- Co-Founder and Chief Scientist, AnswerDash, Inc.
Specializations
- Human-Computer Interaction
- Computing Education
- Software Engineering
Research Areas
Biography
Amy J. Ko is a Professor at the University of Washington Information School and an Adjunct Associate Professor at the Paul G. Allen School of Computer Science and Engineering. She directs the Code & Cognition Lab, where she studies human aspects of programming. Her earliest work included techniques for automatically answering questions about program behavior to support debugging, program understanding, and reuse. Her later work studied interactions between developers and users, and techniques for web scale aggregation of user intent through help systems; she co-founded AnswerDash to commercialize these ideas. Her latest work investigates programming skills and new methods for learning them, including the programming language knowledge, APIs knowledge, and programming strategies. Her work spans over 80 peer-reviewed publications, 11 receiving best paper awards and 4 receiving most influential paper awards. She received her Ph.D. at the Human-Computer Interaction Institute at Carnegie Mellon University in 2008, and degrees in Computer Science and Psychology with Honors from Oregon State University in 2002.
Education
- Ph D, Human-Computer Interaction, Carnegie Mellon University, 2008
- BS, Computer Science, Psychology, Oregon State University, 2002
Awards
- Most Influential Paper - International Conference on Software Engineering, 2018
- Presidential Entrepreneurial Faculty Fellow - University of Washington, 2014
- Most Influential Paper Award - IEEE Symposium on Visual Languages and Human-Centric Computing, 2013
- National Science Foundation CAREER Award - NSF, 2010
Memberships
- IEEE
- Association for Computing Machinery
Publications and Contributions
-
Conference PaperInvestigating Novices' In Situ Reflections on Their Programming Process (2020)Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE '20), Page 149-155
-
Book, Chapter in Scholarly Book-NewA Study Design Design Process (2019)Cambridge Handbook on Computing Education Research, Page 81-101
-
Journal Article, Academic JournalA Systematic Investigation of Replications in Computing Education Research (2019)ACM Transactions on Computing Education, Volume 19, Issue 4
-
Journal Article, Academic JournalA Theory of Instruction for Introductory Programming Skills (2019)Computer Science Education, Page 205-253
-
Conference PaperAn Item Response Theory Evaluation of a Language-Independent CS1 Knowledge Assessment (2019)Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19), Page 699-705
-
Blog postCS education policymaking: how a (state) bill becomes a (state) law (2019)Bits and Behavior
-
Book, Chapter in Scholarly Book-NewHuman-Centered Methods to Boost Productivity (2019)Rethinking Productivity in Software Engineering, Page 147-157, ISBN/ISSN: 978-1-4842-4221-6
-
Book, Chapter in Scholarly Book-NewIndividual, Team, Organization, and Market: Four Lenses of Productivity (2019)Rethinking Productivity in Software Engineering, Page 49-55, ISBN/ISSN: 978-1-4842-4221-6
-
Book, Chapter in Scholarly Book-NewLearning Outside the Classroom (2019)Cambridge Handbook on Computing Education Research, Page 749-772
-
Conference PaperTeaching Accessibility: A Design Exploration of Faculty Professional Development at Scale (2019)Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19), Page 983-989
-
Conference PaperTeaching Explicit Programming Strategies to Adolescents (2019)Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19), Page 469-475
-
Conference PaperTeaching accessibility: A design exploration of faculty professional development at scale (2019)SIGCSE 2019, Proceedings of the 50th ACM Technical Symposium on Computer Science Education
-
Book, Chapter in Scholarly Book-NewTools and Environments (2019)Cambridge Handbook on Computing Education Research, Page 639-662
-
Conference PaperTowards Validity for a Formative Assessment for Language-Specific Program Tracing Skills (2019)Proceedings of the 19th ACM Koli Calling International Conference on Computing Education Research (Koli Calling '19), Page 1-10
-
Journal Article, Academic JournalWhat Distinguishes Great Software Engineers? (2019)Empirical Software Engineering, Page 322–352
-
Book, Chapter in Scholarly Book-NewWhy Not to Measure Productivity (2019)Rethinking Productivity in Software Engineering, Page 21-26, ISBN/ISSN: 978-1-4842-4221-6
-
Conference PaperEmpowering families facing English literacy challenges to jointly engage in computer programming (2018)Proceedings of SIGCHI Human Factors in Computing Systems (CHI 2018)
-
Blog Post
-
Conference PaperInformal computing mentoring of low-income adolescents: Relationships, roles, qualities, and impact (2018)Proceedings of SIGCSE 2018
-
Conference PaperRewire: Interface Design Assistance From Examples (2018)2018 ACM CHI Conference on Human Factors in Computing Systems
-
Blog PostTen years after the Whyline (2018)Medium
-
Conference PaperA Three-Year Participant Observation of Software Startup Software Evolution (2017)International Conference on Software Engineering (ICSE), Software Engineering in Practice
-
Conference PosterAccessibility as a First-Class Concern in Teaching GUIs and Software Engineering (2017)SIGCSE '17: Proceedings of the 48th ACM Technical Symposium on Computing Science Education, ISBN/ISSN: 978-1-4503-4698-6
-
Conference PaperBarriers Faced by Coding Bootcamp Students (2017)Proceedings of ICER ’17
-
Conference PaperBarriers Faced by Coding Bootcamp Students (2017)13th Annual ACM International Computing Education Research (ICER) Conference
-
Conference PaperBarriers faced by coding bootcamp participants (2017)ACM International Computing Education Research Conference
-
Conference PaperComprehension First: Evaluating a Novel Pedagogy and Tutoring System for Program Tracing in CS1 (2017)13th Annual ACM International Computing Education Research (ICER) Conference
-
Conference PaperComprehension First: Evaluating a Novel Pedagogy and Tutoring System for Program Tracing in CS1 (2017)ACM Conference on International Computing Education Research
-
Conference PaperComprehension first: evaluating a novel pedagogy and tutoring system for program tracing in CS1 (2017)ACM International Computing Education Research Conference
-
Conference PaperComputing mentorship in a software boomtown: relationships to adolescent interest and beliefs (2017)ACM International Computing Education Research Conference (ICER)
-
Conference PaperCross-Disciplinary Perspectives on Collaborations with Software Engineers (2017)International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)
-
Journal Article, Academic JournalGeneral Principles for a Generalized Idea Garden (2017)Journal of Visual Languages & Computing, Volume 39, Page 51-65
-
Conference PaperGenie: Input Retargeting on the Web through Command Reverse Engineering (2017)ACM Conference on Human Factors in Computing Systems (CHI)
-
Magazine/Trade PublicationAccessComputing Promotes Teaching Accessibility (2016)ACM Inroads, Volume 7, Issue 4, Page 65-68
-
Magazine/Trade PublicationProgrammers are Users Too: Human Centered Methods for Improving Tools for Programming (2016)IEEE Computer, Volume 49, Issue 7, Page 44-52
-
Conference PaperProgramming, Problem Solving, and Self-Awareness: Effects of Explicit Guidance (2016)ACM Conference on Human Factors in Computing Systems (CHI)
-
Conference PaperThe Role of Self-Regulation in Programming Problem Solving Process and Success (2016)ACM International Computing Education Research Conference (ICER)
-
Book, Chapter in Scholarly Book-NewThe World is Your Test Suite (2016)Perspectives on Data Science for Software Engineering
-
Workshop PaperWhat is a Programming Language, Really? (2016)Proceedings of the 7th International Workshop on Evaluation and Usability of Programming Languages and Tools, Page 32-33
-
ColumnWhy the Software Industry Needs Computing Education Research (2016)The Huffington Post
-
Conference PaperComparing the Effectiveness of Online Learning Approaches on CS1 Learning Outcomes (2015)ACM International Computing Education Research Conference
-
Conference PaperExplaining Visual Changes in Web Interfaces (2015)UIST - ACM Symposium on User Interface Software and Technology
-
Conference PaperFrom User-Centered Design to Adoption-Centered Design: A Case Study of a Research System Becoming a Product (2015)Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ’15), Page 1749-1785
-
Conference PaperIt's the Principle(s) of the Thing! A Principled Evaluation for a Principled Idea Garden (2015)IEEE Symposium on Visual Languages and Human-Centric Computer (VL/HCC)
-
Conference PaperChallenging Stereotypes and Changing Attitudes: The Effect of a Brief Programming Encounter on Adults' Attitudes toward Programming (2014)ACM Symposium on Computer Science Education
-
Conference PaperPrinciples of a Debugging-First, Gender-Inclusive, Help-Yourself Puzzle Game for Computing Education (2014)IEEE Conference on Visual Languages and Human-Centric Computing
-
Workshop PaperThirty Years of Software Problems in the News (2014)International Workshop on Cooperative and Human Aspects of Software Engineering
-
Conference PaperA Multi-Site Field Study of Crowdsourced Contextual Help: Usage and Perspectives of End-Users and Software Teams (2013)ACM Conference on Human Factors in Computing Systems
-
Journal Article, Academic JournalA Practical Guide to Controlled Experiments of Software Engineering Tools with Human Participants (2013)Empirical Software Engineering, Page 1382-3256
-
Workshop PaperCrowdsourced Q&A-based Contextual Help for Web Applications: Challenges and Opportunities (2013)CSCW Workshop on Social Media Question Asking
-
Conference PaperIn-Game Assessments Increase Novice Programmers’ Engagement and Learning Efficiency (2013)International Computing Education Research Conference
-
Conference PaperInteractive Record/Replay for Web Application Debugging (2013)ACM Symposium on User Interface Software and Technology, Page 473-484
-
Conference PaperDesigning for a billion users: A case study of Facebook (2012)ACM Conference on Human Factors in Computing Systems, Extended Abstracts
-
Conference PaperInvestigating the Role of Purposeful Goals on Novices' Engagement in a Programming Game (2012)IEEE Symposium on Visual Languages and Human-Centric Computing
-
Conference PaperIs this what you meant? Promoting listening on the web with reflect (2012)ACM Conference on Human Factors in Computing Systems
-
Conference PaperLemonAid: Selection-based crowdsourced contextual help for web applications (2012)ACM Conference on Human Factors in Computing Systems
-
Conference PaperMining whining in support forums with Frictionary (2012)Extended Abstracts on ACM Conference on Human Factors in Computing Systems
-
Conference PaperA case study of post-deployment user feedback triage (2011)International Workshop on Cooperative and Human Aspects of Software Engineering, Page 1-8
-
Conference PaperCharacterizing the differences between pre- and post-release versions of software (2011)International Conference on Software Engineering, Page 716-725
-
Conference PaperDesign, discussion, and dissent in open bug reports (2011)iConference 2011, Page 106-113
-
Conference PaperFeedLack detects missing feedback in web applications (2011)ACM Conference on Human Factors in Computing, Page 2177-2186
-
Conference PaperPersonifying programming tool feedback improves novice programmers' learning (2011)International Computing Education Research Conference, Page 109-116
-
Conference Short PaperPost-deployment usability: A survey of current practices (2011)ACM Conference on Human Factors in Computing Systems, Page 2243-2246
-
Conference Short PaperThe role of conceptual knowledge in API usability (2011)IEEE Symposium on Visual Languages and Human-Centric Computing, Page 173-176
-
Journal Article, Academic JournalThe state of the art in end-user software engineering (2011)ACM Computing Surveys 43, 3, Article 21 (April 2011), Page 44 pages
-
Conference Workshop PaperUsing crowdsourcing in the design of context-sensitive help for web applications (2011)CHI 2011 Workshop on “Crowdsourcing and Human Computation”
-
Journal Article, Academic JournalWhy-oriented end-user debugging of naive Bayes text classification (2011)ACM Transactions on Interactive Intelligent Systems, Volume 1, Issue 1
-
Conference PaperCleanroom: Edit-time error detection with the uniqueness heuristic (2010)IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), Page 7-14
-
Journal Article, Professional JournalExtracting and answering why and why not questions about Java program output (2010)ACM Transactions on Software Engineering and Methodology, Volume 20, Issue 2
-
Conference PaperGestalt: integrated support for implementation and analysis in machine learning (2010)ACM Symposium on User Interface Software and Technology (UIST 2010), Page 37-46
-
Conference PaperHow power users help and hinder open bug reporting (2010)ACM Conference on Human Factors in Computing Systems (CHI ’10), Page 2337-2346
-
Conference PaperUnderstanding expressions of unwanted behaviors in open bug reporting (2010)IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), Page 203-206
-
Book, Chapter in Non-Scholarly Book-NewUnderstanding software engineering through qualitative methods (2010)Making Software: What Really Works and Why We Believe It, Page 14
-
Conference PaperUnderstanding usability practices in complex domains (2010)Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ’10), Page 2337-2346
-
Conference PaperAttitudes and self-efficacy in young adults' computing autobiographies (2009)IEEE Symposium on Visual Languages and Human-Centric Computer (VL/HCC), Page 67-74
-
Conference PaperComparing bioinformatics software development by computer scientists and biologists: An exploratory study (2009)Proceedings of the 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering, Page 72-79
-
Conference Workshop PaperDesigning software for unfamiliar domains (2009)International Workshop on Cooperative and Human Aspects of Software Engineering
-
Conference PaperFinding causes of program output with the Java Whyline (2009)ACM Conference on Human Factors in Computing Systems (CHI), Page 1569-1578
-
Conference PaperFixing the program my computer learned: Barriers for end users, challenges for the machine (2009)Conference on Intelligent User Interfaces (IUI), Page 187-196
-
Conference PaperPast, present and future of programming in HCI (2009)Human-Computer Interaction Consortium
-
Journal Article, Professional JournalReflections on the future of iSchools from inspired junior faculty. (2009)interactions, Volume 16, Issue 5, Page 69-71
-
Conference PaperAsking and answering questions about the causes of software behaviors (2008)
-
Conference PaperDebugging reinvented: Asking and answering why and why not questions about program behavior (2008)International Conference on Software Engineering, Page 301-310
-
Conference PaperDesigners' natural descriptions of interactive behaviors (2008)IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), Page 185-188
-
Conference PaperHow designers design and program interactive behaviors (2008)IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), Page 177-184
-
Conference Workshop PaperSource-level debugging with the Whyline (2008)International Workshop on Cooperative and Human Aspects of Software Engineering, Page 69-72
-
Conference Workshop PaperEnd User Software Engineering (2007)CHI 2007 Special Interest Group Meeting, Page 2125-2128
-
Conference PaperInformation needs in collocated software development teams (2007)International Conference on Software Engineering, Page 344-353
-
Conference PaperLet's go to the whiteboard: How and why software developers draw code (2007)ACM Conference on Human Factors in Computing Systems, Page 557-566
-
Conference PaperA linguistic analysis of how people describe software problems in bug reports (2006)IEEE Symposium on Visual Languages and Human-Centric Computing, Page 127-134
-
Journal Article, Professional JournalAn exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks (2006)IEEE Transactions on Software Engineering, Volume 33, Issue 12, Page 971-987
-
Conference PaperAnswering why and why not questions in user interfaces (2006)ACM Conference on Human Factors in Computing Systems, Page 397-406
-
Conference PaperBarista: An implementation framework for enabling new tools, interaction techniques and views for code editors (2006)ACM Conferences on Human Factors in Computing Systems, Page 387-396
-
Conference PaperDebugging by asking questions about program output (2006)International Conference on Software Engineering, Page 989-992
-
Conference PaperDimensions characterizing programming feature usage by information workers (2006)IEEE Symposium on Visual Languages and Human-Centric Computing, Page 59-62
-
Conference Workshop PaperEnd-user programming productivity tools (2006)Institute for Software Research
-
Conference Extended AbstractInvited research overview: End-user programming (2006)ACM Conference on Human Factors in Computing Systems, Page 75-80
-
Conference Published VideoThe Crystal framework and editor for answering why and why not questions (2006)ACM Conference on Human Factors in Computing Systems
-
Conference Workshop PaperThe role of science in supporting software development (2006)Workshop on Supporting the Social Side of Large-Scale Software Development, Page 65-69
-
Journal Article, Academic JournalA framework and methodology for studying the causes of software errors in programming systems (2005)Journal of Visual Languages and Computing, Volume 16, Issue 1, Page 41-84
-
Conference PaperCitrus: A language and toolkit for simplifying the creation of structured editors for code and data (2005)ACM Symposium on User Interface Software and Technology, Page 3-12
-
Conference PaperDesign requirements for more flexible structured editors from a study of programmers' text editing (2005)ACM Conference on Human Factors in Computing Systems, Page 1557-1560
-
Conference PaperEliciting design requirements for maintenance-oriented IDEs: A detailed study of corrective and perfective maintenance Tasks (2005)International Conference on Software Engineering, Page 126-135
-
Conference PaperExamining task engagement in sensor-based statistical models of human interruptibility. (2005)ACM Conferences on Human Factors in Computing Systems, Page 331-340
-
Conference Workshop PaperHuman factors affecting dependability in end-user programming (2005)Workshop on End-User Software Engineering (WEUSE), Page 1-4
-
Technical ReportIdentifying types of end users: Hints from an informal survey (2005)Carnegie Mellon University ISRI Technical Report, no. CMU-HCII-05-101 and Human Computer Interaction Institute Technical Report CMU-ISRI-05-110
-
Conference Published VideoThe Citrus language and user interface toolkit. Video figure (2005)ACM Symposium on User Interface Software and Technology
-
Conference PaperUsing objects of measurement to detect spreadsheet errors (2005)IEEE Symposium on Visual Languages and Human-Centric Computing, Page 22-26
-
Journal Article, Academic JournalUsing the Cognitive Walkthrough to improve the design of a visual programming experiment (2002)Journal of Visual Languages and Computing, Volume 13, Page 517-544
-
Conference PaperAn Explicit Strategy to Scaffold Novice Program TracingSIGCSE 2018
-
Conference PaperComputing Students' Learning Difficulties in HCI EducationACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
-
Conference PaperComputing Students' Learning Difficulties in HCI EducationProceedings of the 38th annual ACM conference on human factors in computing systems (CHI '20)
-
Conference PaperExperiences of Computer Science Transfer StudentsACM International Computing Education Research Conference (ICER)
-
Journal Article, Academic JournalExplicit Programming StrategiesEmpirical Software Engineering
-
Conference PaperOn Use of Theory in Computing Education ResearchACM International Computing Education Research Conference (ICER)
-
Conference PaperPedagogical Content Knowledge for Teaching Inclusive DesignACM International Computing Education Research Conference (ICER)
-
Conference PaperPredicting Abandonment in Online Coding TutorialsIEEE Symposium on Visual Languages and Human-Centered Computing
-
Conference PaperScout: Rapid Exploration of Interface Layout Alternatives through High-Level Design ConstraintsProceedings of the 38th annual ACM conference on human factors in computing systems (CHI '20)
-
Conference PaperScout: Rapid Exploration of Interface Layout Alternatives through High-Level Design ConstraintsACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
-
Conference PaperWho Teaches Accessibility? A Survey of U.S. Computing FacultySIGCSE 2018
Presentations
-
21st Century Grand Challenges in Computing Education
(2019)
Koli Calling International Conference on Computing Education Research - Koli, Finland
-
Programming: what it is and how to teach it
(2019)
ETH Zurich, Department of Computer Science, Seminar - Zurich, Switzerland
-
Empowering families facing English literacy challenges to jointly engage in computer programming
(2018)
Proceedings of SIGCHI Human Factors in Computing Systems (CHI 2018) - Montreal, Quebec, Canada
-
Informal computing mentoring of low-income adolescents: Relationships, roles, qualities, and impact
(2018)
SIGCSE 2018 - Baltimore, MD
-
UX + Engineering: Three Years in the Startup Trenches
(2017)
Amazon Stackhouse Talks - Seattle, WA
-
A Human View of Programming Languages
(2016)
ACM SIGPLAN Conference on Systems, Programming, Languages and Applications: Software for Humanity (SPLASH) - Amsterdam, the Netherlands
-
Nine Things I Learned in Startup Land
(2016)
DUB Retreat - Seattle, WA
-
Why Computer Science Departments should Embrace CS Education Research
(2016)
CRA Snowbird (invitation only biennial conference for chairs of departments) - Salt Lake City, UT
-
What Makes a Great Software Engineer?
(2015)
ACM Learning Center - Seattle, WA
-
Understanding Software Understanding
(2013)
ABB, Inc. - Raleigh, NC
-
Defect Detection for the Wayward Web
(2012)
Microsoft - Redmond, WA
-
Explaining the World of Bits and Behavior
(2012)
DUB Group, University of Washington - Seattle, WA
-
A case study of post-deployment user feedback triage
(2011)
International Workshop on Cooperative and Human Aspects of Software Engineering - Honolulu, HI
-
Defect detection for the wayward web
(2011)
Coverity - San Francisco, CA
-
Defect detection for the wayward web
(2011)
MIT - Cambridge, MA
-
Defect detection for the wayward web
(2011)
UNC Charlotte - Charlotte, NC
-
FeedLack Detects Missing Feedback in Web Applications
(2011)
ACM Conference on Human Factors in Computing - Vancouver, BC, Canada
-
The Role of Conceptual Knowledge in API Usability
(2011)
IEEE Symposium on Visual Languages and Human-Centric Computing - Pittsburgh, PA
-
Cleanroom: Edit-Time Error Detection
(2010)
Annual Conference, HICSS - Hawaii International Conference on System Sciences (IEEE) - Madrid, Spain
-
How Power Users Help and Hinder Open Bug Reporting
(2010)
Conference on Human Factors in Computing Systems, SIGCHI - Special Interest Group on Computer-Human Interaction (ACM) - Atlanta, GA
-
Human Factors in Software Development Tools
(2010)
Dagstuhl on Practical Software Testing, German Government - Dagstuhl, Germany
-
User Feedback in Software Evolution
(2010)
Symposium on HCI and Software Engineering, IBM Research Labs - Hawthorne, NY
-
Where HCI and Software Engineering Meet
(2010)
Conference for Software User Assistance, WritersUA - Seattle, WA
-
Where is the User in Software Evolution?
(2010)
Industry Talk, IBM Research Labs - Hawthorne, NY
-
Asking and Answering Questions about the Causes of Software Behavior
(2009)
Intel Research - Seattle, WA
-
Asking and Answering Questions about the Causes of Software Behavior
(2009)
IIT Mumbai - Mumbai, India
-
Where HCI and Software Engineering Meet
(2009)
iEdge Conference, Information School, University of Washington - Seattle, WA
-
Where HCI and Software Engineering Meet
(2009)
CHOOSE Forum, Swiss Informatics Society - Bern, Switzerland
-
Asking and Answering Questions about the Causes of Software Behavior
(2008)
Washington University - St. Louis, MO
-
Asking and Answering Questions about the Causes of Software Behavior
(2008)
SRI International - Menlo Park, CA
-
Asking and Answering Questions about the Causes of Software Behavior
(2008)
Accenture - Chicago, IL
-
Where HCI and Software Engineering Meet
(2008)
DUB Colloquium, University of Washington - Seattle, WA
-
Asking and Answering Questions about the Causes of Software Behavior
(2006)
Industry Talk, IBM Research Labs - Hawthorne, NY
-
Asking and Answering Questions about the Causes of Software Behavior
(2006)
Oregon State University - Corvallis, OR
-
Human-Centered Approaches to Software Engineering Research
(2006)
University of British Columbia - Vancouver, BC, Canada
-
Information Needs in Software Development Work
(2006)
Microsoft Visual Studio User Experience Group - Redmond, WA
-
Information Needs in Software Development Work
(2006)
Industry Talk, Microsoft Research - Redmond, WA
-
User Interface Design in Visual Basic.NET
(2006)
HCI for Technology Executives, Carnegie Mellon University - Pittsburgh, PA