Acceptance Testing

Where does Acceptance Testing fit in with other forms of automated tests?


Why test?

the point of all automated testing is feedback

If a test fails in the woods, does anyone hear it fail?

Can you think of some examples of when acceptance tests might be useful?

Acceptance Tests

Acceptance Criteria Templates

Acceptance Tests in Extreme Programming

http://www.extremeprogramming.org/rules/functionaltests.html

Who owns the tests?

Acceptance Tests are "owned" and often created by the customer / product manager, not by the developers

but if the developer breaks an acceptance test, it must be their responsibility for fixing it -- not a separate QA team

"The developers are the people who will make changes that will break tests. Therefore, they have to be the people who are responsible for making the tests pass when that happens." - Dave Farley

but also! product owners are responsible for verifying the correctness of the acceptance tests.

If a product owner protests that they are "not technical" and can't write or maintain tests, then the developers must support them in translating between human language and code.

Black Box Testing

In theory, Acceptance Tests assume the system is a "black box" -- we can't see inside.

Compared to Unit Tests, Acceptance Tests are usually...

(In practice, the black box is translucent -- tests often rely on some knowledge or manipulation of internals -- but opacity is a good goal.)

Is Acceptance Testing the same as Web Testing?

Testing Pyramid

Base: lots of unit tests (fast and cheap)

Middle: fewer service and integration tests

Apex: small number of UI acceptance tests (slow and expensive)

test pyramid https://martinfowler.com/bliki/TestPyramid.html

BEWARE of testing core code (aka "business rules" or "domain logic") through the UI; better to isolate your logic in a domain layer and unit test the heck out of it

Domain-Specific Language

vs. General Purpose Language

Cucumber

Tries to be a general-purpose DSL :-)

Here's what the actual source code of a Cucmber test looks like:

Feature: Is it Friday yet?
  Everybody wants to know when it's Friday

  Scenario: Sunday isn't Friday
    Given today is Sunday
    When I ask whether it's Friday yet
    Then I should be told "Nope"

Programmers have to write hooks for phrases like When("^I ask whether it's Friday yet$") but once those are coded, the QA or product team can write more tests.

When Cucumber works it's amazingly great, but most teams find it slows them down.

Web Testing Frameworks

Selenium

Cypress

see more at the Cypress Lesson

Cypress

see more at the Cypress Lesson

Good Ideas

What makes a good acceptance test?

Bad Ideas

Bad Idea: Click-and-Record

Bad Idea: Temporally Dependent Tests

BUT some tests might be okay with sharing setup or startup; be open to that especially if your system takes a long time to start or to populate with data, or if you're doing manual testing, which is inherently slow

Bad Idea: Overspecified Test Data

Bad: too much data obscures the test's meaning

createAccount('John Doe', 42, 40, '12 Main St.', 'Dallas', 'TX', ...)
deposit(10)
expect(account.balance).toBe(50);

...is that "age 42, balance $40" or "age 40, balance $42" ?

Better: hide all details except the ones relevant to the test

createAccountWith({balance: 40}) 
deposit(10)
expect(account.balance).toBe(50);

Good Idea: Fixture Data

Bad Idea: Too Much Fixture Data

 Previous Lesson Next Lesson 

Outline

[menu]

/