Where does Acceptance Testing fit in with other forms of automated tests?
the point of all automated testing is feedback
If a test fails in the woods, does anyone hear it fail?
Can you think of some examples of when acceptance tests might be useful?
http://www.extremeprogramming.org/rules/functionaltests.html
Acceptance Tests are "owned" and often created by the customer / product manager, not by the developers
but if the developer breaks an acceptance test, it must be their responsibility for fixing it -- not a separate QA team
"The developers are the people who will make changes that will break tests. Therefore, they have to be the people who are responsible for making the tests pass when that happens." - Dave Farley
but also! product owners are responsible for verifying the correctness of the acceptance tests.
If a product owner protests that they are "not technical" and can't write or maintain tests, then the developers must support them in translating between human language and code.
In theory, Acceptance Tests assume the system is a "black box" -- we can't see inside.
Compared to Unit Tests, Acceptance Tests are usually...
harder to precisely specify what you're testing
(In practice, the black box is translucent -- tests often rely on some knowledge or manipulation of internals -- but opacity is a good goal.)
not all UI testing is acceptance testing
not all acceptance tests go through the UI
you can write acceptance tests without the UI
you can write isolated unit tests for UI components
you can write end-to-end tests that are not acceptance tests
Base: lots of unit tests (fast and cheap)
Middle: fewer service and integration tests
Apex: small number of UI acceptance tests (slow and expensive)
https://martinfowler.com/bliki/TestPyramid.html
BEWARE of testing core code (aka "business rules" or "domain logic") through the UI; better to isolate your logic in a domain layer and unit test the heck out of it
vs. General Purpose Language
best when it's also a Ubiquitous Language to reduce cognitive friction between layers
Tries to be a general-purpose DSL :-)
Here's what the actual source code of a Cucmber test looks like:
Feature: Is it Friday yet?
Everybody wants to know when it's Friday
Scenario: Sunday isn't Friday
Given today is Sunday
When I ask whether it's Friday yet
Then I should be told "Nope"
Programmers have to write hooks for phrases like When("^I ask whether it's Friday yet$")
but once those are coded, the QA or product team can write more tests.
When Cucumber works it's amazingly great, but most teams find it slows them down.
see more at the Cypress Lesson
see more at the Cypress Lesson
What makes a good acceptance test?
BUT some tests might be okay with sharing setup or startup; be open to that especially if your system takes a long time to start or to populate with data, or if you're doing manual testing, which is inherently slow
Bad: too much data obscures the test's meaning
createAccount('John Doe', 42, 40, '12 Main St.', 'Dallas', 'TX', ...)
deposit(10)
expect(account.balance).toBe(50);
...is that "age 42, balance $40" or "age 40, balance $42" ?
Better: hide all details except the ones relevant to the test
createAccountWith({balance: 40})
deposit(10)
expect(account.balance).toBe(50);
in hardware testing, a fixture is a pre-made testing device that you plug your device into that helps you test it
in software testing, a fixture is a pre-made set of test data that you run your tests with
beforeEach
or beforeAll
hook/