| 1 |
|
| 2 | Usecases/requirements
|
| 3 | ---------
|
| 4 | Grouped by functional area. Ordered roughly in order of importance (within groups, between groups).
|
| 5 | Many of these overlap strongly with the general needs when developing and debugging.
|
| 6 |
|
| 7 | Basic
|
| 8 |
|
| 9 | - See existing testcases for a component/graph. Names/asserts
|
| 10 | - Look at a test as example usage of component. Fixture, input/output
|
| 11 | - ? See all tests in a project (regardless of their topic)
|
| 12 |
|
| 13 | Making changes
|
| 14 |
|
| 15 | - Add a new testcase, test suite
|
| 16 | - Changing testcase description. Name/assertion
|
| 17 | - Change testcase input/expected data
|
| 18 | - Change test suite fixture graph
|
| 19 | - Marking/unmarking a test case or suite as skipped
|
| 20 | - Delete a testcase, test suite
|
| 21 | - Rename a component and its associated test suite
|
| 22 | - Changing testcase input temporarily, revert back to original
|
| 23 |
|
| 24 | Running
|
| 25 |
|
| 26 | - Run all tests in project explicitly/manually
|
| 27 | - Run a particular test explicitly/manually
|
| 28 | - Run only tests matching a search "grep"
|
| 29 | - Run all in project tests automatically after change
|
| 30 |
|
| 31 | Results
|
| 32 |
|
| 33 | - Know if a particular version of project passes tests locally
|
| 34 | - See how a particular test run failed. Compare expected/actual values of run
|
| 35 | - (v2) Compare results of two different test runs
|
| 36 |
|
| 37 | Debugging
|
| 38 |
|
| 39 | - See data flowing through network for a test run
|
| 40 | - Able to go into graphs of network, see data
|
| 41 | - (v2) Compare data in network between two runs
|
| 42 | - (v2) Ability to set data-breakpoints on edge
|
| 43 |
|
| 44 | Retroactive debugging
|
| 45 |
|
| 46 | - (v2) Open a flowtrace from a failing test, debug as if ran interactively
|
| 47 |
|
| 48 | Continious Integration
|
| 49 |
|
| 50 | - (v2?) Know if a particular version of code passes in CI env
|
| 51 | - (v2) See details of CI results
|
| 52 |
|
| 53 | Searching
|
| 54 |
|
| 55 | - ? finding a particular test case by looking up on data or description (name/assertion)
|
| 56 |
|
| 57 | Implementation
|
| 58 | --------------
|
| 59 |
|
| 60 | - Reusing Runner code from CLI tool.
|
| 61 | For maximum compatibility between interactive/UI and non-interative/CLI
|
| 62 | - fbp-spec UI library
|
| 63 | - showing & editing testcases
|
| 64 | - showing results overview
|
| 65 | - Collaborates with other UI libraries
|
| 66 | - Graph visualization/editing: the-graph
|
| 67 | - Data/packet editing & comparison. Extract from noflo-ui/node-inspector: the-data?
|
| 68 | Individual editors/comparators should be plugins.
|
| 69 | Should have small on-canvas representation, and (full) details view?
|
| 70 | - Program introspection. Maybe extract from noflo-ui?: the-timeline?
|
| 71 | - Integrated into IDE (Flowhub).
|
| 72 |
|
| 73 |
|
| 74 | UI pieces
|
| 75 | ---------
|
| 76 | Sorted by level-of-detail.
|
| 77 | Should be able to navigate into (more details) and up (less details).
|
| 78 | Some levels might be visible at the same time on desktop view.
|
| 79 |
|
| 80 | (0) Project overview
|
| 81 |
|
| 82 | - Test status. Green/red, N pass/fail
|
| 83 | - CI status. Green/red, N pass/fail
|
| 84 | - Component/graph listing
|
| 85 | - Run all tests in project
|
| 86 |
|
| 87 | (1) Topic overview
|
| 88 |
|
| 89 | - Test case listing, grouped by suite. Suite names
|
| 90 | - Test names+assertion, status (pass/fail). Read-only
|
| 91 | - Run all test for topic
|
| 92 |
|
| 93 | (2) Test details
|
| 94 |
|
| 95 | - Fixture graph. Editable (behind lock?)
|
| 96 | - Expected values
|
| 97 | - Run a single test
|
| 98 |
|
| 99 | (3) Program / test-run details
|
| 100 |
|
| 101 | - Navigate into subgraphs/components
|
| 102 | - Network/edge data introspection
|
| 103 |
|
| 104 |
|
| 105 | # Cell-based
|
| 106 |
|
| 107 | Each row is a test case.
|
| 108 |
|
| 109 | description inports outports/expected actual status
|
| 110 | ==========================================================================
|
| 111 | [ A ] [ B ] [out] [err]
|
| 112 | foo and bar foo bar foobar foobar V 500ms
|
| 113 | should equal foobar
|
| 114 | -----
|
| 115 | foo and baz foo bar msg foobaz X 10ms
|
| 116 | should error
|
| 117 | ----
|
| 118 | .... x y xy ... running ...
|
| 119 |
|
| 120 |
|
| 121 | - Clicking on inport or expected field allows editing the data.
|
| 122 | - Maybe expected/actual should be inside same column, side-by-side or top/bottom?
|
| 123 | - Realistically, might need to show/edit diffs/input/expected data in 'details' widget in many cases?
|
| 124 | - Selecting testcase loads the run into editor/debugger. Shows diff esxpected/actual.
|
| 125 | - Maybe expands horizontally to allow showing(more) input/expected/actual data inline?
|
| 126 | - Whole thing can be search/filterable. Both for finding case, running subsets
|
| 127 | - Can collapse down into one single line, showing current run.
|
| 128 | - When running (and not focused), moves down line-by-line as process. Count overall stats (time/pass/fail) on side
|
| 129 |
|