Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IS-12 & IS-14 Test Suites #810

Open
wants to merge 113 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
ec40a1d
[IS-12] Initial Commit for IS-12 Test Suite (#800)
jonathan-r-thorpe May 9, 2023
419ef85
[IS-12] Check Root Block and ClassManager (#814)
jonathan-r-thorpe Jun 21, 2023
c033475
[IS-12] Add auto tests for validating MS-05 standard class and dataty…
jonathan-r-thorpe Jul 18, 2023
dadf52b
Merge pull request #812 from AMWA-TV/zeroconf-marmoset
garethsb Jun 28, 2023
e36e6cc
[IS 12] Add Get/Set/Add/Remove/Find methods (#822)
jonathan-r-thorpe Aug 4, 2023
f841bb7
[IS-12] Specify explicit URL path for WebSocket endpoint (#827)
jonathan-r-thorpe Aug 8, 2023
f8c63b6
Check response from Find methods is an array
jonathan-r-thorpe Aug 10, 2023
c28ee82
Handle errors where control classes are not advertised by Class Manager
jonathan-r-thorpe Aug 23, 2023
d22005d
Fixed detection of authority keys in non-standard classes
jonathan-r-thorpe Aug 24, 2023
8073aa2
Handle missing device model properties
jonathan-r-thorpe Aug 24, 2023
2bbd542
Defend against missing properties or datatype definitions
jonathan-r-thorpe Aug 24, 2023
fa810fa
Improved error handling,
jonathan-r-thorpe Oct 6, 2023
34678c8
Bump Node 12.x to Node 16.x
garethsb Aug 3, 2023
db8c8cf
python-zeroconf 0.75.0 strict=False for _nmos-registration._tcp
garethsb Aug 14, 2023
6cb8f7c
Python 3.6 (and 3.7!) is End-Of-Life, and so is Node.js 14.x
garethsb Aug 18, 2023
b3008ad
Pass IP address of API under test to testssl.sh
garethsb Oct 5, 2023
353e746
Add MOCK_SERVICES_WARM_UP_DELAY
garethsb Oct 5, 2023
23110f1
Delay running tests until after MOCK_SERVICES_WARM_UP_DELAY
garethsb Oct 5, 2023
ab07908
Tweak message
garethsb Oct 5, 2023
049d674
Fixed send_command function as per suggestion from @alabou
jonathan-r-thorpe Oct 6, 2023
b165316
Add config option to bypass IS12 interactive testing
jonathan-r-thorpe Oct 24, 2023
19f71a8
Add constraints classes
jonathan-r-thorpe Oct 24, 2023
b73cd22
added xeger library
jonathan-r-thorpe Oct 24, 2023
ddc17d7
Add IS1202 test suite
jonathan-r-thorpe Oct 24, 2023
b081bb0
Add primitives to datatype models
jonathan-r-thorpe Oct 31, 2023
aa6faa9
Enhance test_35 (subscription and notification mechanism test) to all…
lo-simon Nov 6, 2023
d862805
Tweaked test_35
jonathan-r-thorpe Nov 6, 2023
6dd111d
Pass **kwargs to ControllerTest
jonathan-r-thorpe Nov 7, 2023
19d5b86
Pass **kwargs to ControllerTest
jonathan-r-thorpe Nov 7, 2023
e581283
Test sequences; remove non UTF-8 character from comment
jonathan-r-thorpe Nov 8, 2023
ce8afc0
Added constrained sequence tests
jonathan-r-thorpe Nov 8, 2023
a0937e1
Additional sequence checks
jonathan-r-thorpe Nov 8, 2023
7f84e40
Add runtime constraints to device model. Fix handling of primitive ty…
jonathan-r-thorpe Nov 10, 2023
a7af313
Added property and datatype constraints checking
jonathan-r-thorpe Nov 14, 2023
fe032c0
Removed disabled tests
jonathan-r-thorpe Nov 14, 2023
995e292
make datatype_schemas an object property
jonathan-r-thorpe Nov 14, 2023
cedacb8
Check for default value sequences when validating class descriptors
jonathan-r-thorpe Nov 14, 2023
1bb41bd
Flake8 formatting
jonathan-r-thorpe Nov 14, 2023
8409b8e
Cache class_manager. Simplify constraint checking.
jonathan-r-thorpe Nov 14, 2023
e0c6d7f
Merge remote-tracking branch 'nmos-testing-amwa/is-12' into is-12-inv…
jonathan-r-thorpe Nov 14, 2023
30d8059
Fix Block object construction. Fix Test_02
jonathan-r-thorpe Nov 15, 2023
72d070b
add missing primitive descriptor path
jonathan-r-thorpe Nov 15, 2023
9a0f6bd
reorder specs in NMOS Testing interface
jonathan-r-thorpe Nov 15, 2023
c5d43f7
Fix constraint violation error message
jonathan-r-thorpe Nov 15, 2023
83afc3b
Disable Testing Facade host and port if not interactively testing
jonathan-r-thorpe Nov 15, 2023
96aaaa9
Flake8 formatting
jonathan-r-thorpe Nov 15, 2023
c8caa23
Fix constraint violation error message
jonathan-r-thorpe Nov 15, 2023
049d62a
Refactor property testing
jonathan-r-thorpe Nov 16, 2023
5f2cb63
Handle property and method deprecated status codes
jonathan-r-thorpe Nov 16, 2023
7c93b49
Check deprecated properties
jonathan-r-thorpe Nov 17, 2023
12f556b
Report unchecked device model in tests
jonathan-r-thorpe Nov 17, 2023
738021f
Renumbered tests
jonathan-r-thorpe Nov 17, 2023
ef33d0f
updated spec URLs in comments
jonathan-r-thorpe Nov 17, 2023
23b3512
Handle property and method deprecated status codes
jonathan-r-thorpe Nov 16, 2023
0841000
Check deprecated properties
jonathan-r-thorpe Nov 17, 2023
63bafa7
Report unchecked device model in tests
jonathan-r-thorpe Nov 17, 2023
e673449
Renumbered tests
jonathan-r-thorpe Nov 17, 2023
dc2d30a
updated spec URLs in comments
jonathan-r-thorpe Nov 17, 2023
8234809
Merge remote-tracking branch 'origin/is-12-invasive' into is-12
jonathan-r-thorpe Nov 17, 2023
088b22e
Updated to conform merged branch
jonathan-r-thorpe Nov 17, 2023
af8d55e
Guard against missing class properties
jonathan-r-thorpe Dec 18, 2023
3640690
Add annotation to check properties test
jonathan-r-thorpe Dec 18, 2023
b53b562
Better error reporting for test_22 and test_23
jonathan-r-thorpe Dec 18, 2023
d5348e0
formatting
jonathan-r-thorpe Dec 18, 2023
3fa3fed
Fix spelling mistake in interactive testing config
jonathan-r-thorpe Jan 10, 2024
b1c91f4
Fixed Blocks with no members being set to None
jonathan-r-thorpe Jan 12, 2024
09f2d39
Fixed config spelling error
jonathan-r-thorpe Jan 12, 2024
b5205ac
Rename _execute_command to execute_command
jonathan-r-thorpe Jan 12, 2024
35cde03
Test discovered methods in Device Model
jonathan-r-thorpe Jan 12, 2024
fd8c9d3
Only test non-standard methods.
jonathan-r-thorpe Feb 7, 2024
95fe9e5
Moved some functions to IS12Utils
jonathan-r-thorpe Feb 14, 2024
245e3c6
Added more error reporting
jonathan-r-thorpe Feb 26, 2024
3483863
Guard against ControllerTest being run in non-interactive mode
jonathan-r-thorpe Feb 28, 2024
5a15d84
Added check constraints hierarchy test
jonathan-r-thorpe Mar 6, 2024
3260af4
Fixed do_check_property_test
jonathan-r-thorpe Mar 6, 2024
98b0a64
Add check read only properties and sequences
jonathan-r-thorpe Mar 6, 2024
3fad9a7
Made sequence tests tolerant to empty or null sequences
jonathan-r-thorpe Mar 6, 2024
a5350cd
Typo and formatting
jonathan-r-thorpe Mar 6, 2024
5ba312d
Initial add of IS-14-01 test suite
jonathan-r-thorpe Apr 23, 2024
af0c67a
Flake8 formatting
jonathan-r-thorpe Apr 26, 2024
7405325
Added Node API to IS-14 (for testing control endpoint)
jonathan-r-thorpe Apr 29, 2024
55eeb69
Initial refactoring
jonathan-r-thorpe Apr 29, 2024
7bb6a5f
Added additional specs to IS-14-01 test suite definition
jonathan-r-thorpe Apr 29, 2024
a9ed819
Move common functions from IS12Utils to MS05Utils
jonathan-r-thorpe Apr 29, 2024
03d411a
Delegate to MS0501Test for auto tests
jonathan-r-thorpe Apr 29, 2024
2fb54b2
API key now required when loading resources
jonathan-r-thorpe Apr 29, 2024
0062040
Delegate to MS0501Test for auto tests
jonathan-r-thorpe Apr 29, 2024
84959a6
flake8 formatting
jonathan-r-thorpe Apr 29, 2024
95a1eac
Move MS-05 tests into MS0501Test
jonathan-r-thorpe May 1, 2024
ee4888a
Fixed get_sequence_item
jonathan-r-thorpe May 1, 2024
223b29d
Refactor IS-14 API utility functions
jonathan-r-thorpe May 2, 2024
2dd9973
generalize get_control_class and get_datatype. Improve error handling.
jonathan-r-thorpe May 2, 2024
089a2b0
Implement IS-14 http request helper functions
jonathan-r-thorpe May 2, 2024
dc9608a
Formatting
jonathan-r-thorpe May 2, 2024
80dedb0
Moved error handling tests to MS0501Test
jonathan-r-thorpe May 3, 2024
36cec82
Add invoke method utility function.
jonathan-r-thorpe May 3, 2024
0e2f8f5
Do not cache spec_branch in construction
jonathan-r-thorpe May 3, 2024
ffc7ea0
Simplify utilities initialization
jonathan-r-thorpe May 3, 2024
1599e8d
Add reset so that state is reset on set_up_tests
jonathan-r-thorpe May 3, 2024
d5b5851
Initiailize IS-12 websocket on set_up_tests so failure is reported on…
jonathan-r-thorpe May 3, 2024
ddb1e7f
make base class generic
jonathan-r-thorpe May 3, 2024
ee57347
Updated comments
jonathan-r-thorpe May 3, 2024
9813fba
Changed " embedd3ed in f-strings to '
jonathan-r-thorpe May 8, 2024
e9aa570
Changed SET to PUT
jonathan-r-thorpe May 8, 2024
c3d58d9
Changed " embedded in f-strings to '
jonathan-r-thorpe May 8, 2024
37301ea
Fix IS-14 utility functions
jonathan-r-thorpe May 9, 2024
bad7453
Add missing sequence methods
jonathan-r-thorpe May 9, 2024
f9aa23a
Add role_path to get_sequence_length call
jonathan-r-thorpe May 9, 2024
5eda47f
Add device-configuration feature set to feature set tests
jonathan-r-thorpe Jun 3, 2024
e228800
Add additional validation of class descriptors
jonathan-r-thorpe Jun 3, 2024
80bcca4
Add is_block helper.
jonathan-r-thorpe Sep 9, 2024
013849d
Add initial IS-14 tests
jonathan-r-thorpe Sep 9, 2024
b32a401
Improve error reporting
jonathan-r-thorpe Sep 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions nmostesting/Config.py
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,37 @@
}
}
},
"is-12": {
"repo": "is-12",
"versions": ["v1.0"],
"default_version": "v1.0",
"apis": {
"ncp": {
"name": "Control Protocol"
},
}
},
"ms-05-02": {
"repo": "ms-05-02",
"versions": ["v1.0"],
"default_version": "v1.0",
"apis": {
"controlframework": {
"name": "Control Framework"
},
}
},
"nmos-control-feature-sets": {
"repo": "nmos-control-feature-sets",
"versions": ["main"],
"default_version": "main",
"apis": {
"featuresets": {
"name": "Control Feature Sets",
"repo_paths": ["identification", "monitoring"]
}
}
},
"bcp-002-01": {
"repo": "bcp-002-01",
"versions": ["v1.0"],
Expand Down
Loading