This has come up quite a lot recently, and it'd be good to have a central document that answers questions people have. I think it should cover:

• How to run the tests
• The different kinds of test
• How to update a test
• How to add a new test
• General guidelines on what makes a test "good", and when to prefer one kind or another

Perhaps it could also include instructions for running:

• The WPT suite
• The test262 suite
• The wasm suite

EDIT: We do have RunningTests.md which I'd forgotten about, which covers some of this.