The more I considered it, the messier it made the code, and didn't provide as many advantages as I was hoping. The Context has been merged into the Test constructor, as well as associated tests.
Major changes: panic client and server have been separated into two separate projects. This prevents a bunch of overhead if you're installing panic on each machine you're testing against. It also helps with separation of concerns and prevents assumptive tests.
Another major change: express was becoming complicated quick for what I needed. Instead of trying to re-implement websockets over http, I'm switching over to socket.io to allow for easier full duplex data streams.
Many of the patches weren't necessary when client code was removed (like Function.parse, Object.keys, Gun extensions, etc.)
Added module for the "this" context that test runners will need. The constructor handles timeout settings, "done" callbacks, "env" variables (and merges), as well as basic input validation. It'll be used during tests as an interface to the exterior controls, like "done" events and iterables/progress reporters.
Pass in an array of objects, with optional conditional expressions/functions, and get back a list of objects that passed the conditional and parsed properly as functions. It's in the format that the server test configuration constructor will send off to the client, so that instead of being terribly verbose, you can say `response.cbs = parse(response.cbs)` and get a validated list of test objects.
There were some race conditions involving the `bump` function, where some tests were failing because synchronous code was not acting as synchronous as it should (and I'm still not quite sure what's causing it). I've removed access to the bump function in the Test constructor (side note: none of the tests broke when I did, so apparently there was no coverage).
The client logic has been revised, as some of it was untested and wouldn't be useful for quite some time. I'm be starting with the context parser, then I'll be moving onto platform/ID filtering, then onto test runners/client context instances.
Created an event emitter instance for reporters and the test stack. Currently, the only event that fires is "begin".
All tests are now named, whether by their given name or as the name "Anonymous".
Each test now has a unique ID to reference by. This will be useful for reporters and tracking of "done" events. Also, each test now saves the name it's been given.
After discussing the goals for panic more with Mark and Sean, I decided to take panic in a different direction. Previously, the interface used an options object to do test setup, and was completely declarative in nature. In order to do more complex things with the framework, it would have needed new features hacked into the existing one, without a solid, well tested foundation to build on. That was a bit frightening. As radical as it was, I decided to mostly rewrite the framework to be similar to a traditional test framework (such as jasmine or mocha), with a major twist - it will send your tests out to the connected clients to be run, and you can filter which platforms/peer IDs to run on. Since the approach is more imperative, it will be more verbose, but it also allows for faster scaling, ease of adding new features, more expressive/flexible tests, greater coverage, and approachable syntax.
With luck, panic will be around for quite some time. If that's the case, it really, really needs high test coverage, so I began with jasmine from the start. In addition, who's going to trust results from an untested test framework?
Some major modifications are coming to panic. For one, it's been decided that there will be a test file that sends test instructions to a variable number of peers and performs checks across machines to ensure the data saved correctly. In order to relay the test instructions and have an easy, flexible test interface, it's been decided that eval will be used to send setup/test functions out to the peers (eval is horrible, we know. This is only for the test framework for browsers that opt into our stress-test DDOS network. I'm gonna stop talking now).
You can now boot up tests from the homepage using the UI. It's powered by jQuery.
The tests are now using the latest, greatest version of gun, v0.3.
Files now organized into folder "lib" for clarity and cleanliness.
Numerous code cleanliness improvements.
Now, if errors are passed to the acknowledgement listener, they're pushed to an array of errors on that request.
Now each time there's a response, the statistics are recalculated and are passed to an optional progress callback.
Patch.js now has stronger type checking against the data it's been given.
Added more polyfills to aid in development, like wrapping console.log in a closure to bypass the console interface error.
The timeout accuracy and code clarity has been improved.
Server is now upgraded to use gun@0.3, and the level options have been better tailored to the application.
New barrage of options now accepted (providing defaults in absence), such as a progress callback, a done callback and timeout configuration, a data generation function, etc...
Browser constructor meant to open new tabs, run the test (using the given options), and close the tab when finished. There is no loss of options by doing this (although it has had a notable effect on performance when the tab is not in focus).
Each progress event and the done event recieve the options object with statistics embeded in the "stats" property. It has details like the average latency, the total elapsed time, the acknowledged packets, the fastest/slowest times, etc...
Acknowledgments can fire more than once, and for now it dedupes those and only fires once. In the future I may change this to include every acknowledgment. For data!
The recursive test runner wasn't running the correct number of times (due to confusion and off-by-1 errors)... that has been fixed.
The test file checks to see if there were options globally declared. If so, run the test using those. In the future we may use selenium to boot up the browsers, and we'll need some way to kick off the test and provide options. That was a nice compromise.
I've included some useful polyfills (like interfacing with console.log so it can be passed naked to other functions, Object.keys polyfill for counting confirmed objects) and some extensions (like valMapDone).
Next steps include adding the gun instance to the options object, statistics for errors, node compatibility and the such.
Interval option built with a recursive setTimeout (not setInterval) that tracks it's invoked number.
Patch function now determines it's own ID if it isn't provided, meaning that the opts-must-be-object requirement isn't needed. In other words, all options are optional (wow).
Bug fix: minor thing where I was using gun before it was defined. This was due to minor tweaking just before a commit and not having unit tests to shout at me :(
Exposed a function named "test" that takes an options object, with things like how many gun requests to fire in total, what data to send, where to put it, what peers to connect to, etc...
Exposed a Browser constructor that opens up a new window and runs the test function with the options provided, returning an interface to the window.
Added a patch function to validate the option input and provide defaults where there aren't any values.
