PQ Show 81: Network Testing With ToDD

If SDN is about automation and orchestration to configure the network, then the unasked (and unanswered question) is how do we test it? And how do we automate the testing? Recently, Matt Oswalt announced ToDD (Testing On Demand: Distributed), an open source tool for validating network changes. As Matt notes on his GitHub page, if you want to define the success of change, “…the only way to do that is to put your network through the same paces that the real-world does, and get detailed information back on how that test performed.” Matt joins us to talk about test-driven automation, and to share details about the ToDD project. Also joining the conversation is Chris Young, a networking professional who’s worked with a preview of ToDD. Links: The Power of Test-Driven Network Automation – Keeping It Classless Network Programmability and Automation by Jason Edelman; Scott S. Lowe; Matt Oswalt An Introduction To ToDD – Youtube Matt Oswalt’s Github page kontrolissues.net – Chris Young’s Blog Show Notes: Part 1: Test-Driven Network Automation * What does success look like? * Automation without verification says to me that speed is more important than consistency and predictability – and that’s just not the case in the real world. Part 2: ToDD * Concepts * Agents – runs the tests * Groups – defines which agents run the tests * Testruns – defines the tests * Testlets – standardizes the tests * Post-Test Analytics * InfluxDB * Graphana Part 3: Future of ToDD and TDNA * Maturation of ToDD itself * Vendor platforms * Maturation of rest of pipeline * Automated baselines (machine learning?) * We need to get better about not doing manual, bespoke things to our networks, including verification of changes. Automation without verification is pointless (and dangerous)